MLPerf Inference v6.1 Released: AMD Sets Record with 512 MI355X GPUs, NVIDIA Vera Rubin Debuts

On September 16, MLCommons released the MLPerf Inference v6.1 benchmarking suite, with AMD and NVIDIA submitting results simultaneously. The benchmark suite measures throughput and latency of different hardware in AI inference tasks. The new version focuses on inference throughput, scaling efficiency, and performance across different GPU configurations.

AMD participated in the DeepSeek R1 test with 512 Instinct MI355X GPUs, achieving 2,901,950 tokens/s in offline mode and 2,405,310 tokens/s in server mode. AMD said this was its largest MLPerf submission to date and set a new record. NVIDIA submitted configurations with 288 GPUs for both GB300 and GB200. The GB300 scored 2,705,130 tokens/s offline and 2,028,030 tokens/s in server mode.

NVIDIA's Vera Rubin VR200 also appeared in the MLPerf inference benchmark for the first time, with 36-GPU and 72-GPU configurations. With the same 72-GPU setup, Vera Rubin (VR200) was 95% faster than GB300, and the 36-GPU configuration was faster than the 72-GPU GB200 configuration. Since GPU configurations vary across vendors, the results are for reference only.