NVIDIA "Vera Rubin" makes its MLPerf debut: up to 3.7x the throughput of GB300

Three numbers inside a preview submission

On September 16 the MLPerf Inference v6.1 results were published, and NVIDIA's next-generation rack-scale platform, Vera Rubin NVL72, appeared on the list for the first time — as a preview submission. In its official blog NVIDIA drew a blunt conclusion: the system delivers up to 3.7x the throughput of the previous-generation GB300 NVL72 on Qwen3-VL, and up to 2.5x on DeepSeek-R1. MLCommons' inference benchmarks have long been the public arena where accelerators are compared head to head, so a debut that already leads effectively shows customers the cadence of the next-generation platform ahead of time.

It helps to be precise about what "preview submission" means. NVIDIA entered only two models this time — DeepSeek-R1 and Qwen3-VL — and described the results as a preview, taken from entries published on mlcommons.org. The submission does not cover the full MLPerf inference suite, so 3.7x and 2.5x are measured numbers, but they describe a set of scenarios rather than an entire report card; NVIDIA itself added continued gains from software optimization to its conclusions. The chart footnotes where the numbers came from: results retrieved on September 16 from mlcommons.org, entries 6.1-0106 and 6.1-0074.

The gap is not uniform across scenarios. NVIDIA's comparison table shows that at the same 72-GPU scale, DeepSeek-R1 reached 1.7x, 1.9x and 2.5x GB300 NVL72 in the offline, server and interactive scenarios respectively, while Qwen3-VL reached 1.8x, 1.9x and 3.7x. Offline measures the ceiling on throughput, interactive measures responsiveness — and the more latency-sensitive the scenario, the larger the new platform's relative advantage. In the same table the largest multiple for both models appears in the interactive scenario and the smallest in the offline scenario, a distribution that itself suggests the new platform cashes in its advantage first on latency-sensitive work.

What stands behind those multiples

The two models also ran different software stacks: Qwen3-VL used vLLM with NVIDIA's open-source Dynamo inference framework, while DeepSeek-R1 used TensorRT-LLM. NVIDIA attributes the results to hardware-software codesign: enhanced Tensor Cores and Transformer Engine accelerate both the prefill and decode stages, while a low-precision format such as NVFP4 shrinks the memory footprint of model weights, attention and KV cache, raising throughput without meaningfully hurting output quality.

Another technique comes up repeatedly: disaggregated serving, which separates prefill and decode onto different resources, combined with large-scale expert parallelism across mixture-of-experts (MoE) layers. Both DeepSeek-R1 and Qwen3-VL are MoE architectures, and how well expert parallelism works directly determines whether such models can actually keep a whole machine busy.

In the same blog NVIDIA does the arithmetic on inference: three levers decide AI inference economics — system performance, scaling efficiency and the pace of software iteration. Higher performance means more tokens per machine, hence more revenue; better scaling efficiency means adding hardware buys proportional throughput, so serving the same users takes fewer resources; continuous optimization means the same hardware keeps gaining value over its lifetime. Those levers map onto three concrete questions: is this machine worth buying, do ten more scale linearly, and will the same fleet still be enough next year.

Platform fungibility sits underneath all three: the same infrastructure must run training and inference, recommenders and reasoning models, language and video, and only when it can switch freely between those workloads does utilization stay high. That is also why NVIDIA refreshes a platform architecture every year — once the hardware cadence is fixed, customers dare to plan capital spending on an annual basis.

From 72 GPUs to 288: 99% scaling efficiency

Beyond the raw performance of the new platform, one number in this round is easy to overlook: on the offline DeepSeek-R1 scenario, GB300 NVL72 went from a single rack of 72 GPUs to four racks of 288 GPUs with 99% scaling efficiency. Hardware quadrupled, and throughput grew almost linearly.

Why does that matter? Because stacking GPUs does not automatically deliver proportional throughput. If doubling the number of GPUs yields only a single-digit percentage gain, infrastructure cost runs far ahead of the performance return. Near-linear scaling requires the rack's high-speed scale-up interconnect, the network between racks and request orchestration to keep pace together. NVIDIA's answer is sixth-generation NVLink and NVLink Switch: the company says packet rates are 10x higher and latency a third of that of off-the-shelf Ethernet. NVIDIA also notes that cloud provider Nebius submitted Vera Rubin NVL72 preview results with strong performance, a sign the platform is already being tested outside the company.

Video generation appears in this round too. On the WAN 2.2 text-to-video benchmark, GB300 NVL72 delivered 0.65 720p videos per second at 5.7 seconds per video — 9x the throughput and 7.5x lower latency than a single node.

A software clock, a hardware cadence

How much faster can the same hardware get within a year? NVIDIA offers a comparable answer: in the v6.1 round, GB300 NVL72 improved up to 1.6x over v6.0 on Qwen3-VL, thanks to lower KV cache precision, more kernel fusion, better kernels and disaggregated serving with vLLM and Dynamo.

Optimization continued after the submission deadline. NVIDIA also disclosed post-submission results that have not been verified by MLCommons, covering GPT-OSS-120B and DLRMv3, and said further gains followed. Such numbers deserve caution — there is no third-party audit — but they still point to a trend: on this product line, a software release can be worth about as much as a hardware generation.

The edge got an update too. NVIDIA submitted Jetson AGX Thor results using TensorRT Edge-LLM on the newly introduced Edge-Agentic benchmark with Qwen3.6-27B. Nineteen partners took part in this round, eight of them on multi-node Blackwell NVL72 systems, including ASUS, Azure, Cisco, CoreWeave, Dell, Fujitsu, HPE, Lambda, Nebius, Oracle Cloud Infrastructure and Supermicro.

One more figure is worth noting: in preview testing on SemiAnalysis' AgentX benchmark, NVIDIA says Vera Rubin NVL72 delivered 30x the performance of GB300 NVL72. That number is vendor-run, but it points at something real — once AI moves from question-and-answer to multi-step reasoning, planning and acting, traditional throughput benchmarks stop being enough, and MLPerf will introduce an Endpoints benchmark to standardize measurement for such workloads. For buyers, only one criterion really matters: on their own models and their own workloads, how far cost per token falls.