NVIDIA’s Vera Rubin NVL72 posted a strong first MLPerf Inference v6.1 showing. The useful question is not whether the headline is exciting, but which model, scenario, system size and software stack produced it.
The short answer
The “up to 3.7x” figure is a narrow, scenario-specific comparison, not a universal Vera Rubin speed-up. In the MLCommons v6.1 preview rows, a 72-GPU Vera Rubin NVL72 delivered 1,306.6 queries per second for Qwen3-VL 235B in the interactive scenario, versus 349.3 queries per second for a 72-GPU NVIDIA GB300 system. That is about 3.74x on that model and scenario.
The same-sized rows show a wider range. DeepSeek R1 is about 1.74x faster offline, 1.97x faster in server, and 2.58x faster interactively. Qwen3-VL is about 1.83x faster offline, 1.92x faster in server, and 3.74x faster interactively. Vera Rubin’s rows are marked preview, the systems use different generation-specific software and precision settings, and the raw result summary does not provide measured power for these entries. Those limits mean the benchmark does not by itself prove a 3.7x reduction in cost per token, energy use, latency, or production availability.
Reader takeaway: treat the result as a signal to run a workload-matched evaluation. If your workload resembles interactive Qwen3-VL serving, the result is relevant. If it is batch DeepSeek inference, use the offline row instead; if you need economics, collect power, price and utilization evidence separately.
What NVIDIA announced
In its September 16, 2026 announcement, NVIDIA described its first Vera Rubin NVL72 submission in MLPerf Inference v6.1. The post emphasizes the connection between throughput, scaling and inference economics. It reports up to 3.7x higher throughput than GB300 NVL72 for Qwen3-VL and up to 2.5x for DeepSeek-R1, with the models run through NVIDIA’s inference software stack.
NVIDIA also reports software improvements of up to 1.6x over its MLPerf v6.0 submission. That is a vendor-reported optimization claim: it is useful context, but it should not be read as a hardware-only gain. The MLPerf result is a system result involving the accelerator, host, interconnect, model implementation, runtime, precision and tuning.
The announcement mentions additional post-submission GPT-OSS-120B and DLRMv3 optimization results. Those are not the basis of the comparison below because they are not the MLCommons-listed rows used for the Vera Rubin-versus-GB300 figures.
What the MLCommons rows actually show
For an apples-to-apples reading, the comparison below uses the same 72-GPU NVIDIA system size on both sides. The figures are transcribed from the MLCommons v6.1 summary data for NVIDIA Vera Rubin VR200 NVL72 and NVIDIA GB300 NVL72 entries. MLCommons’ datacenter benchmark documentation explains the closed division, scenarios and availability labels.
| Model | Scenario | Vera Rubin NVL72 | GB300 NVL72 | Rubin / GB300 |
|---|---|---|---|---|
| DeepSeek R1 | Interactive | 652,750.1 tokens/s | 253,506.0 tokens/s | 2.58x |
| DeepSeek R1 | Offline | 1,183,326.9 tokens/s | 679,740.0 tokens/s | 1.74x |
| DeepSeek R1 | Server | 1,175,890.2 tokens/s | 596,944.0 tokens/s | 1.97x |
| Qwen3-VL 235B | Interactive | 1,306.6 queries/s | 349.3 queries/s | 3.74x |
| Qwen3-VL 235B | Offline | 2,392.7 samples/s | 1,305.0 samples/s | 1.83x |
| Qwen3-VL 235B | Server | 2,323.3 queries/s | 1,210.5 queries/s | 1.92x |
The units are intentionally preserved. A query per second in an interactive test is not interchangeable with an offline sample per second, and neither is a direct latency or cost metric. “Up to” refers to the best row in the selected comparison, not to every model or operating mode.
How to read the 3.7x headline
- It is the interactive Qwen3-VL row. The maximum in this same-size comparison is approximately 3.74x, which rounds to NVIDIA’s “up to 3.7x” statement.
- It is not the DeepSeek result. DeepSeek R1’s interactive ratio is approximately 2.58x, with lower ratios in offline and server scenarios.
- It is not a blanket model multiplier. The scenario changes the workload shape, service constraints and useful unit. Compare like with like before making a procurement claim.
- It is not a cost claim. Throughput alone does not reveal electricity, cloud rental, host cost, utilization, cooling, software licensing or the cost of meeting a quality target.
There is another easy comparison trap. The NVIDIA performance hub also presents larger GB300 submissions, including a 288-GPU result. A 72-GPU Vera Rubin system should not be compared to a 288-GPU GB300 system as if they were the same deployment size. The table above keeps the accelerator count constant.
Configuration boundaries that matter
The raw MLCommons entries describe Vera Rubin as a 72-accelerator, 18-node NVL72 configuration, with four VR200 accelerators per node and two Vera CPUs per node. The entries list 2,300 W TGP per GPU, TRTLLM-rubin for DeepSeek, and a vLLM/CentML plus NVIDIA Dynamo path for Qwen3-VL. The DeepSeek entry uses FP4 weights; the Qwen3-VL entry uses BF16. The comparison GB300 entries are also 72 accelerators across 18 nodes, but use a GB300-generation TensorRT/CUDA/TensorRT-LLM stack with NVIDIA Dynamo and vLLM/CentML.
NVIDIA and secondary analysis describe the Vera Rubin NVL72 as a liquid-cooled rack-scale system and report 288 GB of HBM4 per VR200 accelerator. That configuration context is useful, but it does not turn a benchmark row into a representative bill of materials or operating-cost model. The MLCommons summary marks the Vera Rubin rows as preview and does not include measured power for these entries.
Accuracy targets and precision also need to stay in the discussion. The NVIDIA hub lists the relevant closed-division accuracy target and model-specific reference behavior. A faster quantized path is only useful if its quality and service-level requirements remain acceptable for the intended workload.
What changes for an AI infrastructure decision
For an operator, the announcement is a reason to narrow a test plan, not a reason to copy a headline into a capacity model. Start with the traffic shape and model you actually serve.
- Interactive assistant or agent: measure time to first token, inter-token latency, tail latency and concurrency. The Qwen interactive result is the closest benchmark row, but it is not a substitute for your prompts and quality checks.
- Batch or offline pipeline: use the offline row and your real batch size. The same hardware can look materially different when the scenario changes.
- Multi-tenant serving: test admission control, queueing and isolation. Aggregate queries per second can hide a poor experience for the slowest users.
- Cost or sustainability case: add measured power, facility overhead, utilization, hardware pricing and contract terms. The v6.1 summary alone cannot answer those questions.
- Deployment timing: confirm that the preview result has become an available, supportable configuration and that the required software path is ready for your environment.
Why scenario labels are part of the result
MLPerf’s scenario names are not cosmetic labels. Offline tests model a queue of work that can be processed without an individual-user service constraint. Server tests add arrival patterns and latency requirements. Interactive tests represent a conversational serving shape where users experience the system as a service. A single throughput number therefore cannot be moved from one scenario into another without changing the question being answered.
The model row also matters. DeepSeek R1 and Qwen3-VL 235B exercise different architectures and token or sample behavior. The Qwen rows use queries or samples per second, while the DeepSeek rows use tokens per second. Even within one unit, the amount of generated work and the quality target can affect a practical deployment. A buyer should keep the model, input/output mix, accuracy target and concurrency fixed when reproducing the comparison.
Finally, an MLPerf submission is a tuned implementation. The runtime, kernel choices, compiler, communication path, batching policy and model precision all participate in the observed result. That makes the result valuable for a reproducible system comparison, while also making it unsafe to describe the number as a standalone property of the VR200 silicon. The right follow-up is to reproduce the stack where possible and then test the production workload that the benchmark only approximates.
A simple decision rule
Use the Vera Rubin result as a screening signal when it aligns with your model and scenario, as a test-plan input when the configuration is available, and as a market signal when the result is still preview-only. Do not use it as a signed-off TCO case until power, utilization, pricing, quality and availability have been measured for the proposed deployment.
How the result looks outside NVIDIA’s announcement
The announcement is not the only source of context. MLCommons’ v6.1 release describes a broader benchmark cycle with new RAG and edge agentic tests and warns readers to interpret results through the benchmark architecture and submission rules. Nebius’ report shows a separate 36-GPU VR200 preview submission, which is useful for understanding that system size and per-accelerator normalization matter; its configuration is not directly rankable against NVIDIA’s 72-GPU rows. StorageReview’s coverage adds market context around the first Vera Rubin numbers without replacing the official result data.
DIYAI’s analysis independently recomputes the ratios from the MLCommons values and makes the same important distinction: the 3.7x maximum is not a universal upgrade and the published rows do not establish lower production cost per token. That is a useful cross-check, not a new benchmark submission.
A practical evaluation checklist
- Choose the model and scenario that match your traffic: interactive, server or offline.
- Compare the same accelerator count, division and availability status.
- Record precision, model version, accuracy target, runtime, compiler and serving framework.
- Collect p50 and tail latency, not only aggregate throughput.
- Measure power and utilization, then add your hardware or cloud price assumptions.
- Replay representative prompts and check output quality, not only benchmark conformance.
- Re-check the MLCommons row and software release before a purchase or capacity decision because preview results can change.
How this fits DMT’s existing NVIDIA coverage
DMT already has an NVIDIA-adjacent article on Pinterest and NVIDIA’s multimodal AI foundation. That page explains the distinct visual-search, advertising and multimodal partnership topic. This article instead serves readers evaluating inference infrastructure and benchmark evidence.
For a practical contrast between rack-scale inference and constrained local hardware, see DMT’s low-VRAM local LLM guide. The comparison adds context without suggesting that a local setup is an equivalent benchmark competitor.
FAQ
Is Vera Rubin 3.7x faster than GB300?
Only for the specific Qwen3-VL interactive comparison shown here: 1,306.6 versus 349.3 queries per second on 72-GPU NVIDIA systems. Other models and scenarios produce different ratios.
Are these production-ready results?
The Vera Rubin rows are marked preview in the MLCommons v6.1 summary. Confirm availability, support and software maturity before treating them as a production capacity baseline.
Does the result prove lower cost per token?
No. The result is a throughput comparison. A cost or energy conclusion requires measured power, utilization, pricing, cooling and quality evidence.
Why compare 72 GPUs instead of the biggest result?
Keeping the accelerator count constant avoids confusing a larger deployment with a faster accelerator generation. Larger GB300 submissions are useful scale context but are not an apples-to-apples replacement for a 72-GPU row.
What should a buyer test next?
Run the target model and traffic shape on the candidate stack, collect throughput and tail latency, verify accuracy, and build a power-and-price model. Use MLPerf as a reproducible starting point, not the whole business case.
Sources
- NVIDIA: Vera Rubin NVL72 MLPerf Inference v6.1 announcement
- NVIDIA: AI inference performance hub and result tables
- MLCommons: MLPerf Inference v6.1 results
- MLCommons: v6.1 raw summary data
- MLCommons: Inference Datacenter benchmark documentation
- Nebius: MLPerf Inference v6.1 results
- DIYAI: Vera Rubin NVL72 ratio analysis
- StorageReview: v6.1 market context
Accessed September 17, 2026. Vendor claims are labeled as vendor-reported; MLCommons availability and configuration limits are preserved.
