GeekRadar / The comparison desk
COMPARE.
Tested systems8 × NVIDIA GB300
Tested system
- Processor
- NVIDIA Grace CPU
- Graphics
- NVIDIA GB300
- Accelerator count
- 8
- Nodes
- 2
Recorded results
Only the same benchmark, version, workload, scenario and unit share a row. Missing results are not zero. System results include the full submitted hardware and software configuration.
MLPerf Inference / deepseek-r1Offline · v6.1 · Tokens/s
8 × NVIDIA GB300
77,031.70* Tokens/s
- Accuracy target / result
- exact_match: 81.42661804922516 TOKENS_PER_SAMPLE: 3803.590474020055
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 144
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 8
- Software stack
- TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0
- Operating system
- Ubuntu 24.04
MLPerf Inference / deepseek-r1Server · v6.1 · Tokens/s
8 × NVIDIA GB300
68,138* Tokens/s
- Accuracy target / result
- exact_match: 80.9024612579763 TOKENS_PER_SAMPLE: 3861.967183226983
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 144
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 8
- Software stack
- TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0
- Operating system
- Ubuntu 24.04
MLPerf Inference / llama2-70b-99.9Offline · v6.1 · Tokens/s
8 × NVIDIA GB300
132,253* Tokens/s
- Accuracy target / result
- ROUGE1: 44.8005 ROUGE2: 22.3791 ROUGEL: 29.1527 TOKENS_PER_SAMPLE: 272.6
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 144
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 8
- Software stack
- TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0
- Operating system
- Ubuntu 24.04
MLPerf Inference / llama2-70b-99.9Server · v6.1 · Tokens/s
8 × NVIDIA GB300
127,984* Tokens/s
- Accuracy target / result
- ROUGE1: 44.8018 ROUGE2: 22.3668 ROUGEL: 29.141 TOKENS_PER_SAMPLE: 272.9
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 144
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 8
- Software stack
- TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0
- Operating system
- Ubuntu 24.04
Processing & platform
- Processor
- NVIDIA Grace CPU
- Graphics
- NVIDIA GB300
Connectivity & software
- Operating system
- Ubuntu 24.04
System configuration
- Reported host core count
- 144
- Processors per node
- 2
- Accelerator count
- 8
- Nodes
- 2
- Software stack
- TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0
- Submission division
- closed
0 selected Compare 0
* Provisional data