GeekRadar / The comparison desk
COMPARE.
Tested systemsNVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e · 16 total
Tested system
- Processor
- AMD EPYC
- Graphics
- NVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e
- Accelerator count
- 16
- Nodes
- 2
Recorded results
Only the same benchmark, version, workload, scenario and unit share a row. Missing results are not zero. System results include the full submitted hardware and software configuration.
MLPerf Inference / gpt-oss-120bOffline · v6.1 · Tokens/s
NVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e · 16 total
94,249.30* Tokens/s
- Accuracy target / result
- exact_match: 83.57105741751104
- Weight precision
- mxfp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 16
- Software stack
- vLLM
- Operating system
- Ubuntu 22.04
MLPerf Inference / gpt-oss-120bServer · v6.1 · Tokens/s
NVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e · 16 total
73,357.60* Tokens/s
- Accuracy target / result
- exact_match: 83.43465835281891
- Weight precision
- mxfp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 16
- Software stack
- vLLM
- Operating system
- Ubuntu 22.04
MLPerf Inference / llama2-70b-99.9Offline · v6.1 · Tokens/s
NVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e · 16 total
99,683.80* Tokens/s
- Accuracy target / result
- ROUGE1: 44.4169 ROUGE2: 22.068 ROUGEL: 28.7957 TOKENS_PER_SAMPLE: 286.6
- Weight precision
- fp8,fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 16
- Software stack
- vLLM
- Operating system
- Ubuntu 22.04
MLPerf Inference / llama2-70b-99.9Server · v6.1 · Tokens/s
NVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e · 16 total
95,104.60* Tokens/s
- Accuracy target / result
- ROUGE1: 44.6044 ROUGE2: 22.1926 ROUGEL: 28.9466 TOKENS_PER_SAMPLE: 284.7
- Weight precision
- fp8,fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 16
- Software stack
- vLLM
- Operating system
- Ubuntu 22.04
MLPerf Inference / llama3.1-8bOffline · v6.1 · Tokens/s
NVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e · 16 total
163,292* Tokens/s
- Accuracy target / result
- ROUGE1: 38.5675 ROUGE2: 16.0757 ROUGEL: 24.5063 ROUGELSUM: 35.5154 GEN_LEN: 8141814
- Weight precision
- fp8,fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 16
- Software stack
- vLLM
- Operating system
- Ubuntu 22.04
MLPerf Inference / llama3.1-8bServer · v6.1 · Tokens/s
NVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e · 16 total
161,110* Tokens/s
- Accuracy target / result
- ROUGE1: 38.6437 ROUGE2: 16.1045 ROUGEL: 24.5278 ROUGELSUM: 35.5601 GEN_LEN: 8147699
- Weight precision
- fp8,fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 2
- Total accelerators
- 16
- Software stack
- vLLM
- Operating system
- Ubuntu 22.04
Processing & platform
- Processor
- AMD EPYC
- Graphics
- NVIDIA H200-SXM-141GB + AMD Instinct MI350X 288GB HBM3e
Connectivity & software
- Operating system
- Ubuntu 22.04
System configuration
- Reported host core count
- 128
- Processors per node
- 2
- Accelerator count
- 16
- Nodes
- 2
- Software stack
- vLLM
- Submission division
- closed
0 selected Compare 0
* Provisional data