GeekRadar / The comparison desk
COMPARE.
Tested systems8 × AMD Instinct MI355X 288GB HBM3e
Tested system
- Processor
- AMD EPYC 9755
- Graphics
- AMD Instinct MI355X 288GB HBM3e
- Accelerator count
- 8
- Nodes
- 1
Recorded results
Only the same benchmark, version, workload, scenario and unit share a row. Missing results are not zero. System results include the full submitted hardware and software configuration.
MLPerf Inference / gpt-oss-120bOffline · v6.1 · Tokens/s
8 × AMD Instinct MI355X 288GB HBM3e
118,042* Tokens/s
- Accuracy target / result
- exact_match: 83.861
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0+git8514f05, ROCm 7.2.53211
- Operating system
- Ubuntu 24.04.4 LTS
MLPerf Inference / gpt-oss-120bServer · v6.1 · Tokens/s
8 × AMD Instinct MI355X 288GB HBM3e
108,861* Tokens/s
- Accuracy target / result
- exact_match: 84.039
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0+git8514f05, ROCm 7.2.53211
- Operating system
- Ubuntu 24.04.4 LTS
MLPerf Inference / llama2-70b-99.9Offline · v6.1 · Tokens/s
8 × AMD Instinct MI355X 288GB HBM3e
104,904* Tokens/s
- Accuracy target / result
- ROUGE1: 44.5341 ROUGE2: 22.1837 ROUGEL: 28.9253 TOKENS_PER_SAMPLE: 284.3
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0+git8514f05, ROCm 7.2.53211
- Operating system
- Ubuntu 24.04.4 LTS
MLPerf Inference / llama2-70b-99.9Server · v6.1 · Tokens/s
8 × AMD Instinct MI355X 288GB HBM3e
104,541* Tokens/s
- Accuracy target / result
- ROUGE1: 44.5342 ROUGE2: 22.1734 ROUGEL: 28.9337 TOKENS_PER_SAMPLE: 284.0
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0+git8514f05, ROCm 7.2.53211
- Operating system
- Ubuntu 24.04.4 LTS
MLPerf Inference / llama3.1-8bOffline · v6.1 · Tokens/s
8 × AMD Instinct MI355X 288GB HBM3e
159,278* Tokens/s
- Accuracy target / result
- ROUGE1: 38.5554 ROUGE2: 16.0227 ROUGEL: 24.4331 ROUGELSUM: 35.6963 GEN_LEN: 8150276
- Weight precision
- fp8
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0+git8514f05, ROCm 7.2.53211
- Operating system
- Ubuntu 24.04.4 LTS
MLPerf Inference / llama3.1-8bServer · v6.1 · Tokens/s
8 × AMD Instinct MI355X 288GB HBM3e
151,116* Tokens/s
- Accuracy target / result
- ROUGE1: 38.5121 ROUGE2: 15.9993 ROUGEL: 24.4128 ROUGELSUM: 35.6518 GEN_LEN: 8170944
- Weight precision
- fp8
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 128
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0+git8514f05, ROCm 7.2.53211
- Operating system
- Ubuntu 24.04.4 LTS
Processing & platform
- Processor
- AMD EPYC 9755
- Graphics
- AMD Instinct MI355X 288GB HBM3e
Connectivity & software
- Operating system
- Ubuntu 24.04.4 LTS
System configuration
- Reported host core count
- 128
- Processors per node
- 2
- Accelerator count
- 8
- Nodes
- 1
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0+git8514f05, ROCm 7.2.53211
- Submission division
- closed
0 selected Compare 0
* Provisional data