GeekRadar / The comparison desk
COMPARE.
Tested systems8 × AMD Instinct MI350P
Tested system
- Processor
- AMD EPYC 9455
- Graphics
- AMD Instinct MI350P
- Accelerator count
- 8
- Nodes
- 1
Recorded results
Only the same benchmark, version, workload, scenario and unit share a row. Missing results are not zero. System results include the full submitted hardware and software configuration.
MLPerf Inference / dlrm-v3Offline · v6.1 · Samples/s
8 × AMD Instinct MI350P
4,837.63* Samples/s
- Accuracy target / result
- DLRM_NE: 0.8677279311862482 DLRM_ACC: 0.6964196251791978 DLRM_AUC: 0.7862870079033047
- Weight precision
- int8,bf16
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / dlrm-v3Server · v6.1 · Queries/s
8 × AMD Instinct MI350P
4,503.53* Queries/s
- Accuracy target / result
- DLRM_NE: 0.8677277210685596 DLRM_ACC: 0.6964198708391458 DLRM_AUC: 0.7862873761281074
- Weight precision
- int8,bf16
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / gpt-oss-120bOffline · v6.1 · Tokens/s
8 × AMD Instinct MI350P
48,971.10* Tokens/s
- Accuracy target / result
- exact_match: 83.287
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / gpt-oss-120bServer · v6.1 · Tokens/s
8 × AMD Instinct MI350P
40,867.90* Tokens/s
- Accuracy target / result
- exact_match: 83.631
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / llama2-70b-99.9Offline · v6.1 · Tokens/s
8 × AMD Instinct MI350P
42,286.20* Tokens/s
- Accuracy target / result
- ROUGE1: 44.5012 ROUGE2: 22.1742 ROUGEL: 28.9465 TOKENS_PER_SAMPLE: 284.2
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / llama2-70b-99.9Server · v6.1 · Tokens/s
8 × AMD Instinct MI350P
41,830* Tokens/s
- Accuracy target / result
- ROUGE1: 44.4477 ROUGE2: 22.0889 ROUGEL: 28.8674 TOKENS_PER_SAMPLE: 287.4
- Weight precision
- fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / llama3.1-8bOffline · v6.1 · Tokens/s
8 × AMD Instinct MI350P
73,643* Tokens/s
- Accuracy target / result
- ROUGE1: 38.5604 ROUGE2: 16.0125 ROUGEL: 24.4415 ROUGELSUM: 35.6907 GEN_LEN: 8149596
- Weight precision
- fp8
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / llama3.1-8bServer · v6.1 · Tokens/s
8 × AMD Instinct MI350P
72,305.10* Tokens/s
- Accuracy target / result
- ROUGE1: 38.5633 ROUGE2: 16.0282 ROUGEL: 24.4369 ROUGELSUM: 35.7015 GEN_LEN: 8151565
- Weight precision
- fp8
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / wan-2.2-t2v-a14bOffline · v6.1 · Samples/s
8 × AMD Instinct MI350P
0.03* Samples/s
- Accuracy target / result
- vbench_score: 69.9361
- Weight precision
- bf16, fp8, fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
MLPerf Inference / wan-2.2-t2v-a14bSingleStream · v6.1 · Latency (s)
8 × AMD Instinct MI350P
41.84* Latency (s)
- Accuracy target / result
- vbench_score: 70.3648
- Weight precision
- bf16, fp8, fp4
- Division
- closed
- System type
- datacenter
- Availability
- available
- Cores per processor
- 48
- Processors per node
- 2
- Nodes
- 1
- Total accelerators
- 8
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
Processing & platform
- Processor
- AMD EPYC 9455
- Graphics
- AMD Instinct MI350P
Connectivity & software
- Operating system
- Ubuntu 24.04.4 LTS (kernel 6.8.0-136-generic)
System configuration
- Reported host core count
- 48
- Processors per node
- 2
- Accelerator count
- 8
- Nodes
- 1
- Software stack
- vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
- Submission division
- closed
0 selected Compare 0
* Provisional data