8 × AMD Instinct MI355X 288GB HBM3e

Tested system

Processor: AMD EPYC 9655
Graphics: AMD Instinct MI355X 288GB HBM3e
Accelerator count: 8
Nodes: 1

Recorded results

Only the same benchmark, version, workload, scenario and unit share a row. Missing results are not zero. System results include the full submitted hardware and software configuration.

MLPerf Inference / gpt-oss-120bOffline · v6.1 · Tokens/s

8 × AMD Instinct MI355X 288GB HBM3e

117,804^* Tokens/s

Accuracy target / result: exact_match: 84.217
Weight precision: fp4
Division: closed
System type: datacenter
Availability: available
Cores per processor: 96
Processors per node: 2
Nodes: 1
Total accelerators: 8
Software stack: vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
Operating system: Ubuntu 24.04.4 LTS

MLPerf Inference / gpt-oss-120bServer · v6.1 · Tokens/s

8 × AMD Instinct MI355X 288GB HBM3e

110,188^* Tokens/s

Accuracy target / result: exact_match: 83.843
Weight precision: fp4
Division: closed
System type: datacenter
Availability: available
Cores per processor: 96
Processors per node: 2
Nodes: 1
Total accelerators: 8
Software stack: vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
Operating system: Ubuntu 24.04.4 LTS

MLPerf Inference / llama2-70b-99.9Offline · v6.1 · Tokens/s

8 × AMD Instinct MI355X 288GB HBM3e

103,792^* Tokens/s

Accuracy target / result: ROUGE1: 44.5322 ROUGE2: 22.1855 ROUGEL: 28.9228 TOKENS_PER_SAMPLE: 284.4
Weight precision: fp4
Division: closed
System type: datacenter
Availability: available
Cores per processor: 96
Processors per node: 2
Nodes: 1
Total accelerators: 8
Software stack: vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
Operating system: Ubuntu 24.04.4 LTS

MLPerf Inference / llama2-70b-99.9Server · v6.1 · Tokens/s

8 × AMD Instinct MI355X 288GB HBM3e

103,932^* Tokens/s

Accuracy target / result: ROUGE1: 44.5341 ROUGE2: 22.1716 ROUGEL: 28.9333 TOKENS_PER_SAMPLE: 284.0
Weight precision: fp4
Division: closed
System type: datacenter
Availability: available
Cores per processor: 96
Processors per node: 2
Nodes: 1
Total accelerators: 8
Software stack: vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
Operating system: Ubuntu 24.04.4 LTS

MLPerf Inference / llama3.1-8bOffline · v6.1 · Tokens/s

8 × AMD Instinct MI355X 288GB HBM3e

158,458^* Tokens/s

Accuracy target / result: ROUGE1: 38.5617 ROUGE2: 16.0263 ROUGEL: 24.4381 ROUGELSUM: 35.6994 GEN_LEN: 8151245
Weight precision: fp8
Division: closed
System type: datacenter
Availability: available
Cores per processor: 96
Processors per node: 2
Nodes: 1
Total accelerators: 8
Software stack: vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
Operating system: Ubuntu 24.04.4 LTS

MLPerf Inference / llama3.1-8bServer · v6.1 · Tokens/s

8 × AMD Instinct MI355X 288GB HBM3e

148,563^* Tokens/s

Accuracy target / result: ROUGE1: 38.5211 ROUGE2: 16.0095 ROUGEL: 24.4208 ROUGELSUM: 35.662 GEN_LEN: 8168331
Weight precision: fp8
Division: closed
System type: datacenter
Availability: available
Cores per processor: 96
Processors per node: 2
Nodes: 1
Total accelerators: 8
Software stack: vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
Operating system: Ubuntu 24.04.4 LTS

Processing & platform

Processor: AMD EPYC 9655
Graphics: AMD Instinct MI355X 288GB HBM3e

Connectivity & software

Operating system: Ubuntu 24.04.4 LTS

System configuration

Reported host core count: 96
Processors per node: 2
Accelerator count: 8
Nodes: 1
Software stack: vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2
Submission division: closed

* Provisional data