COMPARE.

NVIDIA

NVIDIA H200 SXM 141 GB

GPU^*

Video memory

141 GB^*

Rated memory bandwidth

4,890 GB/s^*

Thermal design power

700 W^*

GPU cores

16,896^*

LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score#1^* · 2,444.34^* tokens/sPlatform unreported

No Blender rendering score

AMD

Radeon Instinct MI325X

GPU^*

Video memory

Reported value 1: 288 GB^*

Reported value 2: 256 GB^*

Reported value 3: 256 GB^*

Rated memory bandwidth

6,140 GB/s^*

Thermal design power

1,000 W^*

GPU cores

19,456^*

LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score#2^* · 1,945.78^* tokens/sPlatform unreported

No Blender rendering score

NVIDIA

NVIDIA H100 SXM5 94 GB

GPU^*

Video memory

94 GB^*

Rated memory bandwidth

3,360 GB/s^*

Thermal design power

700 W^*

GPU cores

16,896^*

LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score#3^* · 1,668.90^* tokens/sPlatform unreported

No Blender rendering score

NVIDIA

NVIDIA H100 PCIe 80 GB

GPU^*

Video memory

80 GB^*

Rated memory bandwidth

2,040 GB/s^*

Thermal design power

350 W^*

GPU cores

14,592^*

LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score#4^* · 1,222.48^* tokens/sPlatform unreported

No Blender rendering score

AMD

Radeon Instinct MI300X

GPU^*

Video memory

192 GB^*

Rated memory bandwidth

5,320 GB/s^*

Thermal design power

750 W^*

GPU cores

19,456^*

LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score#5^* · 1,056.89^* tokens/sPlatform unreported

No Blender rendering score

NVIDIA

NVIDIA H100 SXM5 96 GB

GPU^*

Video memory

96 GB^*

Rated memory bandwidth

3,360 GB/s^*

Thermal design power

700 W^*

GPU cores

16,896^*

LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score#6^* · 370.58^* tokens/sPlatform unreported

No Blender rendering score