GeekRadar / The comparison desk

COMPARE.

GPU

NVIDIA

NVIDIA H200 SXM 141 GB

Graphics processor

Video memory: 141 GB^*
Memory bus: 6,144 bit^*
Rated memory bandwidth: 4,890 GB/s^*

Benchmark results

LLM Inference: MLPerf: deepseek-r1 (fp8) · version unreported · score1,113.10^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: gptj-99 (fp8) · version unreported · score2,562.73^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: gptj-99.9 (fp8) · version unreported · score2,562.73^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score2,444.34^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-99.9 (fp16) · version unreported · score3,552.63^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-99.9 (fp8) · version unreported · score2,444.34^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-99.9 (UNSET) · version unreported · score3,534.05^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-interactive-99 (fp8) · version unreported · score2,422.63^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-interactive-99.9 (fp16) · version unreported · score1,995.01^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-interactive-99.9 (fp8) · version unreported · score1,524.05^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama3.1-405b (fp16) · version unreported · score40.82^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama3.1-405b (fp8) · version unreported · score25.35^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama3.1-8b (fp8) · version unreported · score5,161.38^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: mixtral-8x7b (fp8) · version unreported · score7,131.80^* tokens/sPlatform unreported Compare results

Processing & platform

Architecture: Hopper^*
Process node: 5 nm^*
GPU cores: 16,896^*
Graphics interface: PCIe 5.0 x16^*
Reported chip die area: 814 mm²^*
Transistors: 80,000 million^*
GPU L2 cache: 60 MB^*
Graphics processor: GH100^*
Texture mapping units: 528^*
Raster output units: 24^*
Streaming multiprocessors: 132^*
Tensor cores: 528^*
Chip foundry: TSMC^*
Card or module format: SXM Module^*
Declared GPU TDP: 700 W^*
Suggested system PSU: 1,100 W^*
External power connectors: 8-pin EPS^*
OpenCL: 3.0^*
CUDA compute capability: 9.0^*

Memory & storage

Memory options: 141^*
Video memory: 141 GB^*
Video memory type: HBM3e^*
Memory bus: 6,144 bit^*
Rated memory bandwidth: 4,890 GB/s^*

Power specifications

Thermal design power: 700 W^*

System configuration

Detailed specifications

Specifications 1 · 26 fields

GPU architecture — NVIDIA H200 SXM — 141 GB HBM3e

Hopper^*

Calculated theoretical peak: BF16 tensor dense per GPU — NVIDIA H200 SXM — 141 GB HBM3e

989.5^* TFLOPS

Scope and conditions

Calculation: 1979 / 2^*

Theoretical peak: BF16 tensor with sparsity per GPU — NVIDIA H200 SXM — 141 GB HBM3e

1,979^* TFLOPS

Calculated theoretical peak: FP16 tensor dense per GPU — NVIDIA H200 SXM — 141 GB HBM3e

989.5^* TFLOPS

Scope and conditions

Calculation: 1979 / 2^*

Theoretical peak: FP16 tensor with sparsity per GPU — NVIDIA H200 SXM — 141 GB HBM3e

1,979^* TFLOPS

Theoretical peak: FP32 non-tensor dense per GPU — NVIDIA H200 SXM — 141 GB HBM3e

67^* TFLOPS

Native hardware support: FP4 tensor — NVIDIA H200 SXM — 141 GB HBM3e

No^*

Theoretical peak: FP64 non-tensor dense per GPU — NVIDIA H200 SXM — 141 GB HBM3e

34^* TFLOPS

Theoretical peak: FP64 tensor dense per GPU — NVIDIA H200 SXM — 141 GB HBM3e

67^* TFLOPS

Native hardware support: FP6 tensor — NVIDIA H200 SXM — 141 GB HBM3e

No^*

Calculated theoretical peak: FP8 tensor dense per GPU — NVIDIA H200 SXM — 141 GB HBM3e

1,979^* TFLOPS

Scope and conditions

Calculation: 3958 / 2^*

Theoretical peak: FP8 tensor with sparsity per GPU — NVIDIA H200 SXM — 141 GB HBM3e

3,958^* TFLOPS

Calculated theoretical peak: INT8 tensor dense per GPU — NVIDIA H200 SXM — 141 GB HBM3e

1,979^* TOPS

Scope and conditions

Calculation: 3958 / 2^*

Theoretical peak: INT8 tensor with sparsity per GPU — NVIDIA H200 SXM — 141 GB HBM3e

3,958^* TOPS

Calculated theoretical peak: TF32 tensor dense per GPU — NVIDIA H200 SXM — 141 GB HBM3e

494.5^* TFLOPS

Scope and conditions

Calculation: 989 / 2^*

Theoretical peak: TF32 tensor with sparsity per GPU — NVIDIA H200 SXM — 141 GB HBM3e

989^* TFLOPS

Physical configuration — NVIDIA H200 SXM — 141 GB HBM3e

SXM^*

GPU interconnect bandwidth (NVLink 4) bidirectional — NVIDIA H200 SXM — 141 GB HBM3e

900^* GB/s

GPU interconnect bandwidth per direction — NVIDIA H200 SXM — 141 GB HBM3e

450^* GB/s

Scope and conditions

Calculation: 900 / 2^*

PCIe interface declaration — NVIDIA H200 SXM — 141 GB HBM3e

PCIe Gen5: 128GB/s^*

Maximum board power — NVIDIA H200 SXM — 141 GB HBM3e

700^* W

Peak memory bandwidth per GPU — NVIDIA H200 SXM — 141 GB HBM3e

4,800^* GB/s

Memory capacity per GPU — NVIDIA H200 SXM — 141 GB HBM3e

141^* GB

Memory reported in a documented driver sample — NVIDIA H200 SXM — 141 GB HBM3e

143,771^* MiB

Scope and conditions

Configuration: Documented driver sample; not nominal VRAM capacity.^*

Memory technology — NVIDIA H200 SXM — 141 GB HBM3e

HBM3e^*

GPU model — NVIDIA H200 SXM — 141 GB HBM3e

NVIDIA H200 SXM^*

Specifications 2 · 72 fields

proc transistors

80,000 million^*

proc die size

814^* mm²

cores

16,896^*

tensor cores

528^*

sm count

132^*

base clock

1,500^*

boost clock

1,980^*

memory clock

1,593^*

ram

141^*

name

NVIDIA H200 SXM 141 GB^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-interactive-99.9 (fp16)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: gptj-99 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: deepseek-r1 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: gptj-99.9 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: mixtral-8x7b (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama3.1-8b (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama3.1-405b (fp16)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-99 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-interactive-99.9 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-99.9 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-interactive-99 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-99.9 (UNSET)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama3.1-405b (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-99.9 (fp16)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

ram type

HBM3e^*

bus width

6,144^*

bus interface

PCIe 5.0 x16^*

ram bandwidth

4.89 TB/s^*

fp16 performance

267.6^*

fp32 performance

66.91^*

fp64 performance

33.45^*

score Graphics benchmark result

40.817125^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,113.10375^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

2,562.725^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

5,161.375^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

2,562.725^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,524.05^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

25.347^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

3,534.05^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

2,444.3375^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

2,422.625^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

7,131.8^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

2,444.3375^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

3,552.625^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,995.0125^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

tdp

700^* W

suggested psu

1,100^* W

slot width

SXM Module^*

power connectors

8-pin EPS^*

api cuda

9^*

is retail board

No^*

gpi value

67.38^*