GeekRadar / The comparison desk

COMPARE.

GPU

NVIDIA

NVIDIA H100 SXM5 94 GB

Graphics processor

Video memory: 94 GB^*
Memory bus: 5,120 bit^*
Rated memory bandwidth: 3,360 GB/s^*

Benchmark results

LLM Inference: MLPerf: gptj-99 (fp8) · version unreported · score1,693.47^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: gptj-99.9 (fp8) · version unreported · score1,693.47^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score1,668.90^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-99.9 (fp8) · version unreported · score2,106.08^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-interactive-99 (fp8) · version unreported · score1,380.19^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-interactive-99.9 (fp8) · version unreported · score1,380.19^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: mixtral-8x7b (fp8) · version unreported · score4,714.20^* tokens/sPlatform unreported Compare results

Processing & platform

Architecture: Hopper^*
Process node: 5 nm^*
GPU cores: 16,896^*
Graphics interface: PCIe 5.0 x16^*
Reported chip die area: 814 mm²^*
Transistors: 80,000 million^*
GPU L2 cache: 50 MB^*
Graphics processor: GH100^*
Texture mapping units: 528^*
Raster output units: 24^*
Streaming multiprocessors: 132^*
Tensor cores: 528^*
Chip foundry: TSMC^*
Card or module format: SXM Module^*
Declared GPU TDP: 700 W^*
Suggested system PSU: 1,100 W^*
External power connectors: 8-pin EPS^*
OpenCL: 3.0^*
CUDA compute capability: 9.0^*

Memory & storage

Memory options: 94^*
Video memory: 94 GB^*
Video memory type: HBM3^*
Memory bus: 5,120 bit^*
Rated memory bandwidth: 3,360 GB/s^*

Power specifications

Thermal design power: 700 W^*

System configuration

Detailed specifications

Specifications 1 · 6 fields

GPU architecture — H100-SXM5 (94GB) — 94 GB: Hopper^*
Native hardware support: FP4 tensor — H100-SXM5 (94GB) — 94 GB: No^*
Native hardware support: FP6 tensor — H100-SXM5 (94GB) — 94 GB: No^*
Physical configuration — H100-SXM5 (94GB) — 94 GB: SXM5^*
Memory capacity per GPU — H100-SXM5 (94GB) — 94 GB: 94^* GB
GPU model — H100-SXM5 (94GB) — 94 GB: H100-SXM5 (94GB)^*

Specifications 2 · 51 fields

proc transistors

80,000 million^*

proc die size

814^* mm²

cores

16,896^*

tensor cores

528^*

sm count

132^*

base clock

1,350^*

boost clock

1,980^*

memory clock

1,313^*

ram

94^*

name

NVIDIA H100 SXM5 94 GB^*

test name Graphics benchmark result

LLM Inference: MLPerf: gptj-99 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-interactive-99 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-interactive-99.9 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-99.9 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-99 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: gptj-99.9 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: mixtral-8x7b (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

ram type

HBM3^*

bus width

5,120^*

bus interface

PCIe 5.0 x16^*

ram bandwidth

3.36 TB/s^*

fp16 performance

267.6^*

fp32 performance

66.91^*

fp64 performance

33.45^*

score Graphics benchmark result

1,693.4675^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,693.4675^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

2,106.075^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,668.9^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,380.1875^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,380.1875^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

4,714.2^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

tdp

700^* W

suggested psu

1,100^* W

slot width

SXM Module^*

power connectors

8-pin EPS^*

api cuda

9^*

is retail board

No^*

gpi value

57.75^*