GeekRadar / The comparison desk

COMPARE.

GPU

NVIDIA

NVIDIA H100 PCIe 80 GB

Graphics processor

Video memory: 80 GB^*
Memory bus: 5,120 bit^*
Rated memory bandwidth: 2,040 GB/s^*

Benchmark results

LLM Inference: MLPerf: gptj-99 (fp8) · version unreported · score1,562.82^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: gptj-99.9 (fp8) · version unreported · score1,562.82^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score1,222.48^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: llama2-70b-99.9 (fp8) · version unreported · score1,222.48^* tokens/sPlatform unreported Compare results
LLM Inference: MLPerf: mixtral-8x7b (fp8) · version unreported · score3,466.13^* tokens/sPlatform unreported Compare results

Processing & platform

Architecture: Hopper^*
Process node: 5 nm^*
GPU cores: 14,592^*
Graphics interface: PCIe 5.0 x16^*
Reported chip die area: 814 mm²^*
Transistors: 80,000 million^*
GPU L2 cache: 50 MB^*
Graphics processor: GH100^*
Texture mapping units: 456^*
Raster output units: 24^*
Streaming multiprocessors: 114^*
Tensor cores: 456^*
Chip foundry: TSMC^*
Card or module format: Dual-slot^*
Declared GPU TDP: 350 W^*
Suggested system PSU: 750 W^*
External power connectors: 1x 16-pin^*
OpenCL: 3.0^*
CUDA compute capability: 9.0^*

Memory & storage

Memory options: 80^*
Video memory: 80 GB^*
Video memory type: HBM2e^*
Memory bus: 5,120 bit^*
Rated memory bandwidth: 2,040 GB/s^*

Power specifications

Thermal design power: 350 W^*

System configuration

Detailed specifications

Specifications 1 · 21 fields

GPU architecture — NVIDIA H100 PCIe — 80 GB HBM2e

Hopper^*

Theoretical peak: BF16 tensor dense per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

756^* TFLOPS

Scope and conditions

Accumulation: FP32^*

Theoretical peak: FP16 tensor dense per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

756^* TFLOPS

Scope and conditions

Accumulation: FP32^*

Theoretical peak: FP32 non-tensor dense per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

51.2^* TFLOPS

Native hardware support: FP4 tensor — NVIDIA H100 PCIe — 80 GB HBM2e

No^*

Theoretical peak: FP64 non-tensor dense per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

25.6^* TFLOPS

Theoretical peak: FP64 tensor dense per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

51.2^* TFLOPS

Native hardware support: FP6 tensor — NVIDIA H100 PCIe — 80 GB HBM2e

No^*

Theoretical peak: FP8 tensor dense per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

1,513^* TFLOPS

Scope and conditions

Accumulation: FP32^*

Theoretical peak: INT8 tensor dense per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

1,513^* TOPS

Theoretical peak: TF32 tensor dense per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

378^* TFLOPS

Physical configuration — NVIDIA H100 PCIe — 80 GB HBM2e

PCIe (FHFL dual-slot)^*

GPU interconnect bandwidth (NVLink 4 (three NVLink bridges, two cards)) — NVIDIA H100 PCIe — 80 GB HBM2e

600^* GB/s

PCIe interface declaration — NVIDIA H100 PCIe — 80 GB HBM2e

PCI Express Gen5 x16^*

Maximum board power — NVIDIA H100 PCIe — 80 GB HBM2e

350^* W

Peak memory bandwidth per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

2,000^* GB/s

Memory interface width per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

5,120^* bit

Memory capacity per GPU — NVIDIA H100 PCIe — 80 GB HBM2e

80^* GB

Memory reported in a documented driver sample — NVIDIA H100 PCIe — 80 GB HBM2e

81,559^* MiB

Scope and conditions

Configuration: Documented driver sample; not nominal VRAM capacity.^*

Memory technology — NVIDIA H100 PCIe — 80 GB HBM2e

HBM2e^*

GPU model — NVIDIA H100 PCIe — 80 GB HBM2e

NVIDIA H100 PCIe^*

Specifications 2 · 45 fields

proc transistors

80,000 million^*

proc die size

814^* mm²

cores

14,592^*

tensor cores

456^*

sm count

114^*

base clock

1,095^*

boost clock

1,755^*

memory clock

1,593^*

ram

80^*

name

NVIDIA H100 PCIe 80 GB^*

test name Graphics benchmark result

LLM Inference: MLPerf: mixtral-8x7b (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-99 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: gptj-99 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: llama2-70b-99.9 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

test name Graphics benchmark result

LLM Inference: MLPerf: gptj-99.9 (fp8)^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

ram type

HBM2e^*

bus width

5,120^*

bus interface

PCIe 5.0 x16^*

ram bandwidth

2.04 TB/s^*

fp16 performance

204.9^*

fp32 performance

51.22^*

fp64 performance

25.61^*

score Graphics benchmark result

1,562.8175^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,562.8175^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,222.48^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

1,222.48^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

score Graphics benchmark result

3,466.125^*

Scope and conditions

benchmark_context

suite: Per-row third-party test name^*
test_configuration_verified: No^*
comparable: No^*

tdp

350^* W

suggested psu

750^* W

slot width

Dual-slot^*

power connectors

1x 16-pin^*

api cuda

9^*

is retail board

No^*

gpi value

44.05^*