GeekRadar / The comparison desk

COMPARE.

Accelerators

NVIDIA

NVIDIA H100 (SXM)

Accelerators

Processing & platform

Architecture: Hopper^*

Memory & storage

Video memory: 80 GB^*
Video memory type: HBM3^*
Memory bus: 5,120 bit^*
Rated memory bandwidth: 3,350 GB/s^*

System configuration

Detailed specifications

Specifications 1 · 22 fields

GPU architecture — NVIDIA H100 SXM — 80 GB HBM3

Hopper^*

Theoretical peak: BF16 tensor dense per GPU — NVIDIA H100 SXM — 80 GB HBM3

989.4^* TFLOPS

Scope and conditions

Accumulation: FP32^*

Theoretical peak: FP16 tensor dense per GPU — NVIDIA H100 SXM — 80 GB HBM3

989.4^* TFLOPS

Scope and conditions

Accumulation: FP32^*

Theoretical peak: FP32 non-tensor dense per GPU — NVIDIA H100 SXM — 80 GB HBM3

66.9^* TFLOPS

Native hardware support: FP4 tensor — NVIDIA H100 SXM — 80 GB HBM3

No^*

Theoretical peak: FP64 non-tensor dense per GPU — NVIDIA H100 SXM — 80 GB HBM3

33.5^* TFLOPS

Theoretical peak: FP64 tensor dense per GPU — NVIDIA H100 SXM — 80 GB HBM3

66.9^* TFLOPS

Native hardware support: FP6 tensor — NVIDIA H100 SXM — 80 GB HBM3

No^*

Theoretical peak: FP8 tensor dense per GPU — NVIDIA H100 SXM — 80 GB HBM3

1,978.9^* TFLOPS

Scope and conditions

Accumulation: FP32^*

Theoretical peak: INT8 tensor dense per GPU — NVIDIA H100 SXM — 80 GB HBM3

1,978.9^* TOPS

Theoretical peak: TF32 tensor dense per GPU — NVIDIA H100 SXM — 80 GB HBM3

494.7^* TFLOPS

Physical configuration — NVIDIA H100 SXM — 80 GB HBM3

SXM5^*

GPU interconnect bandwidth (NVLink 4) bidirectional — NVIDIA H100 SXM — 80 GB HBM3

900^* GB/s

GPU interconnect bandwidth per direction — NVIDIA H100 SXM — 80 GB HBM3

450^* GB/s

Scope and conditions

Calculation: 900 / 2^*

PCIe interface declaration — NVIDIA H100 SXM — 80 GB HBM3

PCIe Gen5: 128GB/s^*

Maximum board power — NVIDIA H100 SXM — 80 GB HBM3

700^* W

Peak memory bandwidth per GPU — NVIDIA H100 SXM — 80 GB HBM3

3,350^* GB/s

Memory interface width per GPU — NVIDIA H100 SXM — 80 GB HBM3

5,120^* bit

Memory capacity per GPU — NVIDIA H100 SXM — 80 GB HBM3

80^* GB

Memory reported in a documented driver sample — NVIDIA H100 SXM — 80 GB HBM3

81,559^* MiB

Scope and conditions

Configuration: Documented driver sample; not nominal VRAM capacity.^*

Memory technology — NVIDIA H100 SXM — 80 GB HBM3

HBM3^*

GPU model — NVIDIA H100 SXM — 80 GB HBM3

NVIDIA H100 SXM^*

Specifications 2 · 16 fields

device name: NVIDIA H100 (SXM)^*
typical power w: 700^* W
on chip sram mb: 50^*
hbm memory gb: 80^* GB
memory bandwidth tb s: 3.35^*
headline metric: 3958 TOPS INT8/FP8 dense (7916 sparse); 80 GB HBM3; the 'Optics vs NVIDIA' reference point^*
source type: Vendor^*
vendor: NVIDIA^*
tops per watt calc: 5.654^*
compute paradigm: Electronic^*
device type: GPU^*
release year: 2022^*
maturity: Commercial^*
precision: INT8/FP8^*
throughput unit: TOPS^*
peak throughput: 3,958^*

* Provisional data