GeekRadar / The comparison desk
COMPARE.
GPURadeon Instinct MI300X
Graphics processor
- Video memory
- 192 GB*
- Memory bus
- 8,192 bit*
- Rated memory bandwidth
- 5,320 GB/s*
Benchmark results
- LLM Inference: llama.cpp Bench: llama-2-7b-Q4_0 (Q4_0) · version unreported · score232.92* tokens/sPlatform unreported Compare results
- LLM Inference: MLPerf: llama2-70b-99 (fp8) · version unreported · score1,056.89* tokens/sPlatform unreported Compare results
- LLM Inference: MLPerf: llama2-70b-99.9 (fp16) · version unreported · score1,739.53* tokens/sPlatform unreported Compare results
- LLM Inference: MLPerf: llama2-70b-99.9 (fp8) · version unreported · score1,056.89* tokens/sPlatform unreported Compare results
- LLM Inference: MLPerf: llama2-70b-99.9 (UNSET) · version unreported · score1,982.96* tokens/sPlatform unreported Compare results
- LLM Inference: MLPerf: llama3.1-405b (fp16) · version unreported · score34.83* tokens/sPlatform unreported Compare results
- LLM Inference: MLPerf: llama3.1-405b (UNSET) · version unreported · score30.43* tokens/sPlatform unreported Compare results
- LLM Inference: MLPerf: mixtral-8x7b (fp8) · version unreported · score5,975.24* tokens/sPlatform unreported Compare results
Processing & platform
- Architecture
- CDNA 3.0*
- Process node
- 5 nm*
- GPU cores
- 19,456*
- GPU base clock
- 1,000 MHz*
- Graphics interface
- PCIe 5.0 x16*
- Reported chip die area
- 1,017 mm²*
- Transistors
- 153,000 million*
- GPU boost clock
- 2,100 MHz*
- GPU L2 cache
- 16 MB*
- Graphics processor
- Aqua*
- Texture mapping units
- 1,216*
- Raster output units
- 0*
- Chip foundry
- TSMC*
- Card or module format
- OAM Module*
- Declared GPU TDP
- 750 W*
- Suggested system PSU
- 1,150 W*
- External power connectors
- None*
- OpenCL
- 3.0*
Memory & storage
- Memory options
- 192*
- Video memory
- 192 GB*
- Video memory type
- HBM3*
- Memory bus
- 8,192 bit*
- Rated memory bandwidth
- 5,320 GB/s*
Power specifications
- Thermal design power
- 750 W*
System configuration
Detailed specifications
Specifications 1 · 22 fields
- GPU architecture — AMD Instinct MI300X — 192 GB HBM3
- CDNA 3*
- Theoretical peak: BF16 tensor dense per GPU — AMD Instinct MI300X — 192 GB HBM3
- 1,307.4* TFLOPS
- Theoretical peak: FP16 tensor dense per GPU — AMD Instinct MI300X — 192 GB HBM3
- 1,307.4* TFLOPS
- Theoretical peak: FP32 non-tensor dense per GPU — AMD Instinct MI300X — 192 GB HBM3
- 163.4* TFLOPS
- Native hardware support: FP4 tensor — AMD Instinct MI300X — 192 GB HBM3
- No*
- Theoretical peak: FP64 non-tensor dense per GPU — AMD Instinct MI300X — 192 GB HBM3
- 81.7* TFLOPS
- Theoretical peak: FP64 tensor dense per GPU — AMD Instinct MI300X — 192 GB HBM3
- 163.4* TFLOPS
- Native hardware support: FP6 tensor — AMD Instinct MI300X — 192 GB HBM3
- No*
- Theoretical peak: FP8 tensor dense per GPU — AMD Instinct MI300X — 192 GB HBM3
- 2,614.9* TFLOPS
- Theoretical peak: INT8 tensor dense per GPU — AMD Instinct MI300X — 192 GB HBM3
- 2,614.9* TOPS
- Theoretical peak: TF32 tensor dense per GPU — AMD Instinct MI300X — 192 GB HBM3
- 653.7* TFLOPS
- Physical configuration — AMD Instinct MI300X — 192 GB HBM3
- OAM*
- GPU interconnect bandwidth (Infinity Fabric) bidirectional — AMD Instinct MI300X — 192 GB HBM3
- 896* GB/s
- GPU interconnect bandwidth per direction — AMD Instinct MI300X — 192 GB HBM3
- 448* GB/s
- Infinity Fabric link count — AMD Instinct MI300X — 192 GB HBM3
- 7* links
- PCIe interface declaration — AMD Instinct MI300X — 192 GB HBM3
- PCIe Gen 5 x16*
- Maximum board power — AMD Instinct MI300X — 192 GB HBM3
- 750* W
- Peak memory bandwidth per GPU — AMD Instinct MI300X — 192 GB HBM3
- 5,300* GB/s
- Memory interface width per GPU — AMD Instinct MI300X — 192 GB HBM3
- 8,192* bit
- Memory capacity per GPU — AMD Instinct MI300X — 192 GB HBM3
- 192* GB
- Memory technology — AMD Instinct MI300X — 192 GB HBM3
- HBM3*
- GPU model — AMD Instinct MI300X — 192 GB HBM3
- AMD Instinct MI300X*
Specifications 2 · 50 fields
- proc transistors
- 153,000 million*
- proc die size
- 1,017* mm²
- cores
- 19,456*
- base clock
- 1,000*
- boost clock
- 2,100*
- memory clock
- 2,525*
- ram
- 192*
- name
- AMD Radeon Instinct MI300X*
- test name Graphics benchmark result
- LLM Inference: MLPerf: llama2-70b-99.9 (UNSET)*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- test name Graphics benchmark result
- LLM Inference: llama.cpp Bench: llama-2-7b-Q4_0 (Q4_0)*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- test name Graphics benchmark result
- LLM Inference: MLPerf: llama2-70b-99.9 (fp16)*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- test name Graphics benchmark result
- LLM Inference: MLPerf: mixtral-8x7b (fp8)*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- test name Graphics benchmark result
- LLM Inference: MLPerf: llama2-70b-99 (fp8)*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- test name Graphics benchmark result
- LLM Inference: MLPerf: llama3.1-405b (fp16)*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- test name Graphics benchmark result
- LLM Inference: MLPerf: llama2-70b-99.9 (fp8)*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- test name Graphics benchmark result
- LLM Inference: MLPerf: llama3.1-405b (UNSET)*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- ram type
- HBM3*
- bus width
- 8,192*
- bus interface
- PCIe 5.0 x16*
- ram bandwidth
- 10.3 TB/s*
- fp16 performance
- 653.7*
- fp32 performance
- 81.72*
- fp64 performance
- 81.72*
- score Graphics benchmark result
- 34.831125*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- score Graphics benchmark result
- 30.431125*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- score Graphics benchmark result
- 1,056.8925*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- score Graphics benchmark result
- 5,975.2375*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- score Graphics benchmark result
- 232.92*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- score Graphics benchmark result
- 1,739.525*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- score Graphics benchmark result
- 1,056.8925*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- score Graphics benchmark result
- 1,982.9625*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- tdp
- 750* W
- suggested psu
- 1,150* W
- slot width
- OAM Module*
- is retail board
- No*
- gpi value
- 94.1*
- unit Graphics benchmark result
- tokens/s*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- unit Graphics benchmark result
- tokens/s*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- unit Graphics benchmark result
- tokens/s*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- unit Graphics benchmark result
- tokens/s*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- unit Graphics benchmark result
- tokens/s*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- unit Graphics benchmark result
- tokens/s*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- unit Graphics benchmark result
- tokens/s*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- unit Graphics benchmark result
- tokens/s*
Scope and conditions
- benchmark_context
- suite
- Per-row third-party test name*
- test_configuration_verified
- No*
- comparable
- No*
- vendor
- amd*
- manufacturer
- AMD*
- arch name
- CDNA 3.0*
- card release date
- 2023-12-06*
- proc foundry
- TSMC*
- proc process size
- 5* nm
Specifications 3 · 31 fields
- OpenCL
- 3.0*
- Graphics interface
- PCIe 5.0 x16*
- Memory bus
- 8,192* bit
- Reported shader processors
- 19,456*
- Reported theoretical FP16 peak
- 81.72* TFLOPS
Scope and conditions
- Arithmetic density
- Not stated*
- Reported theoretical FP32 peak
- 81.72* TFLOPS
Scope and conditions
- Arithmetic density
- Not stated*
- Reported theoretical FP64 peak
- 40.86* TFLOPS
Scope and conditions
- Arithmetic density
- Not stated*
- Declared L1 cache configuration
- 16 KB (per CU)*
- Declared GPU L2 cache
- 16 MB*
- Reported display outputs
- No outputs*
- Theoretical pixel fill rate
- 0 MPixel/s*
- External power connectors
- None*
- Reported chip die area
- 1017 mm²*
- Chip foundry
- TSMC*
- Process node declaration
- 5 nm*
- Transistors
- 153,000 million*
- Graphics processor
- Aqua*
- Reported video memory
- 192* GB
- Rated memory bandwidth
- 5.32 TB/s*
- Video memory type
- HBM3*
- Raster output units
- 0*
- Reported shading units
- 19,456*
- Card or module format
- OAM Module*
- Suggested system PSU
- 1150 W*
- 2:4 tensor sparsity
- No*
- Declared GPU TDP
- 750 W*
- Theoretical texture fill rate
- 2,553.6 GTexel/s*
- Texture mapping units
- 1,216*
- GPU cores
- 19,456*
- Video memory
- 192* GB
- Video memory type
- HBM3*
Specifications 4 · 47 fields
- Brand
- AMD*
- Top BUS WIDTH
- 8,192* bit
- Render Config MCM
- 2*
- Graphics Processor GPU Name
- Aqua Vanjaram*
- Render Config Matrix Cores
- 1,216*
- Graphics Processor Architecture
- CDNA 3.0*
- Graphics Processor Process Size
- 5* nm
- Graphics Processor Transistors
- 153,000 million*
- Graphics Processor Die Size
- 1,017* mm²
- Graphics Card Release Date
- Dec 6th, 2023*
- Graphics Card Generation
- Radeon Instinct (MIx)*
- Name
- Radeon Instinct MI300X*
- Graphics Card Bus Interface
- PCIe 5.0 x16*
- Clock Speeds Memory Clock
- 2525 MHz 10.1 Gbps effective*
- Memory Memory Size
- 192* GB
- Memory Memory Type
- HBM3*
- Memory Memory Bus
- 8,192* bit
- Memory Bandwidth
- 10.3 TB/s*
- Render Config TMUs
- 1,216*
- Top GRAPHICS PROCESSOR
- Aqua Vanjaram*
- Render Config ROPs
- 0*
- Theoretical Performance Pixel Rate
- 0 MPixel/s
- Theoretical Performance Texture Rate
- 2,553.6 GTexel/s
- Board Design Slot Width
- OAM Module*
- Board Design TDP
- 750* W
- Board Design Suggested PSU
- 1,150* W
- Board Design Outputs
- No outputs*
- Graphics Features OpenCL
- 3*
- source file
- amd2023.json*
- Graphics Processor Foundry
- TSMC*
- Graphics Card Predecessor
- FirePro Data Center*
- Graphics Processor Density
- 150.4M / mm²*
- Top TMUS
- 1,216*
- Graphics Processor Chip Package
- MCM*
- Clock Speeds Base Clock
- 1,000* MHz
- Clock Speeds Boost Clock
- 2,100* MHz
- Top CORES
- 19,456*
- Render Config Shading Units
- 19,456*
- Render Config Compute Units
- 304*
- Render Config L2 Cache
- 16* MB
- Top MEMORY SIZE
- 192* GB
- Theoretical Performance FP32 (float)
- 81.72* TFLOPS
- Theoretical Performance FP64 (double)
- 81.72 TFLOPS (1:1)
- Render Config L1 Cache
- 16 KB (per CU)*
- Theoretical Performance FP16 (half)
- 653.7 TFLOPS (8:1)
- Top MEMORY TYPE
- HBM3*
- Render Config L3 Cache
- 256* MB
Specifications 5 · 14 fields
- manufacturer
- AMD*
- rop
- 24*
- igp
- No*
- bus
- PCIe 5.0 x16*
- memType
- HBM3*
- gpuChip
- GH100*
- productName
- Radeon Instinct MI300X*
- releaseYear
- 2,023*
- memSize
- 192*
- memBusWidth
- 8,192*
- gpuClock
- 1,665*
- memClock
- 1,313*
- unifiedShader
- 16,896*
- tmu
- 528*
0 selected Compare 0
* Provisional data