No matched benchmark results are available for this entry yet.
Run it locally
Inference · memory estimate
Hermes 3 405B Instruct
405.85 B total parameters · llama · BF16
Published checkpoint tensor payload: 756.0 GiB. This includes stored copies and buffers; it is separate from the runtime estimates below.
Estimated accelerator memory
Weights
207.9 GiB
KV cache
2.0 GiB
Runtime allowance
20.8 GiB
One user, full model in accelerator memory.
GPUs with memory headroom
No single GPU in this selection has enough memory headroom. Offloading or multi-GPU inference needs a separate plan.
Local AI systems with memory headroom
No catalogued Local AI system meets this memory estimate with the reserved headroom.
Shared-memory systems use a planning budget of 75% of installed memory, leaving 25% for the OS and other allocations. This is an assumption, not a measured GPU allocation. Firmware and driver limits may be lower and may require configuration. Memory matches do not verify a runtime, quantized model artifact or inference speed.
Includes FP16 KV cache, prompt plus generated tokens, quantization metadata and a runtime allowance. GPU matches reserve 10% of VRAM. Memory fit is not a speed or software-compatibility guarantee; quantized weights and runtime support must be available.
API cost
USD / month
Standard text-token estimate. Excludes taxes, caching, batch discounts, reasoning and multimodal surcharges. Provider limits and regional availability may differ.