SmolLM2-135M

Run it locally

Inference · memory estimate

0.13 B total parameters · llama · BF16

Published checkpoint tensor payload: 0.3 GiB. This includes stored copies and buffers; it is separate from the runtime estimates below.

Weight precision Total context Check a GPU Check a Local AI system

Estimated accelerator memory1.2 GiB

Weights: 0.1 GiB
KV cache: 0.1 GiB
Runtime allowance: 1.0 GiB

One user, full model in accelerator memory.

GPUs with memory headroom

Local AI systems with memory headroom

AMD Ryzen AI Halo (Ryzen AI Max+ 395, 128 GB) Planning budget: 96.0 GiB
ASUS Ascent GX10 (128 GB) Planning budget: 96.0 GiB
Beelink GTR9 Pro (Ryzen AI Max+ 395, 128 GB) Planning budget: 96.0 GiB
Framework Desktop (Ryzen AI Max+ 395, 128 GB) Planning budget: 96.0 GiB
NVIDIA DGX Spark (128 GB) Planning budget: 96.0 GiB

Shared-memory systems use a planning budget of 75% of installed memory, leaving 25% for the OS and other allocations. This is an assumption, not a measured GPU allocation. Firmware and driver limits may be lower and may require configuration. Memory matches do not verify a runtime, quantized model artifact or inference speed.

Includes FP16 KV cache, prompt plus generated tokens, quantization metadata and a runtime allowance. GPU matches reserve 10% of VRAM. Memory fit is not a speed or software-compatibility guarantee; quantized weights and runtime support must be available.

Capabilities

Context window: 8,192 tokens
Open weights: Yes
Model license: apache-2.0
Total parameters: 0.13 B
Checkpoint tensor payload: 0.25 GiB

Input modalities

text

Output modalities

text

Processing & platform

Architecture: llama