Best local LLMs for 12GB VRAM
Top picks for 12 GB: Qwen3.5 9B, Qwen3.5 4B, Muse Glimmer. 22 benchmarked models fit at Q4.
Fits = Q4_K_M + ~1.8 GB overhead ≤ 12 GB · updated every refresh
Ranked by measured benchmarks
| # | Model | Score | Q4 size | Headroom | Run it |
|---|---|---|---|---|---|
| 1 | Qwen3.5 9B | 60GPQAMMLU | 6 GB | +4.2 GB | setup → |
| 2 | Qwen3.5 4B | 43GPQAMMLU | 3 GB | +7.2 GB | setup → |
| 3 | Muse Glimmer | 42AAArenaGPQA | 10 GB | +0.2 GB | setup → |
| 4 | Ling 3.0 Tiny | 32GPQA | 5 GB | +5.2 GB | setup → |
| 5 | MiniCPM5-2B | 27GPQA | 1 GB | +9.2 GB | setup → |
| 6 | NVIDIA-Nemotron-Nano-9B-v2 | 27GPQAMMLU | 5 GB | +5.2 GB | setup → |
| 7 | Nemotron 3.5 Lightning | 26AAGPQA | 10 GB | +0.2 GB | setup → |
| 8 | Gemma 4 12B | 24GPQA | 7 GB | +3.2 GB | setup → |
| 9 | Mistral Small 4 | 24AAGPQA | 8 GB | +2.2 GB | setup → |
| 10 | Qwen3-4B-Instruct-2507 | 21GPQAMMLU | 2 GB | +8.2 GB | setup → |
| 11 | Phi-4 | 19ArenaGPQAMMLU | 8.5 GB | +1.7 GB | ollama run phi-4:14b |
| 12 | Granite 4.2 3B | 16AAArenaGPQA | 2 GB | +8.2 GB | setup → |
| 13 | Olmo-3-7B-Think | 15GPQA | 4 GB | +6.2 GB | setup → |
| 14 | granite-4.1-8b | 12ArenaGPQA | 5 GB | +5.2 GB | setup → |
| 15 | Phi-4 Mini | 9GPQAMMLU | 2.5 GB | +7.7 GB | ollama run phi-4-mini:3b |
Also fits, not benchmarked yet
Qwen3 14BSmolLM2-135MQwen2.5-Coder-14B-InstructNVIDIA-Nemotron-3-Nano-4B-BF16Qwen2.5-0.5BKimi-K3-DSparkMiniCPM5-2B-DSparkQwen2.5-14B-Instruct
GPUs with 12 GB–15 GB
NVIDIA RTX 3060 12GB
12 GB · 360 GB/s
~40 tok/s est. Llama 3 8B
$199
Intel Arc B580
12 GB · 456 GB/s
~55 tok/s est. Llama 3 8B
$249
AMD Radeon RX 7700 XT
12 GB · 432 GB/s
~50 tok/s est. Llama 3 8B
$389
NVIDIA RTX 3080 12GB
12 GB · 912 GB/s
106 tok/s Llama 3 8B
$479
NVIDIA RTX 4070
12 GB · 504 GB/s
~60 tok/s est. Llama 3 8B
$499
NVIDIA RTX 3080 Ti
12 GB · 912 GB/s
107 tok/s Llama 3 8B
$549
NVIDIA RTX 4070 Super
12 GB · 504 GB/s
~60 tok/s est. Llama 3 8B
$549
NVIDIA RTX 5070
12 GB · 672 GB/s
~80 tok/s est. Llama 3 8B
$649
Frequently asked
What is the best LLM for 12GB of VRAM?
By measured benchmarks, Qwen3.5 9B, Qwen3.5 4B, Muse Glimmer lead the models that fit 12 GB at Q4_K_M quantization.
How is "fits" decided?
A model fits when its Q4_K_M weights plus about 1.8 GB of runtime overhead (KV cache, activations) are at or under 12 GB. Long contexts need more; use the VRAM calculator for an exact figure.
How are the models ranked?
Each model's score averages the measured signals it has - Artificial Analysis index, LMArena rating, GPQA, MMLU-Pro - as percentiles across all chat models. Models without any measured benchmark are listed separately, not ranked.