Best local LLMs for 8GB VRAM
Top picks for 8 GB: Qwen3.5 9B, Qwen3.5 4B, Ling 3.0 Tiny. 17 benchmarked models fit at Q4.
Fits = Q4_K_M + ~1.8 GB overhead ≤ 8 GB · updated every refresh
Ranked by measured benchmarks
| # | Model | Score | Q4 size | Headroom | Run it |
|---|---|---|---|---|---|
| 1 | Qwen3.5 9B | 60GPQAMMLU | 6 GB | +0.2 GB | setup → |
| 2 | Qwen3.5 4B | 43GPQAMMLU | 3 GB | +3.2 GB | setup → |
| 3 | Ling 3.0 Tiny | 32GPQA | 5 GB | +1.2 GB | setup → |
| 4 | MiniCPM5-2B | 27GPQA | 1 GB | +5.2 GB | setup → |
| 5 | NVIDIA-Nemotron-Nano-9B-v2 | 27GPQAMMLU | 5 GB | +1.2 GB | setup → |
| 6 | Qwen3-4B-Instruct-2507 | 21GPQAMMLU | 2 GB | +4.2 GB | setup → |
| 7 | Granite 4.2 3B | 16AAArenaGPQA | 2 GB | +4.2 GB | setup → |
| 8 | Olmo-3-7B-Think | 15GPQA | 4 GB | +2.2 GB | setup → |
| 9 | granite-4.1-8b | 12ArenaGPQA | 5 GB | +1.2 GB | setup → |
| 10 | Phi-4 Mini | 9GPQAMMLU | 2.5 GB | +3.7 GB | ollama run phi-4-mini:3b |
| 11 | NVIDIA Nemotron 3 Nano 4B | 6AAGPQA | 2 GB | +4.2 GB | setup → |
| 12 | LFM2.5-1.2B-Instruct | 6GPQAMMLU | 1 GB | +5.2 GB | setup → |
| 13 | granite-4.1-3b | 3GPQA | 2 GB | +4.2 GB | setup → |
| 14 | gemma-2-9b-it | 3Arena | 5 GB | +1.2 GB | setup → |
| 15 | MiniCPM5-1B | 2GPQA | 1 GB | +5.2 GB | setup → |
Also fits, not benchmarked yet
SmolLM2-135MNVIDIA-Nemotron-3-Nano-4B-BF16Qwen2.5-0.5BKimi-K3-DSparkMiniCPM5-2B-DSparkgemma-3-1b-itSmolLM3-3BLlama 3.1 8B
GPUs with 8 GB–11 GB
NVIDIA RTX 3050
8 GB · 224 GB/s
~25 tok/s est. Llama 3 8B
$169
Intel Arc A750
8 GB · 512 GB/s
~60 tok/s est. Llama 3 8B
$189
Intel Arc B570
10 GB · 360 GB/s
~40 tok/s est. Llama 3 8B
$219
AMD Radeon RX 7600
8 GB · 288 GB/s
~35 tok/s est. Llama 3 8B
$249
NVIDIA RTX 4060
8 GB · 272 GB/s
~30 tok/s est. Llama 3 8B
$279
NVIDIA RTX 5050
8 GB · 320 GB/s
~35 tok/s est. Llama 3 8B
$279
NVIDIA RTX 2080 Ti
11 GB · 616 GB/s
~70 tok/s est. Llama 3 8B
$299
NVIDIA RTX 3070
8 GB · 448 GB/s
71 tok/s Llama 3 8B
$299
Frequently asked
What is the best LLM for 8GB of VRAM?
By measured benchmarks, Qwen3.5 9B, Qwen3.5 4B, Ling 3.0 Tiny lead the models that fit 8 GB at Q4_K_M quantization.
How is "fits" decided?
A model fits when its Q4_K_M weights plus about 1.8 GB of runtime overhead (KV cache, activations) are at or under 8 GB. Long contexts need more; use the VRAM calculator for an exact figure.
How are the models ranked?
Each model's score averages the measured signals it has - Artificial Analysis index, LMArena rating, GPQA, MMLU-Pro - as percentiles across all chat models. Models without any measured benchmark are listed separately, not ranked.