Best local LLMs for 24GB VRAM
Top picks for 24 GB: Gemma 4 31B, Qwen3.8 27B, A.X-K2. 41 benchmarked models fit at Q4.
Fits = Q4_K_M + ~1.8 GB overhead ≤ 24 GB · updated every refresh
Ranked by measured benchmarks
| # | Model | Score | Q4 size | Headroom | Run it |
|---|---|---|---|---|---|
| 1 | Gemma 4 31B | 76ArenaGPQA | 19 GB | +3.2 GB | setup → |
| 2 | Qwen3.8 27B | 70AAArenaGPQA | 16 GB | +6.2 GB | setup → |
| 3 | A.X-K2 | 69GPQA | 12 GB | +10.2 GB | setup → |
| 4 | Qwen3.6 27B | 66GPQA | 16 GB | +6.2 GB | ollama run qwen3.6:27b |
| 5 | Qwen3.5 35B A3B | 65ArenaGPQAMMLU | 21 GB | +1.2 GB | setup → |
| 6 | Qwen3.5 9B | 60GPQAMMLU | 6 GB | +16.2 GB | setup → |
| 7 | K2 Horizon MoVA 36B A4B | 58GPQA | 22 GB | +0.2 GB | setup → |
| 8 | Gemma 4 26B A4B | 57ArenaGPQA | 15 GB | +7.2 GB | setup → |
| 9 | Qwen3.6 35B A3B | 55GPQA | 21 GB | +1.2 GB | setup → |
| 10 | EXAONE 4.5 33B | 50GPQA | 20 GB | +2.2 GB | setup → |
| 11 | Qwen3.5 4B | 43GPQAMMLU | 3 GB | +19.2 GB | setup → |
| 12 | Muse Glimmer | 42AAArenaGPQA | 10 GB | +12.2 GB | setup → |
| 13 | Nemotron Cascade 2 30B A3B | 42GPQA | 18 GB | +4.2 GB | setup → |
| 14 | NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 40ArenaMMLU | 18 GB | +4.2 GB | setup → |
| 15 | North Mini Code | 40GPQA | 18 GB | +4.2 GB | setup → |
Also fits, not benchmarked yet
Qwen3 14Bdolphin-2.9.1-yi-1.5-34bSmolLM2-135MQwen2.5-Coder-14B-InstructNVIDIA-Nemotron-3-Nano-4B-BF16Qwen2.5-0.5BKimi-K3-DSparkLaguna-XS.2
GPUs with 24 GB–31 GB
NVIDIA Tesla P40
24 GB · 347 GB/s
~40 tok/s est. Llama 3 8B
$159
NVIDIA RTX 3090
24 GB · 936 GB/s
112 tok/s Llama 3 8B
$750
AMD Radeon RX 7900 XTX
24 GB · 960 GB/s
~110 tok/s est. Llama 3 8B
$849
NVIDIA RTX 3090 Ti
24 GB · 1008 GB/s
~120 tok/s est. Llama 3 8B
$899
NVIDIA RTX 4090
24 GB · 1008 GB/s
128 tok/s Llama 3 8B
$1,599
Arc Pro B60
24 GB · 456 GB/s
~55 tok/s est. Llama 3 8B
Radeon PRO V710
28 GB · 504 GB/s
~60 tok/s est. Llama 3 8B
GeForce RTX 4090 D
24 GB · 1010 GB/s
128 tok/s Llama 3 8B
Frequently asked
What is the best LLM for 24GB of VRAM?
By measured benchmarks, Gemma 4 31B, Qwen3.8 27B, A.X-K2 lead the models that fit 24 GB at Q4_K_M quantization.
How is "fits" decided?
A model fits when its Q4_K_M weights plus about 1.8 GB of runtime overhead (KV cache, activations) are at or under 24 GB. Long contexts need more; use the VRAM calculator for an exact figure.
How are the models ranked?
Each model's score averages the measured signals it has - Artificial Analysis index, LMArena rating, GPQA, MMLU-Pro - as percentiles across all chat models. Models without any measured benchmark are listed separately, not ranked.