Best local LLMs for 48GB VRAM
Top picks for 48 GB: Gemma 4 31B, Qwen3.8 27B, A.X-K2. 49 benchmarked models fit at Q4.
Fits = Q4_K_M + ~1.8 GB overhead ≤ 48 GB · updated every refresh
Ranked by measured benchmarks
| # | Model | Score | Q4 size | Headroom | Run it |
|---|---|---|---|---|---|
| 1 | Gemma 4 31B | 76ArenaGPQA | 19 GB | +27.2 GB | setup → |
| 2 | Qwen3.8 27B | 70AAArenaGPQA | 16 GB | +30.2 GB | setup → |
| 3 | A.X-K2 | 69GPQA | 12 GB | +34.2 GB | setup → |
| 4 | Qwen3.6 27B | 66GPQA | 16 GB | +30.2 GB | ollama run qwen3.6:27b |
| 5 | Qwen3.5 35B A3B | 65ArenaGPQAMMLU | 21 GB | +25.2 GB | setup → |
| 6 | Qwen3.5 9B | 60GPQAMMLU | 6 GB | +40.2 GB | setup → |
| 7 | K2 Horizon MoVA 36B A4B | 58GPQA | 22 GB | +24.2 GB | setup → |
| 8 | Gemma 4 26B A4B | 57ArenaGPQA | 15 GB | +31.2 GB | setup → |
| 9 | Qwen3.6 35B A3B | 55GPQA | 21 GB | +25.2 GB | setup → |
| 10 | G9v3-39A5B | 53GPQA | 23 GB | +23.2 GB | setup → |
| 11 | Nemotron 3 Super | 52GPQA | 39 GB | +7.2 GB | setup → |
| 12 | EXAONE 4.5 33B | 50GPQA | 20 GB | +26.2 GB | setup → |
| 13 | Qwen3.5 4B | 43GPQAMMLU | 3 GB | +43.2 GB | setup → |
| 14 | Muse Glimmer | 42AAArenaGPQA | 10 GB | +36.2 GB | setup → |
| 15 | Nemotron Cascade 2 30B A3B | 42GPQA | 18 GB | +28.2 GB | setup → |
Also fits, not benchmarked yet
Qwen3 14Bdolphin-2.9.1-yi-1.5-34bSmolLM2-135MQwen2.5-Coder-14B-InstructNVIDIA-Nemotron-3-Nano-4B-BF16Qwen2.5-0.5BKimi-K3-DSparkLaguna-XS.2
GPUs with 48 GB+
NVIDIA RTX A6000
48 GB · 768 GB/s
~90 tok/s est. Llama 3 8B
$3,299
NVIDIA RTX 6000 Ada Generation
48 GB · 960 GB/s
~110 tok/s est. Llama 3 8B
$5,800
NVIDIA L40S
48 GB · 864 GB/s
~100 tok/s est. Llama 3 8B
$8,500
NVIDIA Jetson AGX Thor (T5000)
128 GB · 273 GB/s
~30 tok/s est. Llama 3 8B
NVIDIA Jetson Thor (T4000)
64 GB · 273 GB/s
~30 tok/s est. Llama 3 8B
NVIDIA DGX Spark (GB10)
128 GB · 273 GB/s
~30 tok/s est. Llama 3 8B
Radeon PRO W7900
48 GB · 864 GB/s
~100 tok/s est. Llama 3 8B
Radeon PRO W7900D
48 GB · 864 GB/s
~100 tok/s est. Llama 3 8B
Frequently asked
What is the best LLM for 48GB of VRAM?
By measured benchmarks, Gemma 4 31B, Qwen3.8 27B, A.X-K2 lead the models that fit 48 GB at Q4_K_M quantization.
How is "fits" decided?
A model fits when its Q4_K_M weights plus about 1.8 GB of runtime overhead (KV cache, activations) are at or under 48 GB. Long contexts need more; use the VRAM calculator for an exact figure.
How are the models ranked?
Each model's score averages the measured signals it has - Artificial Analysis index, LMArena rating, GPQA, MMLU-Pro - as percentiles across all chat models. Models without any measured benchmark are listed separately, not ranked.