GeForce RTX 4080 Max-Q

The GeForce RTX 4080 Max-Q is an NVIDIA GPU built on the Ada Lovelace architecture, released 2023-01-03. For running AI locally, the numbers that matter are its 12 GB of GDDR6 and 432 GB/s of memory bandwidth. VRAM decides which models fit at all; bandwidth sets how fast they generate text.

What you can run on 12 GB

At Q4_K_M quantization (the usual local default), 12 GB holds models up to roughly 19B parameters, leaving headroom for context. On this card you can run, among others:

• Qwen2.5-14B-Instruct - 14.8B parameters
• Qwen2.5-Coder-14B-Instruct - 14.8B parameters
• Phi-4 - 14B parameters
• Qwen3 14B - 14B parameters
• Wan2.1 T2V 14B - 14B parameters

Larger models need a higher-VRAM card, a second GPU, or CPU offload (which is much slower). Check any specific model with the VRAM calculator, or see the full picture on what can I run.

Local LLM speed (LLaMA 3, llama.cpp)

Single-stream token-generation throughput - measured (XiongjieDai/GPU-Benchmarks-on-LLM-Inference):

Model (quant)	Speed on GeForce RTX 4080 Max-Q
Llama 3 8B (Q4_K_M)	106.2 tok/s
Llama 3 8B (F16)	40.3 tok/s
Llama 3 70B (Q4_K_M)	✗ won't fit

Because decode is memory-bandwidth bound, the 432 GB/s figure is the best single predictor of chat speed on this card. These are real llama.cpp runs.

Memory and power

• VRAM: 12 GB GDDR6 (192-bit bus)
• Bandwidth: 432 GB/s
• TDP: 60 W
• Process: 5 nm
• Interface: PCIe 4.0 x16

Quantization and context

Quantization trades a little quality for a lot of VRAM. On 12 GB you can fit roughly a 19B model at Q4_K_M, about a 10B model at the higher-quality Q8, or a smaller model at full FP16. Longer context windows also consume VRAM (the KV cache grows with context length), so leave a few GB of headroom if you plan to use large prompts or many concurrent requests. For most chat and coding use, Q4_K_M on this card is the sweet spot between speed, quality, and the 12 GB budget.

How it compares

Similar cards for local AI, by VRAM and 8B-Q4 speed:

GPU	VRAM	Bandwidth	Llama 3 8B Q4
GeForce RTX 4080 Max-Q	12 GB	432 GB/s	106.2 tok/s
AMD Radeon RX 7700 XT	12 GB	432 GB/s	50.6 tok/s
Arc Pro A60	12 GB	384 GB/s	44.9 tok/s
Intel Arc B580	12 GB	456 GB/s	53.4 tok/s

Bottom line

The GeForce RTX 4080 Max-Q is best for budget-llm, image-gen. 12 GB is the practical entry point for serious local LLMs (7B-13B at Q4). If you need more, compare with AMD Radeon RX 7700 XT and Arc Pro A60.

Sources

• Specifications: RightNow GPU Database (TechPowerUp data)
• Benchmarks: GPU-Benchmarks-on-LLM-Inference (measured)

Specs and benchmarks last checked 2026-06-08. Verify current pricing before buying.