NVIDIA GeForce RTX 4090 Founders Edition product photo
gpuFeaturedllm-inferenceimage-gencomfyui

NVIDIA RTX 4090

Updated Nov 5, 2026
VRAM
24 GB
Bandwidth
1008 GB/s
TDP
450 W
MSRP
$1,599
Category
gpu

Popular models on the NVIDIA RTX 4090

Fit at Q4_K_M and estimated speed on 24 GB

ModelQ4 sizeFitEst. speed
Qwen3.6 35B A3B21 GBTight~238 tok/s
dolphin-2.9.1-yi-1.5-34b20 GBTight~31 tok/s
Laguna-XS-2.119 GBTight~33 tok/s
Gemma 4 31B19 GBTight~33 tok/s
OTel-2.0-LLM-31B-IT19 GBTight~33 tok/s
Qwen3 32B19 GBTight~33 tok/s
OTel-2.0-LLM-31B-IT-QLoRA18 GBFits~35 tok/s
Qwen3.6 27B16 GBFits~39 tok/s
Qwen3.8 27B16 GBFits~39 tok/s
Qwen3.8-27B-OBLITERATED16 GBFits~39 tok/s
Gemma 4 26B A4B15 GBFits~184 tok/s
gpt-oss-20b12 GBFits~208 tok/s

Estimates: Q4_K_M weights + 1.8 GB overhead; speed at Q4_K_M, scaled from memory bandwidth.

Local inference speed

llama.cpp decode speed (tok/s) - measured

Workloadtok/s
LLaMA 3 8B Q4127.74
LLaMA 3 8B F1654.34

Measured - XiongjieDai/GPU-Benchmarks-on-LLM-Inference · updated 2026-11-05

Stacks for the NVIDIA RTX 4090

Complete local setups that run on 24 GB

AI pair programming in your terminal with local models, fully private

intermediate~10 min

Generate photorealistic images locally with node-based AI workflows

intermediate~20 min

High-speed local text generation

advanced~35 min

Experiment with open GGUF models locally

intermediate~25 min

Unify multiple local and cloud models behind a single OpenAI-compatible API

advanced~20 min

A private AI coding assistant in your editor

beginner~20 min

NVIDIA RTX 4090Link to this section

In 2026, despite the 5090's launch, the RTX 4090 remains the most popular GPU on r/LocalLLaMA - and for good reason. 24 GB VRAM runs every model in the "sweet spot" range (Qwen3-30B, Mistral Small 3, Gemma 3 27B, DeepSeek Coder V3) at Q4_K_M or Q5_K_M with room for serious context. The used price has crashed to $1200-1500 since the 5090 launched.

Quick verdictLink to this section

The 4090 is the answer when:

  • You want the cheapest GPU that runs 30B models well
  • You don't need 70B in a single GPU (use Llama 3.3 70B Q3 if you must, but it's tight)
  • You're buying once and not upgrading for 3-5 years

Get a 5090 if you specifically want comfortable 70B inference. Otherwise the 4090 is 80% of the experience at 60% of the price.

Real-world AI inferenceLink to this section

ModelTokens/secNotes
Qwen3-30B Q4_K_M~25 tok/sSweet spot
Llama 3.3 70B Q3_K_M~12 tok/sTight fit, usable
Mistral Small 3 Q5_K_M~42 tok/sPlenty of room
ComfyUI SDXL (1024x1024)~10s/imageExcellent for image gen
ComfyUI Flux Dev~32s/imageReasonable

Spec breakdownLink to this section

  • VRAM: 24 GB GDDR6X
  • Memory bandwidth: 1008 GB/s
  • TDP: 450 W
  • PCIe: 4.0 ×16
  • Slot count: 3-slot
  • Power: 850W PSU minimum, 1000W comfortable

Best models that fitLink to this section

  • Q4_K_M: Qwen3-30B (18GB), Mistral Small 3 (14GB) - comfortable
  • Q5_K_M: Qwen3-30B (22GB) at smaller context
  • Q8_0: 13-14B models like Qwen3-14B
  • Llama 3.3 70B at Q3: tight but works, ~12 tok/s

Used market realityLink to this section

The 5090 launch crashed used 4090 prices. Sweet spots on eBay:

  • $1200-1400: well-treated cards from non-mining sources (last 18 months)
  • $1000-1200: heavier-use cards, verify fans/thermals
  • Under $1000: probably ex-mining, avoid unless you can verify

ALWAYS run the seller's serial through NVIDIA warranty check.

Honest alternativesLink to this section

  • RTX 5090 new ($2000-2200): if 70B is mandatory
  • Dual RTX 3090 used build (~$1400-1800): same 48GB VRAM, worse power efficiency
  • Mac Studio M4 Ultra ($4k+): bigger VRAM ceiling, slower per-token

Frequently asked

Quick answers to common questions

How much VRAM does the NVIDIA RTX 4090 have?

The NVIDIA RTX 4090 has 24 GB of VRAM with 1008 GB/s memory bandwidth. MSRP was $1,599.

What local AI models can run on the NVIDIA RTX 4090?

The NVIDIA RTX 4090 with 24 GB VRAM can run many models depending on quantization. Models up to ~36B params may fit at Q4_K_M. Use our VRAM calculator to check specific models.

Is the NVIDIA RTX 4090 good for local AI inference?

NVIDIA RTX 4090 is best for llm-inference, image-gen, comfyui. With ample VRAM it handles most open models well.

Where can I buy the NVIDIA RTX 4090?

Check our buy links above for the best current prices on Amazon, Newegg, and B&H. Prices vary by retailer and availability.

How does the NVIDIA RTX 4090 compare to other GPUs?

NVIDIA RTX 4090 has 24 GB VRAM and 1008 GB/s bandwidth. This puts it in the high-end category, suitable for most open models. Browse our hardware directory for side-by-side comparisons.

Is the NVIDIA RTX 4090 worth buying right now?

The current price is $1599. The price is at or above MSRP. Consider waiting for sales events like Prime Day or Black Friday.

What power supply do I need for the NVIDIA RTX 4090?

The NVIDIA RTX 4090 has a TDP of 450W. This requires a high-wattage PSU (850W+ recommended). Always check the manufacturer's recommendations for your specific build.

Nearby options

Similar hardware and models that fit