NVIDIA Jetson Thor (T4000)

The NVIDIA Jetson Thor (T4000) is a GPU built on the Blackwell architecture, released 2025-08-27. For running AI locally, the numbers that matter are its 64 GB of LPDDR5X and 273.2 GB/s of memory bandwidth. VRAM decides which models fit at all; bandwidth sets how fast they generate text.

What you can run on 64 GB

At Q4_K_M quantization (the usual local default), 64 GB holds models up to roughly 101B parameters, leaving headroom for context. On this card you can run, among others:

• Qwen3-Next-80B-A3B-Instruct - 81.3B parameters
• Qwen3-Coder-Next - 79.7B parameters
• Qwen-72B - 72.3B parameters
• Qwen2.5 72B - 72B parameters
• Llama-3.1-70B-Instruct - 70.6B parameters

Larger models need a higher-VRAM card, a second GPU, or CPU offload (which is much slower). Check any specific model with the VRAM calculator, or see the full picture on what can I run.

Local LLM speed (LLaMA 3, llama.cpp)

Single-stream token-generation throughput - estimated from memory bandwidth:

Model (quant)	Speed on NVIDIA Jetson Thor (T4000)
Llama 3 8B (Q4_K_M)	32 tok/s
Llama 3 8B (F16)	14.5 tok/s
Llama 3 70B (Q4_K_M)	3.8 tok/s

Because decode is memory-bandwidth bound, the 273.2 GB/s figure is the best single predictor of chat speed on this card. Estimates are calibrated against measured RTX-40-series cards and are typically within ~15%.

Memory and power

• VRAM: 64 GB LPDDR5X
• Bandwidth: 273.2 GB/s
• TDP: 40 W

Quantization and context

Quantization trades a little quality for a lot of VRAM. On 64 GB you can fit roughly a 101B model at Q4_K_M, about a 55B model at the higher-quality Q8, or a smaller model at full FP16. Longer context windows also consume VRAM (the KV cache grows with context length), so leave a few GB of headroom if you plan to use large prompts or many concurrent requests. For most chat and coding use, Q4_K_M on this card is the sweet spot between speed, quality, and the 64 GB budget.

How it compares

Similar cards for local AI, by VRAM and 8B-Q4 speed:

GPU	VRAM	Bandwidth	Llama 3 8B Q4
NVIDIA Jetson Thor (T4000)	64 GB	273.2 GB/s	32 tok/s
RTX PRO 5000 72 GB Blackwell	72 GB	1340 GB/s	156.8 tok/s
NVIDIA RTX A6000	48 GB	768 GB/s	89.9 tok/s
NVIDIA L40S	48 GB	864 GB/s	101.1 tok/s

Bottom line

The NVIDIA Jetson Thor (T4000) is best for llm-inference, image-gen, comfyui. With 24 GB+ it comfortably handles most open models, including 30B-class at Q4. If you need more, compare with RTX PRO 5000 72 GB Blackwell and NVIDIA RTX A6000.

Sources

• Specifications: RightNow GPU Database (TechPowerUp data)
• Benchmarks: GPU-Benchmarks-on-LLM-Inference (basis for the bandwidth estimate)

Specs and benchmarks last checked 2026-10-30. Verify current pricing before buying.