Jetson Orin Nano Super

The Jetson Orin Nano Super is an NVIDIA GPU built on the Ampere architecture, released 2024-12-17. For running AI locally, the numbers that matter are its 8 GB of LPDDR5 and 102.4 GB/s of memory bandwidth. VRAM decides which models fit at all; bandwidth sets how fast they generate text.

What you can run on 8 GB

At Q4_K_M quantization (the usual local default), 8 GB holds models up to roughly 12B parameters, leaving headroom for context. On this card you can run, among others:

• Gemma 4 12B - 12B parameters
• humanizer - 12B parameters
• Bielik-11B-v3.0-Instruct - 11.2B parameters
• Darwin-9B-NEG - 9.7B parameters
• Ornith-1.5-9B - 9.7B parameters

Larger models need a higher-VRAM card, a second GPU, or CPU offload (which is much slower). Check any specific model with the VRAM calculator, or see the full picture on what can I run.

Local LLM speed (LLaMA 3, llama.cpp)

Single-stream token-generation throughput - estimated from memory bandwidth:

Model (quant)	Speed on Jetson Orin Nano Super
Llama 3 8B (Q4_K_M)	12 tok/s
Llama 3 8B (F16)	✗ won't fit
Llama 3 70B (Q4_K_M)	✗ won't fit

Because decode is memory-bandwidth bound, the 102.4 GB/s figure is the best single predictor of chat speed on this card. Estimates are calibrated against measured RTX-40-series cards and are typically within ~15%.

Memory and power

• VRAM: 8 GB LPDDR5 (128-bit bus)
• Bandwidth: 102.4 GB/s
• TDP: 25 W
• Process: 8 nm

Quantization and context

Quantization trades a little quality for a lot of VRAM. On 8 GB you can fit roughly a 12B model at Q4_K_M, about a 6B model at the higher-quality Q8, or a smaller model at full FP16. Longer context windows also consume VRAM (the KV cache grows with context length), so leave a few GB of headroom if you plan to use large prompts or many concurrent requests. For most chat and coding use, Q4_K_M on this card is the sweet spot between speed, quality, and the 8 GB budget.

How it compares

Similar cards for local AI, by VRAM and 8B-Q4 speed:

GPU	VRAM	Bandwidth	Llama 3 8B Q4
Jetson Orin Nano Super	8 GB	102.4 GB/s	12 tok/s
AMD Radeon RX 7600	8 GB	288 GB/s	33.7 tok/s
Arc A530M	8 GB	224 GB/s	26.2 tok/s
Arc A550M	8 GB	224 GB/s	26.2 tok/s

Bottom line

The Jetson Orin Nano Super is best for entry-level, budget-llm. With under 12 GB, stick to small quantized models (up to ~7B). If you need more, compare with AMD Radeon RX 7600 and Arc A530M.

Sources

• Specifications: RightNow GPU Database (TechPowerUp data)
• Benchmarks: GPU-Benchmarks-on-LLM-Inference (basis for the bandwidth estimate)

Specs and benchmarks last checked 2026-10-30. Verify current pricing before buying.