gpullm-inferenceimage-gencomfyui

H100 PCIe 80 GB

Updated Nov 22, 2026
VRAM
80 GB
Bandwidth
2040 GB/s
TDP
350 W
MSRP
-
Category
gpu

Local inference speed

llama.cpp decode speed (tok/s) - estimated

Workloadtok/s
LLaMA 3 8B Q4238.7
LLaMA 3 8B F16108.4
LLaMA 3 70B Q428.1

Estimated - Estimated from memory bandwidth (llama.cpp decode) · updated 2026-11-22

Stacks for the H100 PCIe 80 GB

Complete local setups that run on 80 GB

Run Qwen3.6-27B with vision, 256K context, and hybrid thinking locally

beginner~15 min

High-speed local text generation

advanced~35 min

AI pair programming in your terminal with local models, fully private

intermediate~10 min

A private AI assistant on Windows with document RAG - no CLI, no Docker, all desktop GUI

beginner~15 min

A private AI coding assistant in your editor

beginner~20 min

Prototype and deploy multi-agent RAG applications visually with local models

intermediate~15 min

H100 PCIe 80 GBLink to this section

The H100 PCIe 80 GB is an NVIDIA GPU built on the Hopper architecture, released 2022-10-01. For running AI locally, the numbers that matter are its 80 GB of HBM2e and 2040 GB/s of memory bandwidth. VRAM decides which models fit at all; bandwidth sets how fast they generate text.

What you can run on 80 GBLink to this section

At Q4_K_M quantization (the usual local default), 80 GB holds models up to roughly 126B parameters, leaving headroom for context. On this card you can run, among others:

Larger models need a higher-VRAM card, a second GPU, or CPU offload (which is much slower). Check any specific model with the VRAM calculator, or see the full picture on what can I run.

Local LLM speed (LLaMA 3, llama.cpp)Link to this section

Single-stream token-generation throughput - estimated from memory bandwidth:

Model (quant)Speed on H100 PCIe 80 GB
Llama 3 8B (Q4_K_M)238.7 tok/s
Llama 3 8B (F16)108.4 tok/s
Llama 3 70B (Q4_K_M)28.1 tok/s

Because decode is memory-bandwidth bound, the 2040 GB/s figure is the best single predictor of chat speed on this card. Estimates are calibrated against measured RTX-40-series cards and are typically within ~15%.

Memory and powerLink to this section

  • VRAM: 80 GB HBM2e (5120-bit bus)
  • Bandwidth: 2040 GB/s
  • TDP: 350 W - a 750 W+ power supply is recommended
  • Process: 5 nm
  • Interface: PCIe 5.0 x16

Quantization and contextLink to this section

Quantization trades a little quality for a lot of VRAM. On 80 GB you can fit roughly a 126B model at Q4_K_M, about a 68B model at the higher-quality Q8, or a smaller model at full FP16. Longer context windows also consume VRAM (the KV cache grows with context length), so leave a few GB of headroom if you plan to use large prompts or many concurrent requests. For most chat and coding use, Q4_K_M on this card is the sweet spot between speed, quality, and the 80 GB budget.

How it comparesLink to this section

Similar cards for local AI, by VRAM and 8B-Q4 speed:

GPUVRAMBandwidthLlama 3 8B Q4
H100 PCIe 80 GB80 GB2040 GB/s238.7 tok/s
RTX 6000D84 GB1570 GB/s183.7 tok/s
RTX PRO 5000 72 GB Blackwell72 GB1340 GB/s156.8 tok/s
NVIDIA Jetson Thor (T4000)64 GB273.2 GB/s32 tok/s

Bottom lineLink to this section

The H100 PCIe 80 GB is best for llm-inference, image-gen, comfyui. With 24 GB+ it comfortably handles most open models, including 30B-class at Q4. If you need more, compare with RTX 6000D and RTX PRO 5000 72 GB Blackwell.

SourcesLink to this section

Specs and benchmarks last checked 2026-11-22. Verify current pricing before buying.

Frequently asked

Quick answers to common questions

How much VRAM does the H100 PCIe 80 GB have?

The H100 PCIe 80 GB has 80 GB of VRAM with 2040 GB/s memory bandwidth.

What local AI models can run on the H100 PCIe 80 GB?

The H100 PCIe 80 GB with 80 GB VRAM can run many models depending on quantization. Models up to ~123B params may fit at Q4_K_M. Use our VRAM calculator to check specific models.

Is the H100 PCIe 80 GB good for local AI inference?

H100 PCIe 80 GB is best for llm-inference, image-gen, comfyui. With ample VRAM it handles most open models well.

Where can I buy the H100 PCIe 80 GB?

Check our buy links above for the best current prices on Amazon, Newegg, and B&H. Prices vary by retailer and availability.

How does the H100 PCIe 80 GB compare to other GPUs?

H100 PCIe 80 GB has 80 GB VRAM and 2040 GB/s bandwidth. This puts it in the high-end category, suitable for most open models. Browse our hardware directory for side-by-side comparisons.

What power supply do I need for the H100 PCIe 80 GB?

The H100 PCIe 80 GB has a TDP of 350W. This requires a high-wattage PSU (850W+ recommended). Always check the manufacturer's recommendations for your specific build.

Nearby options

Similar hardware and models that fit