smolmaxxing social preview
other538MIT

smolmaxxing

maxxing the ability to run different LLMs on the smallest devices possible

Updated Nov 13, 2026
Website GitHub
Platforms
linux
Pricing
free-open-source
Status
active
License
MIT

What it does

Core capabilities at a glance

  • Edge AI
  • Llama CPP
  • Llama CPP Alternative
  • Local LLM
  • Local LLM Support
  • ON Device
  • Quantization
  • Quantization Algorithms

Deep dive

The full breakdown - performance, comparisons, and setup

smolmaxxingLink to this section

smolmaxxing is a local-AI tool - maxxing the ability to run different LLMs on the smallest devices possible.

OverviewLink to this section

Run the biggest language model you can on the smallest hardware you own.

TV boxes, old tablets, single-board computers and routers all have a CPU, some RAM and flash storage that sit idle most of the day. smolmaxxing finds the biggest model each of them can really run, then pushes it further with custom kernels, careful memory use and weights streamed from flash. Every number in this repo was measured on the device itself.

The TV box and the iPad run 8B and 16B mixture-of-experts models on about 1 GB of RAM, and the phone runs 35B and 80B ones with about 3 GB free. They all keep only part of each model in memory and read the rest from flash as it's needed.

More devices (single-board computers, routers, more phones) are coming. The techniques aren't tied to one box: the 1-bit kernel needs only basic NEON, and the flash streaming works on any Linux or Android device with more model than RAM.

Got a small device? Run a model on it and submit your numbers: we'll add them to the results.

smolmaxxing is open-source, written primarily in C, with 538 GitHub stars under the MIT license. It was last updated on 2026-11-13.

Key capabilitiesLink to this section

From the project's documentation:

  • Streaming for dense models bigger than RAM, at the flash's full speed.
  • TV box or another ARMv8.2 Android device: docs/tv-box-setup.md
  • Jailbroken iPad on iOS 12: ipad-air/README.md
  • All model files and how to rebuild them: docs/models.md

How it fits a local-AI stackLink to this section

smolmaxxing runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local-AI tools in the directory:

SourcesLink to this section

Stats from GitHub, 2026-11-13.

Frequently asked

Quick answers to common questions

What is smolmaxxing?

smolmaxxing is a other tool for local AI workloads. maxxing the ability to run different LLMs on the smallest devices possible

Is smolmaxxing free and open source?

Yes, smolmaxxing has 538 GitHub stars and is licensed under MIT. You can self-host it for free on linux.

What platforms does smolmaxxing support?

smolmaxxing runs on linux.

What hardware do I need for smolmaxxing?

The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. smolmaxxing has 538 GitHub stars and an active community.

Does smolmaxxing support GPU acceleration?

smolmaxxing supports GPU acceleration via CUDA, Metal, or Vulkan depending on your platform. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.

What are the best alternatives to smolmaxxing?

Popular alternatives include other other tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.

How much does smolmaxxing cost?

smolmaxxing is free-open-source. It is completely free and open source to self-host.

Pairs well with

Complementary tools, models, and hardware