cactus social preview
inference-server14,721Other

cactus

Low-latency AI engine for mobile devices & wearables

Updated Nov 14, 2026
Platforms
macos
Pricing
free-open-source
Status
active
License
Other

What it does

Core capabilities at a glance

  • Android
  • ARM
  • Edge
  • Edge AI
  • Framework
  • IOS
  • Llamacpp
  • LLM Inference

Deep dive

The full breakdown - performance, comparisons, and setup

cactusLink to this section

cactus is a speech (TTS/STT) tool - Low-latency AI engine for mobile devices & wearables.

OverviewLink to this section

[![Docs][docs-shield]][docs-url] [![Website][website-shield]][website-url] [![GitHub][github-shield]][github-url] [![HuggingFace][hf-shield]][hf-url] [![Reddit][reddit-shield]][reddit-url] [![Blog][blog-shield]][blog-url]

  • Fast & accurate: fastest inference on ARM CPU, Cactus quants at 4-bit matches f16 - Low RAM: zero-copy memory mapping ensures 10x lower RAM use than other engines - Multimodal: one engine for speech, vision, and language models - Cloud fallback: automatically route requests to cloud models if needed - Model-Agnostic: Custom PyTorch models can be exported to the Cactus runtime.
  1. Cactus Compute, Inc. (YC S25) 2. UCLA's BruinAI 3. Char (YC S25) 4. Yale's AI Society 5. National University of Singapore's AI Society 6. UC Irvine's AI@UCI 7. Imperial College's AI Society 8. University of Pennsylvania's AI@Penn 9. University of Michigan Ann-Arbor MSAIL 10. University of Colorado Boulder's AI Club

N/B: Scroll all the way up and click the shields link for resources!

[docs-shield]: https://img.shields.io/badge/Docs-555?style=for-the-badge&logo=readthedocs&logoColor=white [docs-url]: https://cactus-compute.github.io/cactus/

[website-shield]: https://img.shields.io/badge/Website-555?style=for-the-badge&logo=safari&logoColor=white [website-url]: https://cactuscompute.com/

[github-shield]: https://img.shields.io/badge/GitHub-555?style=for-the-badge&logo=github&logoColor=white [github-url]: https://github.com/cactus-compute/cactus

[hf-shield]: https://img.shields.io/badge/HuggingFace-555?style=for-the-badge&logo=huggingface&logoColor=white [hf-url]: https://huggingface.co/Cactus-Compute

[reddit-shield]: https://img.shields.io/badge/Reddit-555?style=for-the-badge&logo=reddit&logoColor=white [reddit-url]: https://www.reddit.com/r/cactuscompute/

[blog-shield]: https://img.shields.io/badge/Blog-555?style=for-the-badge&logo=hashnode&logoColor=white [blog-url]: https://cactuscompute.com/blog

cactus is open-source, written primarily in C++, with 14,721 GitHub stars under the Other license. The latest release is v1.14 (2026-04-18).

Key capabilitiesLink to this section

From the project's documentation:

  • Fast & accurate: fastest inference on ARM CPU, Cactus quants at 4-bit matches f16
  • Low RAM: zero-copy memory mapping ensures 10x lower RAM use than other engines
  • Multimodal: one engine for speech, vision, and language models
  • Cloud fallback: automatically route requests to cloud models if needed
  • Model-Agnostic: Custom PyTorch models can be exported to the Cactus runtime.
  • Step 1: brew install cactus-compute/cactus/cactus

InstallLink to this section

A quick way to get started (always check the official docs for the latest):

brew install cactus-compute/cactus/cactus

How it fits a local-AI stackLink to this section

cactus runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related speech (TTS/STT) tools in the directory:

SourcesLink to this section

Stats from GitHub, 2026-06-08.

From the community

What users are saying

“Cactus is a framework - not the app itself. If you're looking for an Android demo, you can go to https://play.google.com/store/apps/details?id=com.rshemetsub...”

“1. Thanks for this! 2. We combine both on Cactus; rolling entropy method helps with per-token signal for early exit, while the probe runs on complete output. 3. Users hate this in production, drains battery on edge devices, but ultimately primarily works ok-ish on 7B+ models, we will share the results too.”

“This is cool! We are working on agentic browser (also launched today https://news.ycombinator.com/item?id=44523409 :)) Right now we have a desktop version with ollama support, but we want to build a mobile chromium fork with local LLM support. Will check out cactus!”

Frequently asked

Quick answers to common questions

What is cactus?

cactus is a inference-server tool for local AI workloads. Low-latency AI engine for mobile devices & wearables

Is cactus free and open source?

Yes, cactus has 14,721 GitHub stars and is licensed under Other. You can self-host it for free on macos.

What platforms does cactus support?

cactus runs on macos.

What hardware do I need for cactus?

The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. cactus has 14,721 GitHub stars and an active community.

Does cactus support GPU acceleration?

cactus supports GPU acceleration via CUDA, Metal, or Vulkan depending on your platform. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.

What are the best alternatives to cactus?

Popular alternatives include other inference-server tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.

How much does cactus cost?

cactus is free-open-source. It is completely free and open source to self-host.

Pairs well with

Complementary tools, models, and hardware