Rapid-MLX

Rapid-MLX is a local inference server - The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.

Overview

Run local AI models on your Mac — no cloud, no API costs. Works with Cursor, Claude Code, and any OpenAI-compatible app.

pip install → serve Gemma 4 26B → chat + tool calling → works with PydanticAI, LangChain, Aider, and more.

Single-user end-to-end throughput (B=1: one request at a time, 256 max output tokens, 'output_tokens / wall-clock' incl. first-token latency), median of 3 rounds. 'chat_template_kwargs.enable_thinking=False' passed where the engine honours it. Tested on M3 Ultra 256 GB / rapid-mlx v0.6.80. ¹ carried over from 2026-04 bench — disk-constrained on this refresh.

Rapid-MLX is open-source, written primarily in Python, with 2,690 GitHub stars under the Apache 2.0 license. The latest release is v0.6.82 (2026-06-07).

Key capabilities

From the project's documentation:

• tok/s (tokens per second) — roughly how many words the AI generates per second. Higher = faster.
• 4bit / 8bit — compression levels for models. 4bit uses less memory (recommended); 8bit is higher quality.
• TTFT (Time To First Token) — how long before the AI starts responding.
• Tool calling — the AI can call functions in your code. Used by Cursor, Claude Code, and coding assistants.
• Subcommand names (serve / chat / agents / bench / doctor)
• Model alias names (qwen3.5-9b) or canonical HF repo IDs (mlx-community/...) — local paths are redacted to

Install

A quick way to get started (always check the official docs for the latest):

pip install rapid-mlx

How it fits a local-AI stack

Rapid-MLX runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local inference servers in the directory:

• ExLlamaV2
• llama.cpp
• LocalAI
• Ollama

Sources

• Source code & docs: raullenchai/Rapid-MLX
• Official website: https://pypi.org/project/rapid-mlx

Stats from GitHub, 2026-06-08.