mlx-serve

mlx-serve is a speech (TTS/STT) tool - Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Includes MLX Core macOS app with chat, agent mode, and tool calling.

Overview

OpenAI- and Anthropic-compatible local inference for Apple Silicon — MLX and GGUF — faster than LM Studio on identical MLX weights. No Python. No cloud. No Electron.

ddalcu.github.io/mlx-serve · Download MLX Core.app · Changelog

mlx-serve is a native Zig server that runs any LLM on Apple Silicon — MLX-format models and every GGUF on HuggingFace (Qwen, Llama, Mistral, Gemma, DeepSeek V4 Flash, thousands more). It exposes OpenAI-compatible and Anthropic-compatible HTTP APIs out of the box, so the same 'http://localhost:11234' works with Claude Code, the OpenAI SDK, Continue, Cursor, Open WebUI, and anything else that speaks one of those wires. Beyond text, the same server generates images, video, music, speech (with voice cloning), and 3D models — all natively on MLX. Ships with MLX Core, a macOS menu-bar app with chat, agent mode, MCP tool calling, and model management.

Download MLX Core.app — latest release for macOS (Apple Silicon)

Short names, 'org/repo' HuggingFace ids, and 'name:tag' all work. And because mlx-serve speaks the Ollama API ('/api/chat', '/api/generate', '/api/tags', '/api/embed', '/api/pull', …) alongside OpenAI and Anthropic, your existing Ollama-connected tools — Raycast, Obsidian, Enchanted, Open WebUI, 'ollama-python'/'js' — work unchanged: point them at 'http://localhost:11234' and keep your workflow, on a faster engine.

mlx-serve is open-source, written primarily in Zig, with 340 GitHub stars under the MIT license. The latest release is v26.7.9 (2026-07-17).

Key capabilities

From the project's documentation:

• Vision — Gemma 4 SigLIP encoder; send images via image_url content blocks.
• Reasoning / thinking — full streaming of thinking tokens as reasoning_content.
• No Python — single Zig binary, no pip, no venv. The MLX Core app ships everything signed and notarized.
• Editable system prompt + persistent memory — ~/.mlx-serve/system-prompt.md and ~/.mlx-serve/memory.md.
• Server management — start/stop, live log buffer, restart-on-flag-change banner.
• Zig 0.16+ (only if building from source)

Install

A quick way to get started (always check the official docs for the latest):

brew install --cask mlx-core # GUI menu bar app

How it fits a local-AI stack

mlx-serve runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related speech (TTS/STT) tools in the directory:

• "cactus"
• "claude-code-local"
• "dograh"
• "DreamServer"

Sources

• Source code & docs: ddalcu/mlx-serve
• Official website: http://mlxserve.com/

Stats from GitHub, 2026-07-18.