mlxcel

mlxcel is a local inference server - High-performance LLM/VLM inference runtime and server for Apple Silicon / NVIDIA CUDA devices.

Overview

High-performance LLM/VLM inference runtime and server for Apple Silicon / NVIDIA CUDA-compatible / (experimental) OpenXLA-compatible devices. The CLI and server are implemented in Rust and execute models through native MLX C++ bindings. Linux/CUDA builds are supported as a secondary target.

'mlxcel' provides a Rust command-line runtime and an OpenAI-compatible model server for MLX-format checkpoints. Loading, scheduling, and inference stay in one native process while model execution goes through MLX C++ bindings. It runs a broad range of text and vision-language model families directly from mlx-community checkpoints, with no conversion step.

The project started as work on structural model fine-tuning and has grown into a general-purpose serving runtime for local and small-cluster inference.

mlxcel is open-source, written primarily in Rust, with 310 GitHub stars under the Apache 2.0 license. The latest release is v0.5.1 (2026-08-15).

Key capabilities

From the project's documentation:

• -m/--model accepts --revision on generate, run, serve, inspect, and mlxcel-server.
• 3 more new vision-language families: LocateAnything grounding, Falcon-OCR early fusion, and Jina VLM.
• 8 new text model families, including Ling/Bailing, OpenELM, TeleChat3, DBRX, Phixtral, AFMoE, and Klear.
• Faster sampling. Sort-free top-p and Gumbel-max sampling improve performance, but may change fixed-seed outputs.
• Xcode Command Line Tools
• Apple Metal toolchain component

Install

A quick way to get started (always check the official docs for the latest):

brew install mlxcel

How it fits a local-AI stack

mlxcel runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local inference servers in the directory:

• "BentoML"
• "candle-vllm"
• ExLlamaV2
• "flashinfer"

Sources

• Source code & docs: lablup/mlxcel

Stats from GitHub, 2026-08-17.