MTPLX

MTPLX is a local inference server - The fastest way to run Qwen 3.8 Flash Next and Qwen 3.8 27B on a Mac: 125 tok/s in OpenCode on an M5 Max. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

Overview

MTPLX is a native Mac app and a command line that runs local language models on Apple Silicon with the model's own multi-token prediction (MTP) heads. It runs Qwen 3.8 Flash Next, the 125B mixture of experts, and Qwen 3.8 27B, plus Qwen 3.6, Qwen 3.5 and Gemma 4. The model drafts several tokens ahead of itself, one batched forward pass verifies the draft, and tokens are committed through exact rejection sampling with residual correction. The output distribution is the model's own at any temperature, and decode runs at around twice the speed of plain decoding: measured 1.6x on a 16 GB M4 Mac mini and 2.24x on an M5 Max.

Every number below was measured on a MacBook Pro M5 Max with 128 GB, fans verified at maximum, sampled at the model's own settings. Conditions and sources for every row, with the raw logs where they exist, are at mtplx.com/benchmarks.

MTPLX is open-source, written primarily in Python, with 2,374 GitHub stars under the Apache 2.0 license. The latest release is v2.11.3 (2026-09-17).

Key capabilities

From the project's documentation:

• Youssofal/Qwen3.8-Flash-Next-MTPLX-Bare-Speed: flat 4-bit, the quickest build. 106.3 GB download, about 74 GB resident.
• Youssofal/Qwen3.8-27B-MTPLX-Bare-Speed: flat 4-bit, 16.0 GB, 17.0 GB peak. Quickest chat speeds.
• Not an external-drafter system. The drafter is the target model's own MTP heads.
• Not a greedy-argmax trick. Acceptance is exact rejection sampling, correct at any temperature.
• Not a CUDA project. MTPLX is MLX-native and Apple Silicon first. For Linux, use vLLM.

Install

A quick way to get started (always check the official docs for the latest):

pip install mtplx

How it fits a local-AI stack

MTPLX runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local inference servers in the directory:

• "BentoML"
• "candle-vllm"
• ExLlamaV2
• "flashinfer"

Sources

• Source code & docs: youssofal/MTPLX
• Official website: https://mtplx.com

Stats from GitHub, 2026-09-19.