backburner

backburner is a local inference server - Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable.

Overview

Plug your iPhone into your MacBook with a 10 Gb/s USB-C cable and it helps run Qwen3.8-27B locally:

• Faster prefill (up to 64k context). For every batch of prompt tokens the Mac runs layers 1-40 and the iPhone runs 41-64 on its GPU, pipelined. Your agent waits less every time it reads a file or a tool result of more than ~512 tokens: 29-44% faster prefill at 16k-48k. - More context. A 24 GB Mac fits 64k tokens of 8-bit context next to the model. The iPhone holds the oldest part past that and computes attention over it: its GPU during prefill, its GPU and Neural Engine while writing. The server sizes the total from the phone's free memory at startup (196k-229k tokens at 8-bit on an iPhone 17 Pro Max). Tested end to end to 128k at 8-bit and 140k at 4-bit. - Same answers. Greedy output is token-identical with and without the phone (256/256 tokens at 8k and 32k; 32/32 at 140k).

The engine is a llama.cpp fork ('llama.cpp/', StayLameBro/backburner-llama.cpp) with its own Mac kernels (SME2, Metal fusions, DFlash2 speculative decoding). Those speed things up on the Mac alone too; the numbers below keep the two apart.

backburner is open-source, written primarily in Python, with 1,138 GitHub stars under the MIT license. It was last updated on 2026-10-27.

Key capabilities

From the project's documentation:

• Split prefill (llama.cpp/src/llama-split.cpp; the phone's tail server is in ios/Backburner). The Mac runs layers
• The iPhone's Neural Engine for old keys (phone-attn/pa-ane.mm). Old keys never change, so each 16,384-key page of a
• SME2 on the Mac CPU. The M4's SME units take ~30% of the rows of each big prefill matmul while the GPU does the rest
• DFlash2 speculative decoding with recurrent-state replay for the hybrid (GDN + attention) model, lossless speculative
• SSD prompt cache (scripts/proxy.py): a known system prompt is restored from disk instead of re-read.
• Memory. The model is loaded wired (--load-mode none) so macOS can't page it out; the token-embedding table is read

Install

A quick way to get started (always check the official docs for the latest):

curl -fsSL https://raw.githubusercontent.com/StayLameBro/backburner/main/install.sh | bash

How it fits a local-AI stack

backburner runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local inference servers in the directory:

• "BentoML"
• "candle-vllm"
• "CTranslate2"
• ExLlamaV2

Sources

• Source code & docs: StayLameBro/backburner

Stats from GitHub, 2026-10-30.