omlx

omlx is a local inference server - LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar.

Overview

oMLX LLM inference, optimized for your Mac Continuous batching and tiered KV caching, managed directly from your menu bar.

Install · Quickstart · Features · Models · CLI Configuration · Benchmarks · oMLX.ai

Download the '.dmg' from Releases, drag to Applications, done. The app includes in-app auto-update, so future upgrades are just one click. The macOS app also installs a lightweight '~/.omlx/bin/omlx' CLI shim so terminal commands and Apple Shortcuts can control the app-managed server.

Launch oMLX from your Applications folder. The Welcome screen guides you through three steps - model directory, server start, and first model download. That's it. To connect OpenClaw, OpenCode, Codex, Hermes Agent, or Copilot, see Integrations.

The server discovers LLMs, VLMs, embedding models, and rerankers from subdirectories automatically. Any OpenAI-compatible client can connect to 'http://localhost:8000/v1'. A built-in chat UI is also available at 'http://localhost:8000/admin/chat'.

The service runs 'omlx serve' with zero-config defaults ('/.omlx/models', port 8000). 'omlx start', 'omlx stop', and 'omlx restart' are the portable lifecycle commands; Homebrew installs delegate them to 'brew services'. To customize, either set environment variables ('OMLX_MODEL_DIR', 'OMLX_PORT', etc.) or run 'omlx serve --model-dir /your/path' once to persist settings to '/.omlx/settings.json'.

Logs are written to two locations: - Service log: '$(brew --prefix)/var/log/omlx.log' (stdout/stderr) - Server log: '~/.omlx/logs/server.log' (structured application log)

omlx is open-source, written primarily in Python, with 16,209 GitHub stars under the Apache 2.0 license. The latest release is v0.4.2rc1 (2026-06-06).

Key capabilities

From the project's documentation:

• Service log: $(brew --prefix)/var/log/omlx.log (stdout/stderr)
• Server log: ~/.omlx/logs/server.log (structured application log)
• Hot tier (RAM): Frequently accessed blocks stay in memory for fast access.
• LRU eviction: Least-recently-used models are evicted automatically when memory runs low.
• Manual load/unload: Interactive status badges in the admin panel let you load or unload models on demand.
• Model pinning: Pin frequently used models to keep them always loaded.

Install

A quick way to get started (always check the official docs for the latest):

brew install omlx

How it fits a local-AI stack

omlx runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local inference servers in the directory:

• ExLlamaV2
• llama.cpp
• LocalAI
• Ollama

Sources

• Source code & docs: jundot/omlx
• Official website: https://omlx.ai

Stats from GitHub, 2026-06-08.