private-gpt

private-gpt is a RAG toolkit - Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference server.

Overview

PrivateGPT is the open-source API layer that turns local models into production AI applications.

Running a model locally is only the first step. To build useful AI applications you need a set of higher-level building blocks. PrivateGPT provides that layer as an open-source API following the Claude API model — so you can build private AI products without rebuilding the same backend primitives from scratch, and without depending on cloud APIs.

Production-tested: PrivateGPT powers Zylon, the on-premise AI platform providing Private AI to enterprises across the globe.

PrivateGPT ships a built-in workbench UI for testing and demos, available at '/ui'. The API is the actual product.

Prerequisites: You need a running OpenAI-compatible LLM server. Ollama is the easiest starting point.

Go to http://localhost:8080/ui. The API is at 'http://localhost:8080' and follows the Anthropic API spec.

• Sending messages. - Selecting models from /v1/models. - Uploading documents. - Testing retrieval with citations. - Enabling tools per chat. - Configuring databases, MCP connectors, skills, and custom tools. - Inspecting requests and responses through the API Debugger.

This UI is a demonstrator, not the core product. Developers are expected to build their own applications on top of the API. That said, the UI is intentionally polished enough for demos, videos, internal pilots, and quick local usage.

private-gpt is open-source, written primarily in Python, with 57,556 GitHub stars under the Apache 2.0 license. The latest release is v1.0.1 (2026-06-18).

Key capabilities

From the project's documentation:

• Standard messages API (streaming, async, token counting)
• File and artifact ingestion
• Retrieval with citations and agentic RAG
• Built-in tools mirroring the Claude API (web search, web fetch, code execution)
• Custom tools and MCP connectors
• Structured access to databases and CSVs

Install

A quick way to get started (always check the official docs for the latest):

brew install private-gpt

How it fits a local-AI stack

private-gpt runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related RAG toolkits in the directory:

• AnythingLLM
• "graphify"
• Haystack
• Khoj

Sources

• Source code & docs: zylon-ai/private-gpt
• Official website: https://www.zylon.ai/private-gpt

Stats from GitHub, 2026-10-01.