textgen

textgen is a local-AI tool - Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.

Overview

A desktop app for local LLMs. Open source, no telemetry. Text, vision, tool-calling, web search. UI + API.

Portable builds for Linux, Windows, and macOS with CUDA, Vulkan, ROCm, and CPU-only options. All dependencies included. Compatible with GGUF (llama.cpp) models.

For additional backends (ExLlamaV3, Transformers), training, image generation, and extensions, see Installation.

• 'instruct' mode for instruction-following (like ChatGPT), and 'chat-instruct'/'chat' modes for talking to custom characters. Prompts are automatically formatted with Jinja2 templates. - Vision (multimodal): Attach images to messages for visual understanding (tutorial). - File attachments: Upload text files, PDF documents, and .docx documents to talk about their contents. - Edit messages, navigate between message versions, and branch conversations at any point. - Notebook tab for free-form text generation outside of chat turns.

• Multiple backends: llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3, and TensorRT-LLM. Switch between backends and models without restarting. - OpenAI/Anthropic-compatible API: Chat, Completions, and Messages endpoints with tool-calling support. Use as a local drop-in replacement for the OpenAI/Anthropic APIs (examples). - Tool-calling: Models can call custom functions during chat, including web search, page fetching, and math. Each tool is a single '.py' file. MCP servers are also supported (tutorial).

textgen is open-source, written primarily in Python, with 47,284 GitHub stars under the AGPL-3.0 license. The latest release is v4.9 (2026-05-20).

Key capabilities

From the project's documentation:

• File attachments: Upload text files, PDF documents, and .docx documents to talk about their contents.
• Edit messages, navigate between message versions, and branch conversations at any point.
• Notebook tab for free-form text generation outside of chat turns.
• 100% offline and private, with zero telemetry, external resources, or remote update requests.
• Dark/light themes, syntax highlighting for code blocks, and LaTeX rendering for mathematical expressions.
• Restart: run the same start_ script.

Install

A quick way to get started (always check the official docs for the latest):

pip install -r requirements/portable/requirements.txt --upgrade

How it fits a local-AI stack

textgen runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local-AI tools in the directory:

• Continue
• Mem0

Sources

• Source code & docs: oobabooga/textgen
• Official website: https://localbench.substack.com

Stats from GitHub, 2026-06-08.