What it does
Core capabilities at a glance
- Chatgpt
- Code Llama
- Codellama
- GPT
- GPT 4
- Gpt4all
- Llama
- Llama 2
Deep dive
The full breakdown - performance, comparisons, and setup
llama-gpt
llama-gpt is a chat UI - A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support.
Overview
A self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.
- Demo 2. Supported Models 3. How to install - On umbrelOS home server - On M1/M2 Mac - Anywhere else with Docker - Kubernetes 4. OpenAI-compatible API 5. Benchmarks 6. Roadmap and contributing 7. Acknowledgements
Currently, LlamaGPT supports the following models. Support for running custom models is on the roadmap.
Running LlamaGPT on an umbrelOS home server is one click. Simply install it from the Umbrel App Store.
You can run LlamaGPT on any x86 or arm64 system. Make sure you have Docker installed.
First, make sure you have a running Kubernetes cluster and 'kubectl' is configured to interact with it.
Thanks to llama-cpp-python, a drop-in replacement for OpenAI API is available at 'http://localhost:3001'. Open http://localhost:3001/docs to see the API documentation.
We've tested LlamaGPT models on the following hardware with the default system prompt, and user prompt: "How does the universe expand?" at temperature 0 to guarantee deterministic results. Generation speed is averaged over the first 10 generations.
Feel free to add your own benchmarks to this table by opening a pull request.
We're looking to add more features to LlamaGPT. You can see the roadmap here. The highest priorities are:
llama-gpt is open-source, written primarily in TypeScript, with 10,918 GitHub stars under the MIT license. It was last updated on 2024-04-23.
Key capabilities
From the project's documentation:
- On umbrelOS home server
- Anywhere else with Docker
- Moving the model out of the Docker image and into a separate volume.
- Add Metal support for M1/M2 Macs.
- Add support for Code Llama models.
- Add CUDA support for NVIDIA GPUs.
How it fits a local-AI stack
llama-gpt runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related chat UIs in the directory:
Sources
- Source code & docs: getumbrel/llama-gpt
- Official website: https://apps.umbrel.com/app/llama-gpt
Stats from GitHub, 2026-10-22.
Frequently asked
Quick answers to common questions
What is llama-gpt?
llama-gpt is a chat-ui tool for local AI workloads. A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
Is llama-gpt free and open source?
Yes, llama-gpt has 10,918 GitHub stars and is licensed under MIT. You can self-host it for free on macos, docker.
What platforms does llama-gpt support?
llama-gpt runs on macos, docker.
What hardware do I need for llama-gpt?
The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. llama-gpt has 10,918 GitHub stars and an active community.
Does llama-gpt support GPU acceleration?
llama-gpt supports GPU acceleration via CUDA, Metal, or Vulkan depending on your platform. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.
What are the best alternatives to llama-gpt?
Popular alternatives include other chat-ui tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.
How much does llama-gpt cost?
llama-gpt is free-open-source. It is completely free and open source to self-host.
Pairs well with
Complementary tools, models, and hardware