llama-cpp-python social preview
other10,653MIT

llama-cpp-python

Python bindings for llama.cpp

Updated Oct 22, 2026
Website GitHub Docs
Platforms
macos, linux, windows, web
Pricing
free-open-source
Status
active
License
MIT

Deep dive

The full breakdown - performance, comparisons, and setup

llama-cpp-python

llama-cpp-python is a local-AI tool - Python bindings for llama.cpp.

Overview

Simple Python bindings for @ggerganov's 'llama.cpp' library. This package provides:

  • Low-level access to C API via 'ctypes' interface. - High-level Python API for text completion - OpenAI-like API - LangChain compatibility - LlamaIndex compatibility - OpenAI compatible web server - Local Copilot replacement - Function Calling support - Vision API support - Multiple Models

Documentation is available at https://llama-cpp-python.readthedocs.io/en/latest.

  • Python 3.8+ - C compiler - Linux: gcc or clang - Windows: Visual Studio or MinGW - MacOS: Xcode

This will also build 'llama.cpp' from source and install it alongside this python package.

If this fails, add '--verbose' to the 'pip install' see the full cmake build log.

It is also possible to install a pre-built wheel with basic CPU support.

'llama.cpp' supports a number of hardware acceleration backends to speed up inference as well as backend specific options. See the llama.cpp README for a full list.

All 'llama.cpp' cmake build options can be set via the 'CMAKE_ARGS' environment variable or via the '--config-settings / -C' cli flag during installation.

They can also be set via 'pip install -C / --config-settings' command and saved to a 'requirements.txt' file:

Below are some common backends, their build commands and any additional environment variables required.

llama-cpp-python is open-source, written primarily in Python, with 10,653 GitHub stars under the MIT license. The latest release is v0.3.36-hip-radeon (2026-10-01).

Key capabilities

From the project's documentation:

  • Low-level access to C API via ctypes interface.
  • High-level Python API for text completion
  • OpenAI compatible web server
  • Linux: gcc or clang
  • Windows: Visual Studio or MinGW
  • CUDA Version is 11.8, 12.1, 12.2, 12.3, 12.4, 12.5, 13.0 or 13.2

Install

A quick way to get started (always check the official docs for the latest):

pip install llama-cpp-python

How it fits a local-AI stack

llama-cpp-python runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local-AI tools in the directory:

Sources

Stats from GitHub, 2026-10-22.

Frequently asked

Quick answers to common questions

What is llama-cpp-python?

llama-cpp-python is a other tool for local AI workloads. Python bindings for llama.cpp

Is llama-cpp-python free and open source?

Yes, llama-cpp-python has 10,653 GitHub stars and is licensed under MIT. You can self-host it for free on macos, linux, windows, web.

What platforms does llama-cpp-python support?

llama-cpp-python runs on macos, linux, windows, web.

What hardware do I need for llama-cpp-python?

The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. llama-cpp-python has 10,653 GitHub stars and an active community.

Does llama-cpp-python support GPU acceleration?

llama-cpp-python supports GPU acceleration via CUDA, Metal, or Vulkan depending on your platform. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.

What are the best alternatives to llama-cpp-python?

Popular alternatives include other other tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.

How much does llama-cpp-python cost?

llama-cpp-python is free-open-source. It is completely free and open source to self-host.

Pairs well with

Complementary tools, models, and hardware