petals social preview
fine-tuning10,654MIT

petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

Updated Oct 22, 2026
Website GitHub
Platforms
macos, linux, windows, docker
Pricing
free-open-source
Status
active
License
MIT

What it does

Core capabilities at a glance

  • Bloom
  • Chatbot
  • Distributed Systems
  • Falcon
  • GPT
  • Guanaco
  • Language Models
  • Large Language Models

Deep dive

The full breakdown - performance, comparisons, and setup

petals

petals is a fine-tuning toolkit - 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading.

Overview

Run large language models at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

Generate text with distributed Llama 3.1 (up to 405B), Mixtral (8x22B), Falcon (40B+) or BLOOM (176B) and fine‑tune them for your own tasks — right from your desktop computer or Google Colab:

🦙 Want to run Llama? Request access to its weights, then run 'huggingface-cli login' in the terminal before loading the model. Or just try it in our chatbot app.

🔏 Privacy. Your data will be processed with the help of other people in the public swarm. Learn more about privacy here. For sensitive data, you can set up a private swarm among people you trust.

Petals is a community-run system — we rely on people sharing their GPUs. You can help serving one of the available models or host a new model from 🤗 Model Hub!

As an example, here is how to host a part of Llama 3.1 (405B) Instruct on your GPU:

🦙 Want to host Llama? Request access to its weights, then run 'huggingface-cli login' in the terminal before loading the model.

🐧 Linux + Anaconda. Run these commands for NVIDIA GPUs (or follow this for AMD):

🐋 Docker. Run our Docker image for NVIDIA GPUs (or follow this for AMD):

petals is open-source, written primarily in Python, with 10,654 GitHub stars under the MIT license. The latest release is v2.2.0 (2023-09-06).

Key capabilities

From the project's documentation:

  • Launch a private swarm: guide
  • Run a custom model: guide

Install

A quick way to get started (always check the official docs for the latest):

pip install git+https://github.com/bigscience-workshop/petals

How it fits a local-AI stack

petals runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related fine-tuning toolkits in the directory:

Sources

Stats from GitHub, 2026-10-22.

Frequently asked

Quick answers to common questions

What is petals?

petals is a fine-tuning tool for local AI workloads. 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

Is petals free and open source?

Yes, petals has 10,654 GitHub stars and is licensed under MIT. You can self-host it for free on macos, linux, windows, docker.

What platforms does petals support?

petals runs on macos, linux, windows, docker.

What hardware do I need for petals?

The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. petals has 10,654 GitHub stars and an active community.

Does petals support GPU acceleration?

petals supports GPU acceleration via CUDA, Metal, or Vulkan depending on your platform. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.

What are the best alternatives to petals?

Popular alternatives include other fine-tuning tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.

How much does petals cost?

petals is free-open-source. It is completely free and open source to self-host.

Pairs well with

Complementary tools, models, and hardware