Soup

Soup is a fine-tuning toolkit - Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

Overview

Fine-tune and post-train LLMs in one command. No SSH, no config hell.

Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.

Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B-Instruct + NF4 at 119.6 tok/s, 3.32 GB peak — bit-exact against a normal resident run, and reproduced independently on an H100 at 113.00 tok/s in the same 3.32 GB. (The tok/s figure was measured on v0.72.2, before the v0.73.0 correctness repair that cost −4.8% at 32B; it has not been re-run on a 4 GB card since.) Opt-in ('stream_layers: true') and still BETA — how it works · all measurements · paper · check it yourself on a free Colab T4 (caps the process to 4 GB, then asserts a streamed model is bit-identical to a normal one)

Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that.

Soup is open-source, written primarily in Python, with 1,999 GitHub stars under the Apache 2.0 license. The latest release is v0.73.2 (2026-08-15).

Key capabilities

From the project's documentation:

• Zero SSH. Never SSH into a broken GPU box again.
• One config. A simple YAML file is all you need.
• Auto everything. Batch size, GPU detection, quantization — handled.
• Works locally. Train on your own GPU with QLoRA. No cloud required.
• A suite scored 0.225 for a model that got it right 40/40. mini_tool_call was
• New: a benign-prompt axis. Leg 2 flagged a drop in refusal rate and had no reverse,

Install

A quick way to get started (always check the official docs for the latest):

pip install soup-cli

How it fits a local-AI stack

Soup runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related fine-tuning toolkits in the directory:

• Axolotl
• "gitleaks"
• LitGPT
• LLaMA-Factory

Sources

• Source code & docs: MakazhanAlpamys/Soup
• Official website: https://trysoup.dev

Stats from GitHub, 2026-08-17.