inference-optimisation-skill
Agent skill for hill-climbing local LLM serving speed (decode, speculative decoding, prefill, KV, startup, TTFT) on any hardware and quantization
What it does
Core capabilities at a glance
- Agent Skill
- Benchmarking
- Claude Code
- Inference
- Speculative Decoding
Deep dive
The full breakdown - performance, comparisons, and setup
inference-optimisation-skillLink to this section
inference-optimisation-skill is a local inference server - Agent skill for hill-climbing local LLM serving speed (decode, speculative decoding, prefill, KV, startup, TTFT) on any hardware and quantization.
OverviewLink to this section
An agent skill for hill-climbing the speed of a local LLM server on any hardware and any quantization. It covers:
It works under a memory cap, and every result has to survive a full-context stability check.
It encodes how we actually got these speedups. It is not a list of flags. The workflow:
- Compute the theoretical ceiling. 2. Change one thing at a time. 3. Measure against interleaved controls. 4. Log every result, including the failures. 5. Gate on long-context retrieval. 6. Qualify a clean, pinned build before publishing.
It works with Claude Code and any agent that loads 'SKILL.md'-style skills. Any model can also follow it from 'PROMPT.md'.
Results from the hill-climbs this skill was distilled from. Every number was measured on the hardware named; see the linked sources for methods and raw data.
- local-ai-registry: - Ornith B390 recipe, with the full hill-climb log in PR #453 - the RTX 3090 and H200 recipes - trellis-serve: - 'xpu/README.md', 'xpu/docs/PROGRESS.md' and 'xpu/docs/SGLANG.md' for B70 - 'cuda/README.md' for the RTX 3090 - qwen-serve
That run is the one this skill grew out of. Its ceiling calculation said:
inference-optimisation-skill is open-source, written primarily in Python, with 1,366 GitHub stars under the MIT license. It was last updated on 2026-11-16.
Key capabilitiesLink to this section
From the project's documentation:
- single-user and concurrent decode
- startup time and time to first token
- the RTX 3090 and H200 recipes
- xpu/README.md, xpu/docs/PROGRESS.md and xpu/docs/SGLANG.md for B70
- cuda/README.md for the RTX 3090
- Prefill at 256K: we reached 628 tok/s against a ~670–684 tok/s ceiling.
How it fits a local-AI stackLink to this section
inference-optimisation-skill runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local inference servers in the directory:
SourcesLink to this section
- Source code & docs: sybil-solutions/inference-optimisation-skill
Stats from GitHub, 2026-11-19.
Frequently asked
Quick answers to common questions
What is inference-optimisation-skill?
inference-optimisation-skill is a inference-server tool for local AI workloads. Agent skill for hill-climbing local LLM serving speed (decode, speculative decoding, prefill, KV, startup, TTFT) on any hardware and quantization
Is inference-optimisation-skill free and open source?
Yes, inference-optimisation-skill has 1,366 GitHub stars and is licensed under MIT. You can self-host it for free on .
What hardware do I need for inference-optimisation-skill?
The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. inference-optimisation-skill has 1,366 GitHub stars and an active community.
Does inference-optimisation-skill support GPU acceleration?
inference-optimisation-skill's GPU support depends on your specific setup. Check the documentation for details. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.
What are the best alternatives to inference-optimisation-skill?
Popular alternatives include other inference-server tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.
How much does inference-optimisation-skill cost?
inference-optimisation-skill is free-open-source. It is completely free and open source to self-host.
Pairs well with
Complementary tools, models, and hardware