StartLux-Prism

StartLux-PrismLink to this section

StartLux-Prism is a local inference server - Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.

OverviewLink to this section

Fast, exact inference for StartLux-Decision models on NVIDIA GPUs. StartLux-Prism serves StartLux-Decision v1 and v1.1 AutoThink (4B, 9B, 27B) straight from their released GGUF files, behind the TypeSafe '/v1/systemone' API, on GPUs from Ampere to Blackwell, one or several. Like a prism that splits light into its spectrum, it turns each question into a calibrated probability for every option.

StartLux-Decision-v1.1-27B-AutoThink on one H200: StartLux-Prism int8 against the official engine in FP8 (vLLM 0.31).

StartLux-Decision-v1-27B on one H200: StartLux-Prism int8 against the official engine in FP8 (vLLM 0.31).

The same StartLux-Decision v1.1 27B checkpoint answers 24 structured questions about one video in a single request: eight frames sampled from the clip (640x360), one H200, 8-bit on both sides. The official engine on vLLM 0.31 (FP8, its launcher's flags) took 0.72 s; StartLux-Prism int8 took 0.21 s (3.38x faster). Both engines ran on the same GPU, one after the other. The animation replays each at its measured speed: end-to-end HTTP time, median of 5 runs, model loading excluded. Watch the MP4. Video: Usman AbdulrasheedGambo / Pexels.

StartLux-Prism is open-source, written primarily in Python, with 330 GitHub stars under the Open Source license. It was last updated on 2026-10-27.

Key capabilitiesLink to this section

From the project's documentation:

• Faster than the official engine. Serving the 27B on one H200, StartLux-Prism answers a short question 2.8× and a
• The official engine's accuracy. Prompts, read-outs, thoughts and decision rules are the model's own; only the
• Reproducible to the bit. int8 GEMMs sum exactly in int32, so a request receives the same probabilities whatever
• All of v1.1. AutoThink thoughts are decoded on the GPU (95 tokens/s for one sequence, 3,832 for 64 at once);
• The released files as they are. Q8_0 files run as int8 (W8A8) and Q4_K_M files on their own 4-bit weights
• One GPU or several. Data parallelism for throughput (3.92× on four H200s) and tensor parallelism (--tp 2 or

InstallLink to this section

A quick way to get started (always check the official docs for the latest):

pip install -r requirements.txt # PyTorch 2.14.1 for CUDA 13, Triton, nvcc 13.0

How it fits a local-AI stackLink to this section

StartLux-Prism runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local inference servers in the directory:

• "airllm"
• "Atomic-Chat"
• "backburner"
• "BentoML"

SourcesLink to this section

• Source code & docs: StartLuxLabs/StartLux-Prism
• Official website: https://huggingface.co/startlux-models

Stats from GitHub, 2026-11-07.