TurboTransformers social preview
inference-server1,551Other

TurboTransformers

a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.

Updated Nov 21, 2026
Platforms
Pricing
free-open-source
Status
active
License
Other

What it does

Core capabilities at a glance

  • Albert
  • Bert
  • Decoder
  • Gpt2
  • GPU
  • Huggingface Transformers
  • Inference
  • Machine Translation

Deep dive

The full breakdown - performance, comparisons, and setup

TurboTransformersLink to this section

TurboTransformers is a local inference server - a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.

OverviewLink to this section

Make transformers serving fast by adding a turbo to your inference engine!

The WeChat AI open-sourced TurboTransformers with the following characteristics.

  1. Supporting both Transformers Encoder and Decoder. 3. Supports Variable Length inputs. No time-consuming offline tuning is required. You can change batch size and sequence length at real-time. 3. Excellent CPU / GPU performance. 4. Perfect Usibility. TurboTransformers supports python and C++ APIs. 5. Smart Batching. Minimize zero-padding overhead for a batch of requests of different lengths. It can be used as a plugin for pytorch. Tthe end-to-end acceleration is obtained by adding a few lines of python code.

TurboTransformers has been applied to multiple online BERT service scenarios in Tencent. For example, It brings 1.88x acceleration to the WeChat FAQ service, 2.11x acceleration to the public cloud sentiment analysis service, and 13.6x acceleration to the QQ recommendation system. Moreover, it has already been applied to build services such as Chitchating, Searching, and Recommendation.

The following table is a comparison of TurboTransformers and related work.

Note that the building scripts only apply to specific OS and software (Pytorch, OpenNMT, transformers, etc.) versions. Please adjust them according to your needs.

  1. Run benchmark (optional) in docker, compare with pytorch, torch-JIT, onnxruntime

TurboTransformers is open-source, written primarily in C++, with 1,551 GitHub stars under the Other license. The latest release is v0.5.1 (2020-11-25).

InstallLink to this section

A quick way to get started (always check the official docs for the latest):

docker run -it --rm --name=turbort -v $PWD:/workspace your_image_name /bin/bash

How it fits a local-AI stackLink to this section

TurboTransformers runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related local inference servers in the directory:

SourcesLink to this section

Stats from GitHub, 2026-10-30.

Frequently asked

Quick answers to common questions

What is TurboTransformers?

TurboTransformers is a inference-server tool for local AI workloads. a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.

Is TurboTransformers free and open source?

Yes, TurboTransformers has 1,551 GitHub stars and is licensed under Other. You can self-host it for free on .

What hardware do I need for TurboTransformers?

The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. TurboTransformers has 1,551 GitHub stars and an active community.

Does TurboTransformers support GPU acceleration?

TurboTransformers's GPU support depends on your specific setup. Check the documentation for details. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.

What are the best alternatives to TurboTransformers?

Popular alternatives include other inference-server tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.

How much does TurboTransformers cost?

TurboTransformers is free-open-source. It is completely free and open source to self-host.

Pairs well with

Complementary tools, models, and hardware