unilm social preview
fine-tuning22,230MIT

unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Updated Oct 12, 2026
Website GitHub
Platforms
Pricing
free-open-source
Status
active
License
MIT

What it does

Core capabilities at a glance

  • Beit
  • Beit 3
  • Bitnet
  • Deepnet
  • Document AI
  • Foundation Models
  • Kosmos
  • Kosmos 1

Deep dive

The full breakdown - performance, comparisons, and setup

unilm

unilm is a fine-tuning toolkit - Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities.

Overview

We are hiring at all levels (including FTE researchers and interns)! If you are interested in working with us on Foundation Models (aka large-scale pre-trained models) and General AI, NLP, MT, Speech, Document AI and Multimodal AI, please send your resume to fuwei@microsoft.com .

Fundamental research to develop new architectures for foundation models and AI, focusing on modeling generality and capability, as well as training stability and efficiency.

The Big Convergence - Large-scale self-supervised pre-training across (predictive and generative), (100+ languages), and (language, image, audio, layout/format + language, vision + language, audio + language, etc.)

General technology for enabling AI capabilities w/ LLMs and MLLMs.

Curating General, Code, Math, and QA Data for Large Language Models.

unilm is open-source, written primarily in Python, with 22,230 GitHub stars under the MIT license. The latest release is yoco.v0 (2024-05-09).

Key capabilities

From the project's documentation:

  • December, 2024: RedStone was released!
  • November, 2022: TorchScale 0.1.1 was released!
  • November, 2022: TrOCR was accepted by AAAI 2023.
  • [Model Release] July, 2021: BEiT - Towards BERT moment for CV
  • March, 2021: InfoXLM was accepted by NAACL 2021.
  • September, 2020: MiniLM was accepted by NeurIPS 2020.

How it fits a local-AI stack

unilm runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related fine-tuning toolkits in the directory:

Sources

Stats from GitHub, 2026-10-12.

Frequently asked

Quick answers to common questions

What is unilm?

unilm is a fine-tuning tool for local AI workloads. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Is unilm free and open source?

Yes, unilm has 22,230 GitHub stars and is licensed under MIT. You can self-host it for free on .

What hardware do I need for unilm?

The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. unilm has 22,230 GitHub stars and an active community.

Does unilm support GPU acceleration?

unilm's GPU support depends on your specific setup. Check the documentation for details. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.

What are the best alternatives to unilm?

Popular alternatives include other fine-tuning tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.

How much does unilm cost?

unilm is free-open-source. It is completely free and open source to self-host.

Pairs well with

Complementary tools, models, and hardware