Forced aligners on news audio: MFA vs Qwen3-ForcedAligner vs Chirp 3

The hookLink to this section

A short clip lives or dies at its edges. Cut 40 ms early and the first word loses its consonant; cut late and you catch the breath before the next sentence. So before cutting news into shorts, we need word timestamps we can trust, and "trust" here has a very specific meaning: when we cut at a word boundary, does the cut land in silence?

The public answer is FA-Bench, which scores aligners against hand-marked word boundaries in TIMIT and Buckeye. MFA leads it. But that is read speech and interviews, scored at a strict 20 ms. News has music beds, stings and interpreters, and our real question is coarser and more practical.

The setup, in one pictureLink to this section

174 English news windows (VOA, White House)
   │ energy-based pause finder, no labels
   ▼
1,465 real pauses of at least 200 ms
   │ ASR transcript (not subtitles) → each aligner
   ▼
nearest word boundary to each pause: inside the pause, or not?
   ▼
cut-in-pause % · 95% CI bootstrapped by video · paired vs MFA

The pauses are measured from the audio itself, so the "truth" costs nothing to build and can't be argued with: either there is a gap between words there or there isn't.

What the numbers sayLink to this section

• MFA 3.4.3 is the open default. 83.3% of its cuts land in a real pause, its median error is 30 ms, and 72% of its boundaries are within 100 ms. It is also by far the fastest open aligner on CPU.
• Chirp 3 is ahead, narrowly. Its own word times put 85.3% of cuts in pauses, a paired lead of 2.8 points over MFA (95% CI 0.9 to 5.2). The lead sits on short pauses: 64% vs 58% on 0.2–0.3 s gaps, and level once pauses pass half a second.
• Feeding ASR text costs nothing. Every open aligner held or gained 0.3–1.5 points against the subtitle-text pilot, so the pipeline can align its own transcript directly.
• Long pauses are safe for everyone. At 0.5 s or longer, every aligner lands 92–99% of cuts in silence.

The dirty laundryLink to this section

• Our first design failed. The original plan joined two windows at a silence and asked where the seam was. Real data broke it, so we switched to the pause test. The failure is recorded in the run notes.
• Empty transcripts cause most failures. Two windows got empty ASR text, which explains the wav2vec2 and MMS-FA failures and two of MFA's three. MFA's third returned 10 words for 9. Handle an empty ASR result and a word-count mismatch, or the aligner will.
• A time cap nearly lost the results. Three aligners shared a CPU under a 50-minute cap, and each saved only at the end. Caught at minute 47. Write results as you go.
• Speed isn't reported for this run, because the aligners shared one CPU.

The recipeLink to this section

1. Use MFA 3.4.3 on your ASR text for English. Lowest error, fastest open option.
2. Prefer cut points at pauses of 0.5 s or more. Every aligner is nearly perfect there, so the choice of aligner matters less than the choice of pause.
3. If your transcript comes from Chirp 3 anyway, use its word times and skip the aligner.

What's nextLink to this section

Non-English aligners (MFA needs a model per language) and the music-bed condition, where news audio is least like FA-Bench.