Align

Platforms

iOS, macOS, tvOS, visionOS, Linux, Windows, Node

Languages

9

Weights

v1.1.0

Install

.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "4.0.0")

npm i @desert-ant-labs/align

Changes in this release

• refine is async and takes languageCode. refine works on the words of any transcript, not only on SpeechAnalyzer output.
• The handle is Align. SpeechTimestampRefiner remains as a deprecated alias for the name only. A 3.x call site doesn't compile through the alias, because Align has no locale: initializer and both refine and isSupported changed shape.
• Streaming callers move to StreamingRefiner. StreamingRefiner wraps an Align and keeps the SpeechAnalyzer integration. SpeechTimestampRefiner(locale:) no longer compiles.
• On Apple platforms, corrected timestamps change at this release. We fixed a bug in the Core ML runtime's audio features, which scaled power by 4 before the log.
• Align now runs on Linux, Windows and Node, through LiteRT.
• The npm package runs only in Node. In the browser, load() throws an error that tells you to import @desert-ant-labs/align/native on a server instead. See Limitations for why.
• Three more changes break 3.x code. isSupported is now an async throws -> Bool method that takes a language code, where it used to be a property. The offline handle no longer has reset(). AlignResourceError is removed.

Usage

Align corrects the word timestamps of any transcript against its audio. On Apple platforms, Align also attaches to SpeechAnalyzer (iOS 26 and later) through StreamingRefiner.

import Align

let align = Align()
guard try await align.isSupported(languageCode: "en") else { return words }
let fixed = try await align.refine(words, audio: samples, sampleRate: 16_000, languageCode: "en")

import Align

let refiner = StreamingRefiner(locale: locale)

try await analyzer.start(inputSequence: inputs.recordingAudio(for: refiner))

for try await result in transcriber.results.refiningTimestamps(with: refiner) {
    result.text     // corrected word-level audioTimeRange attributes
    result.words    // [WordTiming]: text, start, end, refined
}

let input = try await refiner.analyzerInput(buffer)   // buffers the audio, returns Apple's input

let refiner = try await StreamingRefiner(locale: locale, audioFile: file)

import { Align } from "@desert-ant-labs/align/native";

const align = await Align.load();
const fixed = await align.refine(samples, 16000, words, { language: "en" });
align.dispose();

guard try await align.isSupported(languageCode: "sv") else { /* use the original timestamps as-is */ }

let align = Align()
if !align.isDownloaded() {
    try await align.download { fraction in print("\(Int(fraction * 100))%") }
}

let offline = Align(directory: myModelDirectory)   // uses the files as they are, downloads nothing

Files

On Apple platforms, the Swift SDK downloads the compiled .mlmodelc stages, mel_filters.bin, calibrator.bin and refiner_config.json. On Linux, Windows and Node, the .tflite pair replaces the two .mlmodelc directories.

Inputs and outputs

• You pass mono audio and the words of any transcript, with their proposed start and end times.
• You get the same words back with corrected start and end times. A word keeps its original time when a correction isn't structurally safe.

Accuracy

On clean audio, averaged over the nine languages, Align cuts the timing error of the input timestamps by two-thirds. Per language, the cut ranges from a third to over three quarters. The cut is smaller on noisy audio.

We score both runtimes on gold-en-us, a set of 258 boundaries corrected by hand against the waveform. On Apple platforms, the Core ML runtime's mean boundary error on that set is 44.8ms (darwin-arm64, 2026-09-21). The training-time reference scores 45.0ms. The input timestamps, before Align, score 100.8ms. The LiteRT runtime matches the training-time reference to five significant figures. These are averages over one English set, and a single boundary can be further off.

The clean, noisy, and per-language figures come from the training-time reference, not from an on-device runtime. The on-device Core ML figures from v1.0.0 no longer apply, because the Core ML runtime's audio features changed at this release. The model card has the same per-condition figures and names the three weakest languages.

Languages

English, Spanish, French, Italian, Portuguese, German, Japanese, Korean, and Chinese. Align passes a locale outside this set through unchanged.

Limitations

• The averaged figures compare against a forced aligner, not against boundaries marked by hand. The figures show that Align cuts the timing error of the input timestamps. They don't show how close Align gets to the true boundary.
• Align doesn't improve every boundary. The structural fallback keeps the original timestamp when a correction looks unsafe. The fallback can't catch every plausible-looking error.
• Japanese, Korean, and Chinese were the weakest languages before v1.0.0. Align now improves their input timestamps by 33%, 55%, and 51%.
• Numbers are the weakest remaining case. On a small sample, refinement moved digit boundaries further from the reference than leaving them alone. Treat spoken numbers as unimproved until a larger sample settles the question.
• The Core ML and LiteRT runtimes can disagree by 10ms on the same audio. We test both runtimes against Core ML's recorded output on synthetic audio. Core ML on the CPU matches that output (0.0ms, against a 25ms tolerance). LiteRT drifts 10.4ms (linux-arm64, 2026-09-17). Don't expect the two runtimes to agree to below a millisecond.
• Align has no browser build. Align runs two models in sequence, and our WebAssembly build loads only one model.
• Align has no Android SDK.