AssemblyAI ranks 50+ LLMs on real voice tasks: the best picks are mostly the cheap ones
- Released October 13, 2026; arrives alongside an expansion of AssemblyAI’s LLM Gateway to more than 50 models, all behind the same API key customers already use for speech-to-text.
- Tests real speech-to-text output — filler words, false starts and the occasional misheard word included — at four transcript lengths: about 30 seconds, 2 minutes, 15 minutes and 60 minutes; every model gets the same transcripts and prompt.
- Two judge models, Claude Opus 5.5 and GPT-5.6 Sol, score each answer against a fixed rubric per task, with offsets measured and combined; every metric is normalized to 0–100; dictation is scored mechanically with no judge.
- Also tracks reliability: format failures, refusals, timeouts and error rates for every model-task combination — models that fail the reliability check still appear in the charts but don’t get recommended.
- Headline picks: Summarization — Claude Opus 5.5 leads at 98 quality, GPT-6 Luna hits 94 at the lowest price, GLM-5.3 Flash delivers 91 at the lowest latency; price and latency picks must land within 8 points of the top score.
AssemblyAI released Voice Standard Bench today, a public benchmark that scores more than 50 language models on the four tasks voice products actually run through an LLM — summarization, information extraction, text correction, and dictation — with quality, speed, and cost reported side by side. The main finding, in AssemblyAI's own words: most voice tasks don’t need frontier models, or frontier-model prices.
Most voice products do more than produce a transcript. After speech-to-text returns the text, the next step is usually an LLM call: cleaning up dictation, summarizing a call, pulling out details, or drafting a follow-up. At volume, AssemblyAI says, that step quickly becomes a large share of the bill — often paid at frontier-model prices for tasks that don’t need a frontier model. The benchmark arrives alongside an expansion of AssemblyAI’s LLM Gateway to more than 50 models, all available behind the same API key customers already use for speech-to-text, including fast, low-cost open models the company hosts on its own infrastructure.
What it tests
Voice Standard Bench scores models on the inputs voice products actually send: real speech-to-text output, filler words, false starts, and the occasional misheard word included, at production-typical lengths. Every model gets the same transcripts and the same prompt, at four lengths — about 30 seconds, 2 minutes, 15 minutes, and 60 minutes.
Two judge models, Claude Opus 5.5 and GPT-5.6 Sol, score each answer against a fixed rubric per task, with scores tracked separately so the offset between the two judges can be measured, then combined. Every metric is normalized to a 0–100 scale; dictation is scored mechanically against known-correct output, with no judge at all. The bench also tracks the unglamorous stuff: format failures, refusals, timeouts, and error rates for every model-task combination. Models that fail the reliability check still appear in the charts with their scores, but they don’t get recommended.

The take-home picks
AssemblyAI publishes a grid of picks per task and transcript length — top quality, lowest price, and lowest latency, with the price and latency picks required to land within 8 points of the top score so a cheap model that fails the task can’t win the column. The headline results from the current run:
Summarization: Claude Opus 5.5 leads at 98 quality, but GPT-6 Luna hits 94 at the lowest price, and GLM-5.3 Flash delivers 91 at the lowest latency. Information extraction: Claude Opus 5 tops at 87, GPT-6 Luna again takes the budget slot at 86, GPT-4.1 is fastest at 79. Text correction: three OpenAI models — GPT-5.6 Sol, GPT-6 Astra, and GPT-6.1 Sol — tie at 91, while the open-weight gemma-4-31b reaches 84 at the lowest price. Dictation: Claude Opus 4.6 and Gemini 3.5 Flash tie at 99, gemma-4-31b lands 97 on a budget. Weighted across all tasks, GPT-6 Astra scores 92 overall, GPT-6 Luna 90 at the lowest price, and GPT-4.1 84 at the lowest latency.
The number that should make CTOs reach for a calculator: in the current run, the top scorer costs 55 to 65 times more per hour of audio than a model within two quality points of it. Only on hour-long transcripts does that premium narrow, to about five times.
Where small models win — and where they don’t
Open models win on speed for short transcripts. On a 30-second voicemail, gemma-4-31b answers in under a second at 93 quality, against GPT-6 Luna’s 1.7 seconds; on a two-minute call it answers in 1.5 seconds at 89 versus 2.4. Qwen3.5 4B Fast is the fastest model in the run at every length up to 15 minutes. That said, as transcripts approach meeting length, larger models pull ahead on quality and the speed gap closes.
Surprisingly, open models rarely win on cost. GPT-6 Luna, at its default reasoning setting, scores within a point of the best open model at every transcript length while costing between 0.4 and 1.6 cents per hour of audio — the open models that undercut it save fractions of a cent and, past two minutes, give up at least 10 quality points to do it.

Run it on your own transcripts
Voice Standard Bench is open source: developers can download the code from GitHub, point it at their own transcripts, and run a private version with the same tasks, rubrics, and judges, scored on their own data. For those who don’t want to run code, there’s a Playground compare mode that runs one prompt against several models side by side, and “Try in Gateway” links on the results page that open the playground with that model and task prompt already loaded.
The honest caveat comes from AssemblyAI itself: its transcripts aren’t yours. Different accents, domain vocabulary, prompts, and cost limits can change which model works best, so the bench is a starting point for which two or three models to test — not the final answer. Results refresh as new models launch on the Gateway, with each run dated.
Sources: AssemblyAI — “Introducing Voice Standard Bench: 50+ models, ranked by performance, speed and cost on real audio” (October 13, 2026); AssemblyAI — Voice Standard Bench results page; Blockchain.News — “AssemblyAI Launches Voice Standard Bench to Rank 50+ LLMs on Real Audio Tasks” (October 13, 2026).
Frequently asked questions
What is Voice Standard Bench?
AssemblyAI’s public benchmark that scores 50+ language models on the four tasks voice products actually run through an LLM — summarization, information extraction, text correction and dictation — with quality, speed and cost reported side by side.
How does the benchmark keep things fair?
Every model gets the same real-world transcripts (filler words, false starts, misheard words included) at the same four lengths, judged by Claude Opus 5.5 and GPT-5.6 Sol against a fixed rubric with offsets measured — and cheap models can only win the price/latency columns within 8 points of the top score.
What’s the main takeaway?
In AssemblyAI’s words, most voice tasks don’t need frontier models, or frontier-model prices: the per-task picks are mostly fast, low-cost models that land close to the quality leaders — e.g. GPT-6 Luna scoring 94 in summarization at the lowest price versus Claude Opus 5.5’s 98.