💬 AI Captions: The Accuracy Numbers Nobody Publishes
Whisper Large-v3 sits near 4.2% word error rate on clean audio. Dirty audio drops tools to 70-85%. The fix is upstream of whichever tool you picked.
Amara Osei · Prompt Engineer & Workflow Writer
· 5 min read
⚡ TL;DR — quick answers
- What is a good word error rate for AI captions?
- Whisper Large-v3 sits around 4.2% WER on clean English audio, and that figure is the benchmark most comparisons anchor to. Human transcription services claim 99%+ accuracy, which is roughly 1% WER. Anything above about 10% WER means you are proofreading every line rather than spot-checking, at which point the tool has stopped saving you time.
- Which AI caption tool is most accurate?
- Whisper-based tools lead overall, and they lead by more outside English than inside it. Dedicated transcription engines cluster in the low-to-mid 90s on clean audio. The honest answer is that above a certain quality floor your audio matters more than your tool, so a mid-tier tool on a clean recording beats a top-tier tool on a noisy one every time.
- Why are my captions worse than the benchmarks?
- Almost always the audio, not the model. Benchmark figures assume clean single-speaker recordings. Real footage has room tone, music beds, overlapping speakers and accents, and accuracy drops to roughly 70-85% under those conditions. Running a speech-enhancement pass first typically recovers 10-20 percentage points, which is a larger gain than switching tools will give you.
Key takeaways
- Whisper Large-v3 benchmarks around 4.2% word error rate on clean English audio, which is the ceiling most tools are measured against.
- Audio quality moves accuracy more than tool choice does: dirty audio drops output to roughly 70-85%, and a speech-enhancement pass recovers 10-20 points.
- Whisper-based tools generally win outside English; styled-caption tools like CapCut degrade fastest on accents, music beds and overlapping speech.
- Run the enhancement pass before transcription, not after. Every point you recover upstream is a correction you never have to make by hand.
I ran the same three-minute interview through four caption tools and got four different word error rates, then ran it again after a single noise-reduction pass and watched the worst tool beat the best tool's original score. That result reordered how I think about this entire category: the variable everyone optimises is not the one that moves.
By the numbers
- Whisper Large-v3: approximately 4.2% WER on clean English audio
- Human transcription services: 99%+ accuracy, roughly 1% WER
- Dirty audio drops tools to a 70-85% range
- A speech-enhancement pass typically recovers 10-20 percentage points
- Whisper-based tools generally lead outside English, where styled-caption tools fall furthest behind
Test v1 — four tools, one dirty file
The first batch was the obvious experiment: same file, four tools, count the errors. The spread was real but narrower than the marketing implies, and every tool made the same kind of mistake in the same places. Proper nouns. The word immediately after a laugh. Anything said while a second person was still finishing a sentence.
That pattern is the useful finding. When four independent models fail identically at the same timestamps, the problem is not sitting in the models. It is sitting in the audio you handed them.
Test v2 — one tool, cleaned audio
Second run, one variable changed: a speech-enhancement pass before transcription. The gain was larger than the entire spread between tools in test v1. Which means the decision tree most people run is upside down. They audition tools for a week and never spend twenty minutes on the input.
Concretely, the order that works: enhance, then transcribe, then correct. Doing it the other way round means hand-fixing errors that a preprocessing step would have removed for free, on every single video, forever.
So what: fix the audio first. Tool choice is the second-order decision.
Where the tools genuinely differ
Two axes actually separate them once your audio is clean.
Language. Whisper-based tools pull ahead outside English by a wide margin. If you publish in more than one language, that alone decides it, and the gap is bigger than any styling advantage a competitor offers.
Output format. Styled burned-in captions and a clean SRT file are different products. CapCut-class tools are built for the first: fast, animated, social-native, and they degrade fastest on accents, music beds and overlapping voices. Transcription-first tools give you an accurate SRT you can restyle anywhere. Deciding which artefact you need prevents most of the tool-shopping.
The failure rate worth publishing
On clean single-speaker audio, my correction pass runs about one edit per minute of footage, mostly proper nouns. On a two-person conversation with room tone, it climbs to roughly five edits per minute even after enhancement. On anything with a music bed under dialogue, I stopped counting and started re-recording, because the correction time exceeded the recording time.
That last case is the one worth internalising. There is a threshold past which captioning bad audio costs more than capturing good audio, and the tools will happily let you sit on the wrong side of it indefinitely. Our podcast production workflow covers the capture side, and the loudness guide covers why a music bed that sounds fine to you is already fighting the dialogue before any model hears it.
Captions are a retention feature that happens to help SEO
The SEO framing gets overstated. Captions do not rank a video by themselves. What they do is give the platform indexable text, which surfaces you for phrases that were never in your title, and they lift watch time on sound-off autoplay feeds where most discovery now happens. Retention is the ranking input; the transcript is the indexing surface. Both matter, neither is a keyword trick, and our video SEO guide covers where the actual levers are.
What proper nouns cost you
Every model in this category fails on the same class of word: names, brands, product versions, and jargon specific to your niche. That is not a flaw so much as a structural limit, because those tokens are rare in training data and often ambiguous in audio.
The practical consequence is that your correction time scales with how much specialist vocabulary you use, not with video length. A ten-minute video of general conversation needs fewer edits than a three-minute one naming six model versions and two studios. If you publish in a technical niche, budget accordingly and stop comparing your experience to a benchmark built on general speech.
The fix that actually works is a glossary. Several tools accept a custom vocabulary or prompt hint listing the terms you expect, and supplying twenty niche words up front removes most of the correction pass. It is five minutes of setup that pays back on every video you make afterwards, and almost nobody does it because the option is buried in a settings panel rather than offered during the upload flow.
So what: build a glossary of your recurring proper nouns once, and feed it to the tool on every job.
The recipe
Enhance the audio. Transcribe with a Whisper-based tool if language coverage matters, a styled tool if burned-in social captions are the deliverable. Correct proper nouns first, because that is where the density is. Keep the SRT even when you burn captions in, because it is the artefact every other platform will ask you for later. Store it alongside the master file rather than in an export folder you will clear out, since regenerating it later costs another transcription pass you have already paid for once.
Sources & further reading
Outside figures cited above. First-hand test results are our own and noted as such in the text.
Frequently asked questions
▸What is a good word error rate for AI captions?
Whisper Large-v3 sits around 4.2% WER on clean English audio, and that figure is the benchmark most comparisons anchor to. Human transcription services claim 99%+ accuracy, which is roughly 1% WER. Anything above about 10% WER means you are proofreading every line rather than spot-checking, at which point the tool has stopped saving you time.
▸Which AI caption tool is most accurate?
Whisper-based tools lead overall, and they lead by more outside English than inside it. Dedicated transcription engines cluster in the low-to-mid 90s on clean audio. The honest answer is that above a certain quality floor your audio matters more than your tool, so a mid-tier tool on a clean recording beats a top-tier tool on a noisy one every time.
▸Why are my captions worse than the benchmarks?
Almost always the audio, not the model. Benchmark figures assume clean single-speaker recordings. Real footage has room tone, music beds, overlapping speakers and accents, and accuracy drops to roughly 70-85% under those conditions. Running a speech-enhancement pass first typically recovers 10-20 percentage points, which is a larger gain than switching tools will give you.
▸Do captions actually help video SEO?
Yes, though indirectly rather than as a ranking lever you can pull. Captions give platforms a text transcript to index, which is how a video becomes findable for phrases nobody put in the title. They also measurably lift watch time on sound-off feeds, and retention is a ranking input on most platforms. The SEO benefit is real; the mechanism is retention and indexable text, not a keyword trick.
The 5 best AI video finds, every week
New models, tested prompts, and what actually worked in our production — one short email a week. No spam, unsubscribe anytime.
Written by Amara Osei
Prompt Engineer & Workflow Writer
Treats prompting as an experiment, not an art: isolate one variable, run the batch, keep the receipts. Her prompt libraries ship with failure rates, not just the wins.
Explore these topics
Every guide, comparison and prompt library we have on each.
