Urdu speech recognition for voice agents
Callers in Pakistan speak Urdu and switch to English mid-sentence, and off-the-shelf models miss exactly the words that matter. The base model wrote no Urdu at all. I added an Urdu token to its tokenizer, fine-tuned it on 121 hours of speech, and made it stream fast enough for live calls.
| Metric | Base model | Latest test |
|---|---|---|
| Word error rate | ≈100% | 22.1% |
| Character error rate | ≈100% | 17.0% |
| English words kept in mixed speech | none | 74.6% |
| Speed vs. real time | 5.5× faster | |