Hugging Face has added private Egyptian- and Nigerian-accented English to its Open ASR Leaderboard. The November 6 announcement describes Liva AI conversational sets of 1.34 and 1.55 hours, respectively.

The big change

  • The new sets broaden accent coverage, but short recordings cannot represent every speaker or situation in either country.
  • Keeping evaluation data private is intended to make it harder to optimize directly for a known public test. It does not eliminate the need for testing on speech that resembles an application's own users.
  • Recognition quality can vary across speakers and conditions. A score does not guarantee that captions or transcriptions work equally well for everyone.

What the new sets measure

Automatic speech recognition converts audio into text. The announcement describes spontaneous conversations with punctuated, cased transcripts retaining disfluencies; it does not give speaker counts or selection details.

The private view compares US and non-US speech and groups India, Nigeria and Egypt under Kachru's circles, with a finer split planned.

Why a private test matters

In a May 2026 explanation of its earlier private-data expansion, the leaderboard team said public test data can be directly targeted by model developers or approximated through similar training data. Keeping examples private is meant to reduce that pressure, although models exposed to similar distributions could still benefit.

Word error rate compares recognized words with a reference transcript, counting substitutions, deletions and insertions. A lower aggregate error can help compare systems, but it does not identify which individual names or phrases were missed or whether errors fall evenly across speakers.

What remains unknown

The November announcement reports no per-model results for the additions. Small sets cannot isolate accent from recording conditions or represent whole populations.

The stakes of evaluation across groups are illustrated by a separate 2020 study of five speech-recognition systems on US interviews. Its researchers reported higher average word error rates for Black than white speakers in that particular sample. It did not test the Liva sets.

No independent model test was conducted for this article.