“Close enough” is not feedback.

A pronunciation coach that scores speech phoneme by phoneme rather than word by word. The sentence you're meant to say is turned into its phonetic spelling; the sentence you actually said is read back the same way; the two are lined up, and every symbol gets its own score. 44+ phoneme classes, under two seconds per utterance, running on the phone. Built with a team at the Apple Developer Academy in 2025.

The part that separates it from a spelling checker for speech is what it catches. Most models are trained to ignore the sounds that don't belong, because they're looking for meaning. A pronunciation coach has to do the opposite — the sound you added is exactly the thing worth telling you about.

iOS developer and AI engineer — the app and the model under it. SwiftUI on the front, the phoneme recognition and the alignment that scores it, the latency work, and the conversion that moved the whole thing onto the device. Interaction design was my teammates'.

Learning a language is intimidating less because of grammar than because of the fear of sounding wrong — and most pronunciation apps answer that fear with a pass/fail. Two gaps in the existing tools do the damage:

• The black box. Speech-to-text tells a learner what they said, never how. The engine flags the whole word wrong even when one sound was off, so a red line under “Think” doesn't say whether the mistake was the Th (tongue position) or the ink (vowel shape).
• Extra sounds are invisible. Learners don't only substitute sounds, they insert them — “school-uh” for “school” — or swallow the ending consonant. Speech models are built to treat that as noise on the way to meaning. For a coach, that noise is the lesson.

So the question the build had to answer: how might we give precise, phoneme-level feedback so a learner can practise with confidence?

Work below the word, on both sides of the comparison. eSpeak-NG converts the target sentence into the phoneme string it should be — grapheme-to-phoneme, deterministic, no model involved. A Wav2Vec2Phoneme model reads the recording into the phoneme string it actually was. Aligning those two strings is where every kind of error becomes visible at once.

The two strings are scored against each other with Levenshtein distance, and picking an edit-distance metric is what makes the second problem from above disappear on its own. Its three operations are the three ways a learner goes wrong: a substitution is a swapped sound, a deletion is a swallowed one, and an insertion is the extra sound nobody else was looking for. The gaps in the table below are those deletions and insertions — no special case, just what the algorithm already returns.

The feedback lands on the text itself rather than in a report: a 0–100% phonetic match score for the utterance, correct phonemes in green and mispronunciations in red, and any error word tappable to see how the articulation differed from the target sound.

It didn't start on the phone. The first working version served a PyTorch model from a REST backend and the round trip dominated everything — the latency pass that got it under two seconds was byte-stream decoding, not a better model. Converting to CoreML afterwards took the network out of the loop entirely, which is also what made it usable offline.

Both halves run on the phone, which is why the score arrives while the learner is still looking at the sentence they just read.

The three error types are the whole argument, and they come free with the metric — substitution, deletion and insertion are exactly what Levenshtein distance counts. A word-level checker collapses all three into one red cross.

• SwiftUI
• CoreML
• PyTorch
• eSpeak-NG
• Wav2Vec2Phoneme

Latency is a product decision. Nothing about the model changed when I got it under two seconds — I changed how the bytes arrived. Testers called the fast version “smart” and the slow one “broken.” Same scores, both times.