AI likes Shibugamo, but we don't

A client calls our phone agent to create a contract at one of their sites in Chibougamau. The speech-to-text engine hears it perfectly well, and writes “Shibugamo”. The agent then searches the database for Shibugamo, finds nothing and politely says that there is no worksite in “Shibugamo”. The AI likes Shibugamo. We don't.

The engine wasn't wrong

The first reflex is to blame the transcription. But say both words out loud: Chibougamau, Shibugamo. On the phone, they sound exactly the same. The engine wrote down what it heard, as a person who has never seen the name written would. Ask a colleague from Paris to write it down after hearing it once, and you may well get yet another spelling.

And asking the caller to repeat doesn't help. They will say Chibougamau again, in the same way, and the engine will write Shibugamo again, in the same way. The problem isn't the caller's pronunciation, it's that the sound carries no spelling. Spelling out a city name letter by letter on the phone? Possible, but by then the conversation no longer feels natural, and that was the whole point of a voice agent.

A few things make it worse in practice:

• Proper nouns. Cities, streets, business names and people's names are exactly the words a speech engine knows least, and exactly the ones your tools search on.
• Phone audio. A phone line cuts off the high frequencies that a microphone would pick up, and those are exactly where “ch” and “sh” differ.
• Accents and languages. A Quebec French name, said by a francophone, transcribed by an engine that hesitates between French and English: every guess is plausible.

Our tools were built for typing

In a chat, the client types “Chibougamau”. If they make a typo, they see it, and so do we. Our search tools, and most of the APIs they rely on, were designed for that world: exact match, “contains”, prefix, full-text search. All of them assume that the words being searched were typed by someone who knew how to write them.

With voice, that assumption no longer holds. The words that reach the tool are the engine's best guess at a spelling. One letter off, and an exact search returns zero results. The AI then does exactly what it was told: it reports that nothing was found. Technically correct, and completely useless to the person on the phone.

Most business APIs, from CRMs and ERPs to booking and inventory systems, offer nothing better. You can search by name, sometimes by “starts with”. Searching by sound is almost never offered, because nobody imagined the query would come from an ear rather than a keyboard.

Changing strategy: search by sound

The lesson for anyone building tools for a phone AI agent: don't search for what was written, search for what was heard. In practice, that comes down to three layers.

1. Help the engine hear right

Speech engines accept a list of terms to expect, often called keyterms or vocabulary hints. Give it “Chibougamau”, and it is much more likely to write it correctly. The catch is that the list is short (a few dozen terms) and paid for. So you choose carefully, according to who is calling: for a client, the cities and names of their own worksites; for an employee, perhaps nothing at all. It helps a lot, but it will never be enough on its own.

2. Search by sound, not by letters

When the exact search finds nothing, try again by sound. Well-known techniques describe a word by the way it is pronounced rather than the way it is spelled: “ch” and “sh” count as the same sound, silent letters are ignored, vowels matter less. Chibougamau and Shibugamo then match, simply because they sound alike. A second technique catches names that are only slightly misspelled.

These techniques are not new, and most databases already offer them. The trick is to use them as a fallback: exact search first, and only when it comes back empty, the few closest names by sound.

And when the API you depend on can't do it? Then your tool does it: fetch the short list of candidates that make sense for this caller (their worksites, their accounts, their products), and compare them by sound yourself, before calling the API with the right identifier.

3. Confirm instead of assuming

A fuzzy search finds candidates, not certainties. The tool must tell the agent so: “no exact match, here are the closest results”. And the agent must then do what a good receptionist does: “Do you mean Chibougamau?” One short question, and the conversation moves on. Above all, the agent must never act on a guess, especially not to create a contract or book an appointment.

What we do at Netcoder

At Netcoder, this isn't a lesson learned the hard way and then forgotten. It's how we build every tool our voice agents call, from day one:

• The ear is prepared for each caller. The terms given to the speech engine are chosen according to the type of agent and who is calling, within the engine's limits.
• Every search has a plan B. Exact first, then by sound, then by similarity, and only the few closest results.
• The agent confirms before it acts. An approximate result is always presented as such, and nothing is created or booked on a guess.
• Every tool explains itself. When something can't be found or done, the tool says why, so the agent can correct course instead of repeating the same mistake.
• Your systems don't have to change. When your CRM or ERP can only search by exact name, our tool layer does the phonetic work before calling it with the right identifier.

And we test with real calls, in Quebec French, with the names of real places, like Chibougamau. That's where the generic approach falls apart, and where ours holds.

Designing for the ear

A phone agent isn't a chatbot with a voice. At Netcoder, that's the conviction behind everything we build.

None of this is magic. What's hard is knowing it, because our reflexes come from screens and keyboards. Every tool a phone agent calls must expect words that are approximate, but faithful to what was said, and react the way a person would: by looking for what sounds the same, then confirming.

The AI will keep liking Shibugamo. It's up to us to make sure the caller still gets to Chibougamau.

All articles