← Lab
Field note·Voice & Multimodal AI·5 min read

Building a Voice AI Agent for Arabic Call Centers

The most valuable work in a call-centre voice project happens before any architecture decision: listening to the calls.

Building a Voice AI Agent for Arabic Call Centers

Pull a random sample of calls and categorise every one. That single exercise tells you your ROI ceiling, which intents to build first, and where human escalation is non-negotiable — and it costs a few days rather than a few months of building the wrong thing.

The distribution decides the roadmap

01Status enquiries ~31%Lookup, answer, close. Automate first.
02Coverage questions ~24%Answerable from policy documents.
03Renewals and changes ~18%Automatable with confirmation on writes.
04Complaints ~12%Escalate — value comes from a person.
05Complex medical ~9%Escalate.
06Emergencies ~6%Escalate immediately, no exceptions.

In a typical centre, around 70–75% of calls follow patterns an agent can handle. The remaining quarter is where humans add value that cannot be automated, and routing those wrongly does real harm.

Dialect-specific transcription is not optional

Generic models trained on one variety of a language perform poorly on others. A generic Arabic model might sit around an 18% word error rate on regional dialect call audio — roughly one word in five wrong, which destroys every downstream intent decision.

Collecting real call recordings and tuning on them can bring that to roughly 7%, which is workable for most intents. The method matters more than the number: record your own calls, transcribe a sample by hand, and test every candidate model against that — never against a benchmark dataset.

Intent matters more than transcription

Understanding what someone said is not the same as understanding what they want. “I want to know where my claim is” is a status enquiry. “The doctor said the surgery isn't covered and I don't understand why” is a coverage dispute *and* an escalation. Same language, different backend call, different outcome.

What to take away
  • Categorise real calls before making any architecture decision.
  • Around 70–75% of call volume typically follows automatable patterns.
  • Evaluate transcription on your own audio, never on benchmarks.
  • Build the intent classifier separately from the generation model.

Field notes on building production AI systems — collected, verified and written up so they are useful to anyone working on the same problems.

smoke
“Innovation distinguishes between a leader and a follower.”

Need an AI system that holds up in production? Tell me what breaks today and I'll tell you what I'd build.

Call Me now !

mail

abdelmajidzaddi088@gmail.com

address

Mohammedia, Morocco

phone

+212 695 557 631