AI Customer Service Platform
An on-prem, multi-channel support platform that answers customers on WhatsApp, voice and commerce channels — with provider-agnostic LLMs, tool-calling guardrails and full audit trails.
Interface — not publicly deployed.
The problem
Support teams running on WhatsApp and e-commerce platforms face three problems at once: response volume they cannot staff for, customer data they are not allowed to send to third-party APIs, and AI vendors they cannot switch away from. Most AI support tools solve the first and make the other two worse.
Architecture
Channel layer — WhatsApp Cloud API and WaSenderAPI behind a single MessagingProvider interface; Shopify and Magento webhooks with HMAC validation and idempotency checks. Webhooks ACK immediately and hand off to Celery.
Orchestrator — billing guard, agent routing, context builder, and a class-based tool registry with per-tool confirmation rules and per-agent allow-lists.
LLM router — OpenAI, Groq, Anthropic, Cohere and Ollama behind one provider interface, swapped with a single env var.
Voice loop — inbound audio downloaded and transcribed with Whisper; outbound replies synthesized by a local TTS service and uploaded back as WhatsApp media.
Platform — RBAC, API-key server-to-server auth, PII masking and optional Fernet encryption, Prometheus metrics, and audit logging on every financial action.
Key decisions
One LLM provider interface with five implementations
Coding directly against a single vendor SDK
Cost, latency and data-residency requirements change per tenant. Swapping OpenAI for a local Ollama model is an env var, not a refactor — which is what makes the on-prem story credible.
A deterministic orchestrator with an LLM inside it
Letting agents decide the control flow
Multi-agent control flow is where latency and error accumulation come from. Keeping flow deterministic and using the LLM only as a reasoning step keeps the system testable and debuggable under load.
Class-based tool registry with per-tool confirmation and audit rules
A flat list of functions handed to the model
Refunds and payment captures are irreversible. Confirmation requirements and allow-lists belong in the registry as data, not scattered through prompt text where a model can talk its way past them.
Immediate webhook ACK with async Celery processing
Processing inline in the request handler
WhatsApp and Shopify retry on slow responses. Answering fast and processing after is the difference between clean delivery and a storm of duplicate messages.
Results
Stack
Related field notes
MAJID
“Innovation distinguishes between a leader and a follower.”
Need an AI system that holds up in production? Tell me what breaks today and I'll tell you what I'd build.