The AI is the easy part. Observability, error handling and evaluation are where these systems live or die — and all three are unglamorous infrastructure work that is easy to defer until the first incident makes it urgent.
Observability before anything else
If you cannot see what the agent did, you cannot fix it when it breaks. Every tool call needs a span — not just input and output, but the whole decision: what it planned, which tool it chose, what happened when the tool ran.
Assign a unique run id to every execution and correlate on it. With that in place, a failure at 2am is a replay of the exact run. Without it, you are reading raw logs and guessing.
Error handling is not optional
Treat a failed tool call as a value, not an exception. Return a structured error, let the agent retry with backoff, and escalate to a human when retries are exhausted. The difference between a run that recovers and a run that dies is usually just this.
Evaluation, or you are shipping on vibes
Without a fixed question set scored on every change, nobody can say whether this week's prompt edit helped or quietly broke three intents. A golden set of representative questions, run on every deploy, turns prompt engineering from taste into engineering.
