Demo agents are judged on whether they can do the task once. Production agents are judged on what happens the other times — when a tool times out, when the model loops, when someone asks for a refund it should not grant. Treating the agent as an execution system rather than a chat interface is what closes that gap.
Start narrower than feels useful
The most common structural mistake is one large agent that does everything. It cannot be tested, because there is no defined success condition. It cannot be debugged, because every failure has a dozen possible causes. Scope can always be widened later; a monolith cannot be un-tangled under pressure.
- One use case
- A small tool set
- A measurable success condition
- Support triage, document extraction
- A full autonomous workflow
- Dozens of tools
- Vague goals
- “handle everything”
State belongs outside the conversation
If the only record of what an agent has done lives in its message history, a restart loses everything and the agent repeats work it already completed. Persist state externally, keyed by a run id, and let the conversation be a view of that state rather than the state itself.
Errors are information, not crashes
A failed tool call should return a structured error the agent can reason about — retry, try a different tool, or escalate. Raising an exception ends the run and throws away everything learned up to that point.
