AI agents in production: what nobody tells you before the first deploy

The demo lies, and it lies well

Every AI agent looks brilliant in the first fifteen minutes. You give it a task, it chains three tool calls, returns a tidy answer and the room applauds. The problem is that the demo runs on the happy path: clean input, one attempt, nobody watching the cost.

Production is the opposite of that. Dirty input, a thousand runs a day, and someone — probably you — on call at two in the morning.

An agent in production is not a model with a prompt. It is a distributed system in which one of the nodes is non-deterministic.

The four places it breaks

1. The context blows up at the worst moment. While the history is short, everything fits. By conversation number forty, the agent has forgotten the instruction from the start and begins inventing with confidence. Treat the context window as a budget with an owner: what goes in, goes in by explicit decision.

2. The tool fails and the model improvises. When an API returns a 500, the model tends to carry on with a plausible guess. Return errors as structured data and say, in the prompt, what to do about each one. A handled error is an error that does not turn into a hallucination.

3. The cost grows in silence. Retries, the whole history resent at every step, tools that return giant JSON. The bill triples with no new line of code. Measure tokens per run from day one — the metric nobody collects is the one that explodes.

4. You cannot debug what was never recorded. Without the full trace of every step — prompt, response, call, return — you are not investigating: you are guessing.

What actually helps

• Output contract. Ask for JSON with a schema and validate before using it. Free-form output is technical debt with interest.
• One agent, one job. Several small, verifiable agents beat one omniscient agent.
• Adversarial verification. A second step whose only job is to try to knock down the first one's answer knocks down a lot of wrong things before the user sees them.
• Explicit ceilings. A limit on steps, on time and on cost per run. An infinite loop with a credit card attached is a new category of bug.
• Record everything. Keep the whole run. That is what turns an incident into a lesson.

task → plan → run tool → verify → answer
            ↑__________ failed __________|
                  (with a retry ceiling)

The bar I use before shipping

Before putting any agent live, I answer three questions. If any of them has no answer, it does not ship.

1. When it gets something wrong, how do I find out — before the user tells me?
2. What is the worst-case cost of a single run?
3. Is there a path where the agent causes irreversible damage without human confirmation?

A good agent is not the one that gets it right in the demo. It is the one that fails in a predictable, cheap and visible way.

For the next article

I will unpack the most ignored item on that list: how to build the observability layer of an agent without drowning the team in logs. If you want to follow along, the channels are right below, in the signature.