A working prototype tells you almost nothing about production readiness, because a developer is always standing nearby to catch the weird outputs. Remove that person and the agent runs unattended against real systems, real money, and real consequences. The gap that opens up has almost nothing to do with model quality.
The Details:
- Retries create silent duplication. An agent that resends a failed step can trigger a second charge or a second ticket if the original action partially succeeded. Fixing this means giving every action a stable key so downstream systems can deduplicate, coordination work that gets skipped until the first incident forces it.
- Standard monitoring answers the wrong question. Latency and error rates do not explain why an agent made a bad call. Teams need a full, replayable reasoning trace: what it saw, what tools it weighed, what path it chose. Most discover the gap only after something has already gone wrong.
- Blanket human approval collapses within a week. Reviewers start rubber-stamping the ninety-five percent of actions that are obviously fine, and the safety checkpoint becomes theater. The fix is calibrating oversight to blast radius: reversible actions run autonomously, humans concentrate on money, external communication, and anything irreversible, reviewed asynchronously rather than gating every step.
- Rollouts fail as workflow changes disguised as tooling changes. When an analyst shifts from triaging tickets to evaluating an agent's triage, that is a different job requiring different training and buy-in. Teams that skip this renegotiation see quiet workarounds and uneven adoption instead of outright rejection.
Bottom Line: The agent is the easy part; the scaffolding around it, the permissions, the trace logs, the escalation paths, is where production systems actually get built or quietly fail.
