A pipeline can run, log success, and quietly produce nothing, and the code that caused it will look perfectly reasonable in review.
The Details:
- Language models learn code from tutorials, docs, and public repos, and that corpus almost always shows the input working: the file exists, the API answers, the query returns rows. Nobody writes a blog post titled what to do when the server returns 200 with an empty body, so the model never learned to ask that question unless the prompt forces it.
- Four failure modes recur across generated code because none of them show up in training examples: empty results treated as valid, missing timeouts that turn a slow dependency into a hang, multi-step jobs that leave a database row with no matching file after a mid-process crash, and no built-in check for whether a retried job already ran. Each one hides behind code that technically did not throw an exception.
- Catching an exception is not the same as handling one. A broad try/except with a log line looks careful but converts a loud crash, the kind that gets noticed immediately, into a silent data gap that surfaces weeks later as missing invoices or thin numbers nobody can explain.
- Fixing this after the fact does not work well, because resilience is a structural property, not a patch. Transaction boundaries and idempotency have to be decided before generation, not found in a diff. Two added sentences before prompting, describing the observable success state and what must hold true on a second run, change what the model treats as a valid outcome.
Bottom Line: The gap in AI-generated code is not a bug in the model, it is a gap in what the prompt specified about failure.