A green test suite proves a function returns the right answer. It says nothing about whether that answer arrives fast enough once real data shows up, and that gap is where AI-generated code quietly breaks in production.
The Details:
- An AI model writes against what it can literally see: a function signature, a schema, a twelve-row test fixture. It has no access to your dashboards, your on-call history, or the table that grew ten times over last quarter, so it cannot account for constraints nobody typed into the prompt.
- Given incomplete information, the model defaults to the cleanest-looking solution, not the fastest one. A lazy loop that fires one query per record reads well in review and passes every test at small scale, then fires ten thousand queries the moment a real user shows up with ten thousand records.
- Load testing after the fact only catches the failure once the expensive part, the data access design, is already built and needs to be redone. The cheaper fix is naming the constraints before any code exists: the actual row count, the p99 latency target, how often the endpoint gets hit.
- Asking the model to list the three most likely ways an approach fails at scale, given stated constraints, turns it into a reviewer of its own work. It will surface the query that becomes a table scan at ten million rows or the retry storm triggered by a slow downstream service, risks invisible in a small fixture.
Bottom Line: Correctness at the wrong scale is not correctness, and the fix is writing down row counts and latency budgets so they enter the prompt before the code does, not after production finds them for you.
