Why it matters: AI-written functions can be locally flawless and still wreck a system, because the rules holding that system together often live nowhere a model can read them.
The Details:
Most working systems run on unwritten rules: one payment per order, a counter that only climbs, one worker per queue item. These constraints sit in a senior engineer's memory or a long-dead Slack thread, never in a type, a schema, or a test. A model has no way to consult tribal knowledge, so it generates code that is correct in isolation and wrong in context.
The failure is not a model defect, it is a storage problem. Any rule that exists only as a mental model was already one resignation away from disappearing. AI-assisted development just raises the rate of small, locally sound changes past the point where a team can track what is quietly piling up underneath them.
Fixing it means converting a sentence into something that executes. A precondition at the top of a function, a schema validator guarding a shared file, a CI check that fails the build on violation. Property-based testing tools like Hypothesis or fast-check are especially useful because they probe for invariants nobody got around to naming, by throwing inputs nobody thought to try.
The habit has to move into incident response. When a bug traces back to a broken unwritten rule, the fix is not finished until that rule runs somewhere in the pipeline, not documented, enforced. Review then asks a sharper question than "does this diff look right": what assumption does this change lean on, and what actually checks it.
Bottom Line: Writing a precondition takes thirty seconds; the debugging it prevents takes a weekend, so the real cost was never the check, it was skipping it.
Enjoy this article?
Listen to the Claude Code Conversations radio show or join the community.
