A retry policy that starts as three attempts with loud failures can become seven attempts with swallowed exceptions inside six months, and every single change that got it there passed review. That gap between what a team believes it built and what actually runs in production is specification drift, and AI coding assistants make it happen faster than ever before.
The Details:
AI tools optimize for the diff in front of them, not the contract behind it. Ask an assistant to handle errors more gracefully and it will do exactly that, cleanly and locally, without knowing a retry policy was ever a deliberate decision someone made two years ago. There is no comment or test asserting the rule, so nothing stops the change.
Once behavior drifts, the code itself becomes the new source of truth. The assistant does not see seven retry attempts as a mistake to correct; it sees them as the existing design and will defend and extend that version, writing tests that lock the drifted behavior in place.
Documentation fails here because nobody rereads a wiki while editing a function. A test named for the exact rule it protects, something like refunds_never_exceed_original_charge, works because breaking it turns CI red and forces a conversation. That test is an artifact allowed to disagree with a prompt, which a paragraph of prose never is.
Prompting matters as much as testing. Asking an assistant to preserve a named invariant while adding a feature keeps the original intent in the room before code gets written. A monthly check, where the AI reconstructs a critical path's behavior from code alone and a human compares it to what the team believes, surfaces drift while it is still small.
Bottom Line: Care alone cannot stop drift; only a structural checkpoint, a named test or a stated invariant, keeps fast AI changes honest to original intent.
Enjoy this article?
Listen to the Claude Code Conversations radio show or join the community.
