When AI writes code ten times faster than humans can review it, testing and code review stop working as trust mechanisms. They were built for a world where people were the slow step. Now the slow step is figuring out whether any of the output deserves to ship.
The Details:
Formal verification proves code matches a specification, not that the specification is correct. A flawless proof can rigorously confirm a bug, because the math only checks consistency between code and spec, never whether the spec captures the real requirement.
Verification pays off for narrow, well-defined properties: memory safety, type soundness, absence of certain concurrency errors. It struggles with ordinary business logic, where writing a complete, correct spec is often harder than writing the code itself.
As AI absorbs more implementation work, the engineer's real job shifts to specification: naming the invariants, edge cases, and conditions that must never occur before any code gets written. AI can help draft that spec by surfacing hidden assumptions, but a human has to own what 'correct' actually means, the same way nobody signs a contract an AI drafted without reading it first.
There's a graduated path short of full formal methods. Write down the two or three properties a module must never violate. Use property-based testing tools like Hypothesis or fast-check to throw thousands of generated inputs at those rules instead of just the examples a human thought to write. Then turn on the strictest static analysis available, strict TypeScript, Rust's borrow checker, mypy strict mode, since these already prove small things about code on every compile.
Bottom Line: Velocity from AI is now cheap and tells you nothing; the teams that win will be the ones who can say exactly what a diff does and does not do.
Enjoy this article?
Listen to the Claude Code Conversations radio show or join the community.
