Context
Feature work and bug fixes where "done" must mean something objective. Especially where the agent also has write access to the test suite - which, in most harnesses, it does by default.
Problem
An agent asked to make tests pass will make tests pass. When it authored both the implementation and the assertions, the tests encode whatever the code does, and the whole suite becomes a tautology with good formatting. The failure is silent: everything is green.
Pattern
- The human writes the acceptance test first - or at minimum reviews and freezes it before implementation starts. It fails for the right reason before any code exists.
- The test files are marked read-only for the run. Either by permissions or by an explicit instruction with a check afterwards: if the test file's hash changed, the run failed.
- A mutation probe after the run. Deliberately break the implementation - invert a condition, drop a guard - and confirm the test notices. If it does not, the test is decorative.
- Revert probe. Run the new tests against the previous implementation. They must fail.
Steps 3 and 4 are mechanical, cheap, and can be performed by the agent itself, which converts "I believe this is tested" into an artefact you can look at.
Tradeoffs
Writing the test first is real work, and it is front-loaded into the moment when you were hoping to delegate. For exploratory work the test may be wrong in ways you only discover while implementing - in which case stop the run and fix the test, rather than letting the implementation define truth.
When not to use it
Spikes, prototypes, and anything that will be thrown away. Also genuinely subjective work - visual design, copy, API ergonomics - where an assertion would encode taste badly.