Why self-grading does not work

Whoever wrote it holds the mental model of what they meant to do. When reviewing, they compare the code against the intention, not the result. The question “does this work?” becomes “is this how I imagined it?” — and the answer is yes, because they wrote it.

Worse: the same model has an incentive to finish. Saying “done” closes the task; saying “not yet” opens more work. Without role separation, the short path wins.

What separates the two roles

• The critic does not read code, it runs commandsIt runs the test, the lint, the build, the endpoint. Opinion about the code does not count: the exit code does.
• The score is numeric and has a floor0-100 with a declared threshold (85, say). “I think it is fine” does not pass; below the threshold, it fails.
• Failing requires the reasonSaying “it failed” is not enough: the report lists what failed, with the command output. Without it, the agent “fixes” blind.
• The critic does not fixIt measures. The one who fixes is the author — and goes back to the same ruler until it passes.

In practice

The same loop, the same code, two different answers because the second round had a deliberately broken test. The critic did not know that — it just measured.

What changes day to day

1. You stop reviewing promisesThe report arrives with a score and a reason, not with “implemented successfully”.
2. The agent improves on its ownWith the failure reason in hand, it fixes what broke instead of rewriting everything.
3. You see the regressionA score that drops between two rounds signals something broke — before it reaches production.

See the AI Test Loop skill See all skills

Want this wired into your project? Guided setup from R$ 1.500, with the first cycle run together.