Skills.

The most expensive agent mistake is not a wrong command: it is validating what already existed and presenting it as work done. The report looks flawless and the artifact does not exist.

The symptom

You ask “publish today's post” and get “I confirmed the post is published, all good”. Except the post is yesterday's. The agent read the state, found something close to the request and declared success — without producing anything.

Verifying is cheap and always goes green when the world already looks the way you wanted. Producing requires creating something that did not exist.

Three questions that separate the two

  1. Which new artifact exists now?A file, a record, an ID, a URL. If the answer is “the one that was already there”, there was no delivery.
  2. What is its creation timestamp?Before the request = not yours. Ask for the identifier the system returned on creation (not the one you found afterwards).
  3. What changed in the world?If nothing changed and the task was to create, the task was not done — however green the check looks.

The check that catches this

Write the check against the expected state after the action, not against the existence of something similar:

# weak: passes with yesterday's post grep -q "post" page.html # strong: requires the artifact from NOW test "$(stat -c %Y page.html)" -ge "$START"

The same idea holds for message delivery: the script's exit code does not prove the message arrived — the delivered message id does.

Why this happens so often

Because the path of least effort is always to read the state and report. And because “all good” is socially more comfortable than “I could not”. The antidote is demanding the creation identifier on every delivery — whoever has no ID has no delivery.

See the AI Test Loop skill See all skills

Want this wired into your project? Guided setup from R$ 1.500, with the first cycle run together.

← all skills

Get the next article

Sent when I publish, together with the free skill used here. No spam, one-click unsubscribe.

Payment processed by Stripe — no card data touches this server. Instant delivery once confirmed. Prices in BRL (Brazilian real) — you are charged in BRL. Terms, delivery and refund.