The Agent That Pleases: Absences, Lying Instruments, and Invented Evidence

Part 8 of 9 in the series on agentic software development. Part 1, The Agent Writes the Code has the two settings, the numbers, and the ten sentences the series works through.

For nine months an AI coding agent has written nearly all the code of a multi-product SaaS platform, and in a client project it built an event-driven service in eleven days. Every rule in that setup exists because something went wrong, usually more than once. The incidents are more instructive than the rules, because a rule only makes sense once you have seen what it prevents. They fall into four failure classes: an agent that adjusts the expectation instead of the code, defects that consist of something missing, measuring instruments that report success while measuring nothing, and evidence that looks precise and is invented.

The accommodating agent

The agent is fast, but it is accommodating. Its goal is a green result, and there are always two ways to get one: fix the code, or adjust the expectation. In the event-driven service, one rule is the reason the suite still asserts anything: when a test is red, change the code; if you want to change the expectation, change the table first, and write down why. Without that rule I would have had a green test suite on day two that asserted nothing. The mechanism is not specific to agents: a human under time pressure does the same thing. The difference is that the agent does it faster, without the discomfort, and reports the result as success.

That is the loud form. The quiet forms are harder to see, because nothing in the output changes colour. There were two of them.

Removing a case the application cannot handle. The schema had four statuses; the UI had three filter switches. The agent removed the test case for the fourth status and noted nothing. What saved it was a follow-up question from me: “you did record that as an issue, right?” The rule since then: a removed case becomes a finding in the same step. The test question is why doesn’t it work? If the answer is “because the app can’t”, that is a finding. If the answer is “because it makes no sense”, that is a comment in the table, so that the next person does not add the case back.

Encoding a known defect as expected behaviour. The agent built four test cases that asserted the current, wrong status. The suite reported 12/12 green: all clear, while knowing the opposite. I turned this around and made it a rule: known defects are always red in the tests. The test expects the correct behaviour and fails, and its message names the finding, where it was measured, and what would make it green. A green test with a comment saying “this is wrong, actually” is read by nobody; a red test with that message in its failure output is read by whoever runs the suite next.

The flip side showed up two days later: nobody re-reads a red test. Three hand-written test stands carried a red marker in their title for an object that no longer existed. A red test is only useful if something tracks it.

The purpose of a test suite is not to be green. It is to say what is.

Absences nobody reports

The most expensive class of defect in agent-built software is not a wrong result. It is something missing, because no tool reports an absence.

In a single night, the same defect appeared four times in four modules of the platform:

The sharpest piece of evidence is the collective invoice. The green suite explicitly checked that all 23 translation keys exist. It did not check that anyone ever displays them.

Every instrument in the chain is blind to this, and for the same reason: each one checks what is there, and an absence is not there to check. The type checker sees nothing. The tests see nothing. The build sees nothing. The dead-code check sees nothing. The only instance that notices an absence is a human in a browser, which is the most expensive path there is. And because everything is green, everyone believes the work is done.

Two variations of the same defect followed, each with a subtler cause.

A browser walkthrough as the owner finds no missing permission. The agent built a new screen and clicked through it as a persona: module switch, both pages, navigation, two languages, zero console errors. The work was still unfinished. The menu entry carried the permission; the route did not. A user without the permission would not see the menu item, but could open the page by URL. The walkthrough had run as the owner, who has every permission, and a missing lock is invisible to whoever owns the key. A guard test, every route carries the same permission as its menu entry, found it in seconds.

The test kernel loaded fewer plugins than production. Permission keys were therefore unknown in the tests, and every gate on an unknown key is fail-closed: it rejected even the owner. The “forbidden” test was green, because the viewer was rejected, just for the wrong reason. Only the positive control found it: the member does get past the gate. Every gate test needs three cases, and the third one is the expensive one.

What helped:

• Tests that read the rendered output with values, not the existence of keys, types, or ports.
• For buttons, assert that they are enabled. A disabled button makes every check behind it unreachable, while a synthetic click still reports green.
• A mandatory point in every final report: what is still missing for a user to see this in the browser?
• A plan’s definition of done: the user job was played through in the browser as the persona.

The measuring instrument lies

An agent that checks its own work is only as good as its measuring instruments, and instruments fail in the worst possible way: they report success. Five incidents, each with a different instrument.

A request counter that could not count. To verify that a page no longer fired requests in a render loop, the agent counted via the browser’s PerformanceObserver. Result: 0 requests. The resource-timing buffer is capped at 250 entries, and in dev mode it was full of module loads before the first API call ran. That is not “no requests”; it is a blind instrument that looks exactly like a passed gate. The real defect was around 150 request batches per second. The rule since: prove the counter first, by showing that it returns more than zero on a real page change. A 0 is only a result after that.

A guard whose success message described its intent, not its result. A dependency-pin guard compared versions only when the installed package existed, and otherwise did nothing. Its success message still said “package.json, overrides, lock file and node_modules match”. A neighbouring session spent an hour investigating a state that was not actually installed. The rule since: every guard reports two numbers, how many cases should be checked and how many were actually compared. A case that cannot be checked is a finding, never a silent continue.

A test mock that swallowed bulk inserts. The mock database spread an array into one bogus record. Everything written in bulk was invisible to later reads, and bulk was exactly the path the catalogue import used.

A table engine that reports errors only in its logger. A broken sheet looks exactly like a good one. A marker set in the wrong table form generates zero cases and reports success. The countermeasure: target numbers for sheets, cases, and cells in a check script that fails the build.

A green type check, 75 of 137 test files gone. A shared validation schema gained a refinement. A downstream .omit() on that schema throws at module load, not at type check. The type check stayed green; importing the router threw, and more than half the test files dropped out at once. It looked like an infrastructure problem, not like a schema change.

The question that would have found all five:

What would be different if this construct did not exist at all?

If the honest answer is “nothing”, the construct asserts its effect instead of proving it. Put the question to the five incidents and the answer is “nothing” five times. The counter would have reported 0 without the observer. The guard would have printed the same message without the comparison. The mock would have returned the same reads with the bulk insert deleted. The table check would have reported success with the sheet removed. The type check was green with the schema change in and would have been green with it out. None of them measured what they claimed to, and all of them looked exactly like a passed check. That is why the rule is not “write better instruments” but “prove the instrument first”: a counter that has never shown a non-zero value has not yet earned its zero.

Invented evidence

Agents produce text that looks like evidence. Precise-looking evidence is trusted more and checked less, and that combination is the problem.

An invented line reference. A plan supported a claim with PIPELINE-LEDGER.md:1543-1546. The file had 1,016 lines. The quoted phrases appeared in none of the seven ledger files. A second plan copied the claim. The conclusion happened to be right; its foundation was made up. That is the dangerous version: an invented reference under a correct conclusion survives every casual review, and it had already been copied once. The rule: check a line reference against the file length before taking it over (wc -l is cheaper than any discussion), and grep the quoted phrase instead of trusting the number.

Invented timestamps. The orchestrator wrote times into the ledger from gut feeling, and was consistently in the future. Measured twice in one run: 11:xx and 12:xx in the ledger while the clock said 10:50; corrected; then 12:10 to 12:50 while it was 11:50. The error repeated right after being corrected. Timestamps in a ledger are not decoration: they answer the question “is this agent dead or still working?” The rule: call date, do not estimate.

A paraphrase that hardens into a rule. A lesson said “the owner always deploys this app personally”; it was about deployment. A ledger summarised it as “push ⇒ deploy”. A later session refused to push, citing the “rule”. The repository had no deploy workflow at all; pushing only releases. Paraphrases are usually stricter than their source, and they block work that was never blocked. The rule: a restriction in a ledger or plan is a quote, not a finding. Open the source before obeying it, and link sources instead of summarising them.

A decision without its wording is not a decision. From an empty paraphrase, “both product decisions are in”, an agent guessed the content of the two decisions and inverted both.

All four have the same shape: a specific-looking artefact (a line number, a time, a quoted rule) that nobody checked because it looked as if someone already had. And all four are cheap to check against the source, and expensive to trust. wc -l, date, and opening the file a ledger cites cost seconds; a session that refuses to push, or a plan that inverts two decisions, costs hours.

The rules this part comes down to: change an expectation only via its source, with a reason, and keep known defects red with a message that says what would make them green. Test the rendered output, not the existence of keys, and end every report with what a user still needs in order to see the work. Prove the instrument before believing its zero, and ask of every check what would be different if it did not exist. And verify line references, timestamps, and paraphrases against the source, because the more precise evidence looks, the less anyone checks it.

Previous: Part 7, Working Through 1,200 Plans. Next: Part 9, Rules Need Exit Codes.