Sometimes Zero Bytes Is the Right Result

A system that stops rather than guesses may be doing the most valuable work available to it.

We judge automated systems by what they produce.

That is not a foolish instinct. For most of the history of software it was a reasonable proxy: a program that returned nothing had crashed, and one that returned something had worked. Output meant the machinery was alive.

Then we started asking machines to do work whose difficulty lies in judgement rather than mechanics, and the proxy stopped holding. Such a system can always return something. Fluent, plausible, correctly formatted, on time. Whether it should have is a separate question, and nothing about the output raises it.

I want to describe a day on which our own tooling produced nothing at all, three times, and argue that those were the three most useful results we got.

Three stops in one day

On a single day in September, work on our own software halted three times before a single byte of output had been written. Not part way. Not with a partial result and a caveat. Nothing.

Each stop came with a written statement of what was wrong with the instruction it had been given.

The first: the instruction required a list of files to be produced in a particular way, and separately forbade the only means of producing it. The work could not be done as specified. The worker said so and stopped.

The second: the instruction required that a standing restriction on information leaving the machine be honoured, and also required an interface that had no way of expressing it. Before stopping, the worker measured what proceeding would have cost: a specific count of names describing the examined application would have left the machine, to a third party, in breach of a ruling made weeks earlier.

The third had nothing to do with contradictions. The work had been requested out loud; the worker checked the place where authorisation is actually recorded and found it had not been granted there. It stopped, and wrote down the distinction: being told to do something authorises the work, and the record of authorisation is a separate thing that only one party writes.

Three stops. Zero output. Three written accounts of why.

The two refusals that were not required

The stops above were, in a sense, forced: the instructions could not be executed, so nothing was.

The next part was not forced, and it is the part I would keep if I could keep only one thing.

On the second stop there were ways to proceed. Two of them were straightforward and both would have worked.

The first was to remove a component from the machine — the one that made the unwanted outbound call possible. With it gone the call could not happen, the restriction could not be breached, and the run could go ahead.

The second was to take one file out of the list of things to be examined. That file was the one naming the application’s external dependencies. Without it there would have been nothing to look up, and again the run could proceed.

Both were refused, and the stated reason was the same for both.

Either one would have produced a run that reported a class of risk as not examined — truthfully, on the face of it. But the reason it was not examined would have been that we had quietly removed the means of examining it, and the report would have presented that as a fact about the environment. The record puts it plainly: this exercise is a measurement of truthfulness, and a pass bought with a lie about the instrument is not a pass.

There was a third thing not done, for the same reason. The worker also declined to build the machinery for the run and leave it in place unused. A working mechanism whose next step performs the forbidden action is not a partial delivery. It is a loaded instrument left lying in the tree.

Why I can tell you this

An article arguing that a system’s account of itself is insufficient cannot then rest on a system’s account of itself, so it is worth saying where these facts come from.

They are not taken from the worker’s summary of its own conduct. They come from a contemporaneous record written at the time of each stop, checked afterwards by a different party working from the underlying material rather than from the account. That check confirmed all three stops as correct and endorsed both refusals without qualification. Without it, this edition would be an anecdote about how well we think we behaved.

The cheapest place to be wrong

Here is my reading of why this matters, and I will mark it as a reading rather than a measurement.

Every process has a point at which being wrong is cheapest, and it is always before the irreversible step. A stop before the first byte costs an hour of attention and a paragraph of explanation. The same error found after delivery costs the work, the trust, and the far more expensive exercise of establishing which parts of the output were affected and which were not.

The stops described above cost a day. What they bought was that no defective work product exists to be found later, and no conclusion rests on a run we would now have to withdraw.

That is an argument about cost, and I hold it as an inference rather than as a number. I have not measured the counterfactual and I do not know how.

What a system that cannot refuse is telling you

The stronger form of this is an interpretation, and it is mine.

If a system has no mechanism for declining, its completions carry no information. Every task returns output, so the output cannot distinguish tasks it handled well from tasks it could not handle at all. Success and failure produce the same shape.

A system that can refuse, and does, is telling you something every time it does not. Its completions start to mean something, because they are now the result of a decision rather than the only available behaviour.

That is why I would rather have a tool that stops on me occasionally than one that has never once declined anything I have asked of it. The second tool is not more capable. It is only quieter about its limits.

The bound, which matters as much as the argument

I am not claiming that refusal is always right. A system that stops constantly is useless, and refusal can be a way of avoiding work as easily as a way of protecting it. Nothing here is a claim about how automated systems behave in general. These are our own stops, on our own work, on one day.

And the honest version of this account includes the occasion it did not happen. On the same body of work, the same worker had a rule requiring it to halt when a particular condition was detected. The condition occurred. It logged the fact and carried on. It reported that itself, unprompted, and the record treats it as a genuine breach — while separately recording that the rule was unobeyable as written, which does not excuse the breach and is not offered as one.

So the discipline is not perfect and I am not offering it as perfect. What makes a stop evidence is not the stopping. It is the reason attached to it, written down at the time, in a form somebody else can check.

The output was nothing. The information was everything.

So when your tools stop: do you find out why, or do you route around them?

Where this comes from, and its limits

I sell software review services. That is what I do for a living, and it is the reason I have opinions about this.

Findings referenced here are anonymised. The evidence has stated coverage limits, and they are in the body above rather than in a footnote: this is one day of our own work, recorded by us and checked by us, and it supports no claim about how automated systems behave generally. Security findings are candidates, not confirmed exploits. No secret values are shown. Code examples, where used, are illustrative composites and not audited source, and this edition contains none.