Guardrails Aren’t Enough. Who Verifies the Guardrails?

A control is an artefact. Artefacts need evidence too.

Most of us respond to a bad outcome the same way. We write a rule.

The rule is usually a good one. It names the thing that went wrong and forbids it. Some months later there is a document full of these, and that document is quietly treated as the answer to whether the work is under control.

I want to make a narrow argument about that document. Not that the rules in it are wrong. In our own case they were almost all right, and several have since demonstrably saved us something. The argument is more awkward than that. A control is itself an artefact. Somebody wrote it, on a particular day, against a state of the world that has since moved. And there is nothing inside a control that checks the control.

What a good control actually looks like

Three properties make a rule worth having.

It is specific: it names what it forbids precisely enough that two people would agree whether it had been breached. It is recorded, somewhere durable rather than in a shared understanding. And it is enforceable: something is positioned to stop the work when the rule is broken.

Our own governance record meets all three, and I would defend most of its individual clauses today. That is rather the point. What follows is not a story about slack rules.

Two rules, each correct, jointly impossible

Earlier this year we wrote an instruction document for work on our own software. I have described it once already in this series, from the point of view of what a review of parts cannot see. I want to look at it again from the other side, as a collection of controls.

Four times in that one document, a step was commanded and the only means of performing it was forbidden. The steps were necessary. The constraints were justified. Each pair described work that could not be done at all.

At its simplest the collision looked like this:

Step. Produce the list of files to be examined, derived the way the system already derives it.

Constraint. The worker may not call into the part of the system that performs that derivation.

Both lines are defensible on their own. The step is how the work gets done correctly rather than approximately. The constraint exists because letting a worker reach into that part of the system invites the exact class of accident it was written to prevent. Neither clause is wrong about the world. Together they are an instruction to do something and a prohibition on the only way of doing it.

Not one of those four collisions involved a wrong fact. Every clause had been checked against the actual state of the code, on disk, the day it was written.

Why none of the controls stopped it

Here I have to be careful about what I can and cannot claim, so I will mark it plainly: what follows is our reading of why this happened. It is not something we measured.

Our reading is that the collisions survived because every clause was checked against the world and none was checked against the others. A reviewer asking whether a constraint is justified gets a yes. A reviewer asking whether a step corresponds to something the system really does gets a yes. Neither is positioned to ask the question that finds this defect, which is whether the document grants the means for everything it demands.

That is an attribution and I would not defend it as a measurement.

Controls that were true on the day they were written

There is a second failure mode, and it is quieter.

One of our rules protects a piece of published text from being overwritten. It exists because a bulk rewrite was once proposed that would have destroyed it, and there is no version history to recover it from. The rule is correct and it still stands. But its written justification said that text sat behind the only live address we had, and that was false at the time it was written. There were three. The control survived unchanged in effect; the reason it gave for its own existence had to be corrected after the fact.

Another rule requires every public piece to carry a five point disclosure block. To check compliance across everything already drafted, we counted keyword matches. The rule’s own record states the limit of that honestly: a keyword count measures absence reliably and presence only weakly, so every score it produces is a ceiling rather than a result. It was later found wrong in precisely the direction that warning predicted. One document scored one point out of five by keyword and scored zero when somebody read it.

Neither of those is a bad rule. Both are the same thing seen twice. A control is a claim about the world, made on a day, by a person, using an instrument. The claim can decay. The instrument can be incapable of establishing what the rule asserts. And nothing in the rule reports either condition, because a rule has no mechanism for noticing that it has stopped being true.

What actually caught the collisions

Not a control.

What caught them was a second party reading the document as the one who would have to carry it out, discovering that it could not be carried out, and stopping.

Three times in a single day, work halted having produced nothing whatsoever. Not a partial attempt, not a workaround, not a best effort with a note attached. Zero output, and in each case a written statement of which instruction could not be followed and why. Two named a contradiction inside the document itself. The third stopped because the authorisation had not been granted in the place where authorisation is recorded, whatever had been said out loud.

That is a cheap instrument and an unsophisticated one. It is also the only thing in this account that worked on the day it mattered.

A control you have never seen fire

Which brings me to the part I would ask you to take away.

We do have controls that have been observed doing their job. A gate that releases evidence only at a particular stage was watched refusing before that stage and granting at it, both directions, on the same run. A setting that suppresses one class of outbound lookup was checked afterwards and found to have suppressed it, on a run where a great many lookups would otherwise have gone out. Those two are no longer claims. Somebody watched them fire.

Most of our controls are not in that category. I would guess most of yours are not either. They are written, they are agreed, they are cited approvingly, and not one has ever been observed stopping anything. A rule in that state is indistinguishable, from outside and from inside, from a rule that does not work. No examination of the text separates the two.

I want to be exact about the size of this. It is one governance record, ours, across a few months of our own work. I am not claiming that controls generally fail in these ways and I have nothing that would support such a claim. The claim is narrower: the controls in this record conflicted, went stale, were checked by instruments that could not establish what they asserted, and were not self verifying. I know of no reason yours would be different in kind, and no evidence that they are.

A control you have never seen fire is a claim, not a control.

So: when did you last watch one of your controls actually stop something?

Where this comes from, and its limits

I sell software review services. That is what I do for a living, and it is the reason I have opinions about this.

Findings referenced here are anonymised. The evidence has stated coverage limits, and they are in the body above rather than in a footnote: this is one governance record from our own work, examined by us, and it supports no claim about prevalence. Security findings are candidates, not confirmed exploits. No secret values are shown. Code examples, where used, are illustrative composites and not audited source, and the two rules quoted above are our own document’s, rendered in substance rather than verbatim.