Keeping rules honest
A set of instructions is not free to grow. Compliance with any one of them falls as the number rises, which is measured behaviour in language models. So a working rule set has to be small, and staying small means rules get removed. This is the test each rule has to pass to stay in, and it is four questions.
The four declarations
Every rule in the set states these four things about itself, in the same place the rule lives. A rule that cannot state them is re-engineered until it can, or it is cut.
What counts as a catch
The third question is the one that does the work, and it is easy to answer dishonestly. Two answers do not count.
The first is what the rule is for. Every rule has a purpose, including the ones that do nothing, so a purpose cannot separate them.
The second is how often it fired. A count of firings looks like evidence and is not. What matters is a log of catches, not a log of firings, because a mechanism that fires constantly and has never changed an outcome is reliable theatre. The bar is counterfactual: name the specific thing that would have gone out wrong had the rule not been there. If nothing can be named, the honest entry is that nothing can be named, and that entry is what the review acts on.
Removal is the normal outcome
The test only means something if rules are actually removed by it, and a set where nothing has ever been cut is a set where the test is not being run. Four shapes of removal recur, each with the evidence that justified it.
| what the record showed | what it means |
|---|---|
| fired regularly, caught nothing in its lifetime | the class of error it targets does not occur in this work, or something upstream already catches it. The mechanism is cost with no return. |
| caught one thing early, then went cold for months | usually a real catch against a habit that has since changed. The rule was right once. That is not a reason to keep it now. |
| caught nothing the surrounding tooling was not already catching | redundant enforcement. Something upstream already does the job, so this mechanism is paying for a boundary that is already held. |
| never used in practice, whatever the intent | the honest case, and the most common. Something was built, it was reasonable, nobody reached for it. It goes. |
Recording why each one went is part of the removal. A cut with its reason on record can be reversed on evidence. A cut with no recorded reason leaves nothing to stop the same mechanism being proposed again.
What the test does not do
It does not tell you whether a rule is a good rule. It tells you whether the rule is doing anything. Those are different questions and only the second one can be answered from a record, which is the whole reason the test is written this way.
It also cannot force the removal it recommends. A rule with a blank catch column is usually somebody's considered principle, and the declaration only helps if a blank is allowed to stand as a verdict rather than as a task for later. The test makes the state visible. Acting on it stays a decision.