Hunter Iceton

Choosing the control

Use: an AI agent or a person keeps missing something it should do. This decides what kind of fix to build (a generated file, a blocking check, a warning, a tool, a scheduled job, an instruction, or a question for a human) and then how to check that the fix is actually working.


The card

Part 1: choose the control. Ask in order and stop at the first yes.

  1. Can the wrong state be made impossible? Then eliminate it, and build no check.
  2. Can the pass/fail rule be written as a test? Then block on it.
  3. Can the right path be made the easy path? Then build a tool for it.
  4. Is there an event at the moment of the mistake? Then detect it there, and warn.
  5. Is it periodic? Then schedule it, and name who reads the report.
  6. None of these? Then write it as an instruction, and call it monitored, not enforced.

Then label it. Can you name who checks it and what happens if it is missed? If yes, it is a rule. If no, it is a norm, a hope. Norms are legitimate, but they are not coverage, so count them separately.

Separately, always ask: could the failure be the work answered a different question than the one asked? No automation can see that. Add a human review or an adversarial second pass, whatever you chose above.

Part 2: check the control works. It is only real if it is fired (something runs it without anyone remembering), honest (it reports how much it looked at, and says so when it couldn't look) and landed (its report goes somewhere a person reads before acting).

identify the failure  →  choose the control (Part 1)  →  verify it fires, reports and lands (Part 2)  →  monitor

Part 1 — Choose the control

The decision tree

Ask in order. Stop at the first answer that resolves it.

1. Can the wrong state be made impossible?
   (derive it, generate it, remove it, move it so it cannot diverge)
   ├─ yes → Do that. Build no check at all.
   └─ no  → 2

2. Can the pass/fail rule be written as a mechanical test,
   decided in advance, with no judgement needed at the moment?
   ├─ yes → BLOCK: a refusing check (pre-commit hook, CI job,
   │        type checker, schema, permission rule).
   └─ no  → never build a blocking check for it → 3

3. Can the right path be made the easy path?
   (a script, a tool the agent can call, a slash command)
   ├─ yes → Build the TOOL. Then the mistake is the long way round.
   └─ no  → 4

4. Is there an event at the moment the mistake happens?
   ├─ yes → DETECT there, and let the agent/person decide
   │        (a hook that adds a warning to context, not one that refuses).
   └─ no  → Is it periodic?
            ├─ yes → a SCHEDULED job, with its report read at a fixed point.
            └─ no  → an INSTRUCTED step in a named checklist.
                     Label it "monitored, not enforced".

Then, for whatever you chose:

   LABEL — Can you name WHO checks it and WHAT HAPPENS if it is missed?
   ├─ yes → it is a RULE.
   └─ no  → it is a NORM (a hope). Either add a checker and a consequence,
            or keep it and call it a norm. Do not reword it more strongly;
            that changes nothing.

   PARALLEL — Could the failure be "the work answered a different question
   than the one asked"?
   └─ yes → No automation can see this. Add a HUMAN review and/or a
            deliberate adversarial second pass, on top of the control above.

Why the norm question comes last. Asked first, it is a gate everything walks through: "is it a rule? sure, I'll call it one." Asked last, it is an honest label on what you actually built.

Why the wrong-question failure is a separate axis. It is not the weakest kind of control; it is a different kind of failure. A system can pass every test, every hook and every review of its own output and still have solved the wrong problem. That failure (goal substitution) is the one AI agents are best at producing plausibly. This framework does not solve it. It makes sure you never think you have covered it by accident.


Which mechanism is good for what

mechanism use it for qualifying question do not use it for
eliminate (derive / generate / relocate) removing a whole class of drift can the wrong state be made impossible? — it is always the first thing to try
blocking check rules expressible as a test can I write the predicate down? anything needing judgement at the time
tool (script / MCP tool / slash command) making the right way the easiest way is there one correct way to do this, done often? rules the agent must follow when not using the tool
context hook (adds a warning) catching a mistake as it happens is there an event at that moment? enforcing — it only informs
scheduled job periodic sweeps with no trigger does it need no trigger? anything whose report nobody is sent to read
skill (loaded on demand) long reference material, needed rarely is it long and situational? making a rule more reliable
standing instruction (system prompt / agent file) short rules, always relevant is it short and always relevant? long procedures
human wrong-question failures, taste, priorities can only a person see it? anything a test could check

A skill is not a reliability upgrade. A skill and an instruction are both triggered by the model's judgement. Moving a short, always-relevant rule into a skill makes it less reliable, because it now depends on the skill being retrieved.

A tool is not enforcement either. It works by making the wrong path more effort than the right one, so agents and people take the right one most of the time. If the wrong path must be impossible, you need a block as well.


Tools — making the right path the easy path

Agents, like people, take the path of least effort. If doing something correctly takes six steps and doing it wrong takes one, a well-worded instruction will lose to the shortcut on a busy day. So, before you write the instruction, ask whether you can package the six steps as one.

form what it is use it when
script a file anyone can run (./release.sh, make deploy) the steps are shell commands, and people as well as agents need them
slash command a saved prompt the person types (/review, /deploy) a person starts the task and wants it done the same way every time
MCP tool a named function the agent itself can call, with typed inputs the agent should be able to do the task mid-work, without being told the steps

MCP versus an API, plainly. An API is a door a program offers to other programs: send this request, get that response. Your code has to know the door exists, how to knock and what comes back. MCP (Model Context Protocol) is a standard way of describing doors to an AI agent: each tool has a name, a plain-English description and a list of inputs, so the agent can read the description, decide the tool fits and call it. An MCP server is usually a thin wrapper around an API. The API does the work; MCP is how the agent finds out the work can be done and how to ask for it.

Why it matters for control. A tool carries its rules inside it. A deploy tool that always runs the tests first means "run the tests before deploying" no longer depends on anyone remembering. Its description is what the agent reads, so write it as carefully as you would write the instruction it replaces.


Practitioner controls

The tree names kinds of control. These are the everyday forms they take in a codebase, sorted by which question they answer.

control kind what it guarantees the trap
generated files (code from a schema, docs from the code) eliminate the copy cannot disagree with the source someone edits the generated file by hand; mark it do not edit and regenerate in CI
types (TypeScript, Python type hints with a checker) block a whole class of wrong calls cannot compile or pass CI any and ignores quietly switch it off; count them
schemas (JSON Schema, Pydantic, database constraints) block bad data is refused at the boundary, not discovered later a schema that is checked only in one path; validate where the data enters
unit tests block (in CI) a known behaviour stays true tests the agent wrote to pass its own code; see second-model review
CI (tests and checks on every push or pull request) block nothing merges red a red that everyone has learned to ignore; see Part 2, known versus new
pre-commit hooks block the mistake never reaches the shared history they run only where they are installed; repeat the important ones in CI
permissions (deny rules, read-only paths, scoped tokens) block the agent cannot do the dangerous thing permissions granted once "to get unblocked" and never taken back
sandboxing (containers, no network, a throwaway copy of the repo) eliminate a mistake cannot reach anything that matters the sandbox quietly has more access than you think; test it by trying
second-model review (a different model, or a fresh session, reviews the work against the original request) human-adjacent a second reader who did not write it a reviewer given the author's summary instead of the request; give it the request

On tests written by the agent. An agent that writes the code and the tests can write tests that check what it built rather than what was asked. Pin tests to real examples and to the original requirement, and have someone (a person or a second model) read the tests as well as the code.


Ranking rule

Prefer controls that depend least on the compliance of whoever is being controlled:

eliminate  >  block  >  tool  >  detect  >  schedule  >  instruct  >  norm

Stronger is not always better. Blocking a judgement call produces false refusals, and people learn to route around them. Use the strongest control whose qualifying question is honestly yes.


Definitions


Worked examples

situation tree result fix
Two files must always hold the same list, and drift Q1 — eliminate generate one from the other; delete the check
Commits must never contain a secret Q2 — block pre-commit secret scanner, repeated in CI
An agent edits a config file it should only read Q2 — block a permission rule denying writes to that path
A function is called with the wrong argument types Q2 — block type hints plus a type checker in CI
Releases need six manual steps and one keeps being skipped Q3 — tool one release script; the agent and the person both call it
An agent writes a file without choosing a line ending Q4 — detect a hook that warns on the diff; the agent fixes it
Nightly: is the test suite green? Q5 — schedule nightly job plus a start-of-day step that reads its report
"Always read the brief before editing" Q6 — instruct a named checklist step; call it monitored
"Be concise, match the codebase style" label — norm keep as a short instruction; label it a norm
The agent solved a neighbouring problem, not the one asked parallel — human review against the original request, not the agent's summary
A 300-line procedure used once a month table — skill skill, loaded when triggered

Questions to run over an existing set of rules

  1. For each rule: what is its actuator? If "someone remembering", is that acceptable?
  2. Which "rules" are actually norms? Count them separately.
  3. Which checks could be deleted by eliminating the state they guard?
  4. Which instructions describe a procedure that could be a tool instead?
  5. Which blocking checks enforce a judgement call? Downgrade them to detection.
  6. Where is the wrong-question failure covered at all? If nowhere, say so.

Part 2 — Is the control working?

A control you have built is not yet a control you have. It becomes one when three links hold, and each fails on its own schedule.

1 · Fired — something runs it

actuator what runs it example
blocking the action can't happen until it passes a pre-commit hook that refuses the commit
automatic a timer or an event, and nobody decides a nightly job
instructed a documented command someone has to choose to run "run this at the start of every session"
prose nothing. It fires when someone notices it applies a paragraph in a guidelines document

2 · Honest — it reports what it actually saw

3 · Landed — the report reaches someone who acts

The pattern behind most failures here: the write worked and the read never happened. If a check's output lands in a place nobody is sent to, it has the first two links and none of the value.


Hunter Iceton. Draws on the institutional-grammar tradition (Crawford & Ostrom: a rule names who monitors and what sanction follows), the hierarchy of controls from occupational safety (eliminate before you guard), error-proofing from lean manufacturing (prevent where possible, detect where not), and levels-of-automation research (automate detection up to the decision, not through it).