Hardening takes a thunk that already works on its main path and removes the causes of the failures that are left. Each change it makes adds structure — a fixed list of allowed values, a required output, a tool, a check — that makes the design more exact, and also harder to change.
This article has two parts. Concepts explains when to harden and how to work through the failures. Details lists the mechanisms that remove a cause, and how to avoid fitting the thunk to its test cases. For how the platform keeps AI work reliable in general, see AI reliability concepts and principles.
Concepts
Harden a design that has settled
Harden a thunk when its steps, properties and tools have stopped changing, its main-path tests pass, and you want reliability as high as it will go: before you move it into production, or before you measure it against a benchmark.
Do not harden while the requirements and logic are still changing. Each hardening change fits the design as it is today, and the next change in the requirements has to undo it. While a thunk is being built, keep the design simple and fix only what the main-path tests show is broken. See Building a test plan for the tests to have in place first.
Work through the failures by cause
Run the test work items and read every failed assertion, test by test, not only the totals (Automated tests). Run the same set more than once without changing anything, to see how much the totals move on their own. A difference smaller than that is noise.
Sort each recurring failure by its cause before you change the thunk:
The test is wrong. The expected value is not in the input, or it contradicts a written rule. Fix the test.
Two answers are defensible. Accept both, or write the rule down so that only one follows.
A rule nobody wrote down. The expected answer follows a convention the agent cannot read. Add it to the documents the thunk reads.
The rules are unclear or contradict each other. Fix the wording, in the documents or the instructions, so the rule says one thing.
A real agent error. This is what the next change targets. Read the step's run to see where it went wrong (Step diagnostics).
Change one thing at a time, and compare test by test with the previous run: which tests the change fixed and which it broke. A design change and a model change made together cannot be told apart.
Stop at diminishing returns: when the failures left are scattered single errors with no common cause to remove.
Details
Remove the cause with a mechanism, not a sentence
A sentence asking the agent not to fail is read and interpreted; a property type, a binding or a tool is applied exactly. Prefer the mechanism that makes the failure impossible. Roughly in this order:
A typed property. A result with a known set of values gets a fixed list of allowed values, also when the property holds a list. A yes/no answer is a Yes/No property. A value with several parts is a Structure Type. See "Choosing a property type" in Schematized state.
Bindings. A step has as inputs the properties its rules need. A property a step must not change is bound as an input only.
Required outputs. A result the step must produce is marked Required to finish, so the step cannot end without it.
The tools a step can use. Turn off the tools a step does not need (Control which tools a step can use), including a search over a folder that has nothing in it.
The property description. What a value means goes in the property's description, where every step that reads or records it sees it.
The shape of the instructions. When the agent skips an action or does it too early, check the numbered process first: one action per item, detail nested under its item, and a later action naming the earlier result it uses. Don't add a step to force an order. See "Number the process" and "When the agent skips or reorders an action" in How to write effective AI instructions.
A code tool for a fixed rule. When the agent keeps getting wrong something the written rules already decide (a priority from recorded criteria, a deadline from a date), move that decision into a code tool that takes the recorded, typed facts as inputs. Keep reference data in a spreadsheet the tool reads, keep judgment about free text with the agent, and keep the policy document as the written source of the rule. See "What belongs in a code tool" in How custom code tools work.
A finish check for a condition the step's own properties can show (Control when a step runs and proceeds). Never one that counts or measures.
Name a tool only on evidence
Instructions never name tools: the tools turned on for a step decide which tool does the work. When the agent calls the wrong tool, turn off the tools the step should not use, and make the right tool's name and description say what it is for.
Naming a tool in a step's instructions is a last resort, for hardening only. Use it only when the step's runs show the agent calling the wrong tool, or not calling the right one, on recurring failures after the tool set and descriptions were fixed. Note why you named it, so the next person to edit the step knows. The design review reports instructions that name a tool; for this one case, that finding is expected.
Don't fit the thunk to its tests
The test work items are a sample of the work the thunk will see. A change that only fits them makes the thunk worse on everything else.
No examples shaped like a test case. Instructions, property descriptions and tool descriptions state rules in general terms; they never paraphrase a test input.
No test-specific data. No expected values, case numbers or lookup tables keyed to the test inputs.
No rule the documents do not state. If a fix needs a rule your business documents leave open, decide it with the people who own the process and add it to the documents, not only to the thunk.
Check before you finish. Search the instructions, descriptions and tool code for words from the test inputs, and read every match.
After hardening
Run the design review once the set of changes is finished (Workflow review), and run the main-path tests on a second AI model (Move a thunk to a new AI model). A step that passes on one model and fails on another usually has a fault in its design that one model happened to work around.
