The mechanics of testing a thunk are straightforward — mark some work items as test work items, write assertions, re-run them. The hard part is deciding which work items and which assertions. A suite of thirty tests over near-identical inputs gives you a comforting pass rate and almost no information.
This article is about that decision. For the controls themselves, see Automated tests for workflow steps and Automated tests for custom tools.
Two laws
Everything below follows from two facts that hold without exception:
Low coverage produces a low-quality thunk. What you never test, you never find out about until a user does.
Low-quality test data produces a low-quality thunk. A test whose expected answer is wrong actively trains you to accept wrong behavior.
Test at two levels
A thunk needs tests at two levels, and each level catches problems the other misses:
The workflow as a whole. Test work items run a realistic input through every step and check what comes out the other end. They catch instructions that are unclear, steps that hand each other the wrong data, and branches that go the wrong way. See Automated tests for workflow steps.
Each custom tool on its own. Tool test cases call one tool directly and check what it returns. They are faster and cheaper than a workflow run, and they pin a fault on the tool instead of on the step that called it. See Automated tests for custom tools.
Write both as you build, not afterwards, and give every test at either level at least one assertion. A test without an assertion runs, but it cannot fail, so it tells you nothing about whether the answer was right.
Choosing test data
Use realistic inputs
The single most common failure of a test suite is that its inputs are cleaner than production. Someone writes six tidy example documents, everything passes, and the thunk falls over on the first real one — which arrives as a scanned PDF, with the fields in a different order, in a language nobody mentioned.
The best inputs come from the actual source: the mailbox, the drive, the system that will feed the thunk. If you cannot use real data for privacy reasons, redact it rather than rewriting it — the messiness is the part you are trying to keep. Before you have real data, for example while a new thunk is still being built, make inputs that look like it: a sample document or image of the same kind from the web, or one made for the test in the real format, with realistic names, amounts and wording. A one-line placeholder in place of a document is not a test.
When you make documents or images for a test, do not make them all from one template. A set of receipts that differ only in the store name and the total tests one layout. Vary what real ones vary:
Layout. Different issuers put the same fields in different places, under different labels, in different fonts and sizes: a narrow till receipt, a full-page hotel bill, an emailed PDF invoice.
Content. Several line items, discounts, tips, more than one tax line, other currencies and date formats, abbreviated or misspelled item names, handwritten notes or totals.
How it was captured. Real files are often photos and scans, not clean exports: taken at an angle, rotated, cropped at the edge, creased or curled paper, faded thermal print, shadows and glare, blur, low resolution, heavy compression, a busy background behind the paper, or two pages in one file.
Keep the values your tests assert on readable, unless the case is about what the thunk should do with a document it cannot read.
Cover the shapes, not the volume
Coverage is about distinct kinds of input, not counts. Twenty invoices from the same supplier in the same format are one test. Aim to have at least one case for each:
Typical case per meaningfully different source, format, or region.
Edge of the range — the largest document, the shortest message, the zero amount, the date at a boundary.
Legitimately missing data — an optional field that is absent, a document with no attachment. What should the thunk do here? Decide, then assert it.
Bad input — the malformed file, the wrong document type, the empty submission. A workflow's behavior on garbage is part of its specification.
A required input left out — decide what the workflow should do when a request arrives without one of its required inputs, and test that it does.
Each branch your workflow can take. If a step routes on a condition, you need an input on each side of it, or that branch is untested.
Do not forget the negative cases
Most suites test that the thunk does the right thing. Fewer test that it does not do the wrong thing — approves what it should escalate, sends what it should hold, fills in a value it should have left empty. These are usually the failures that matter most, and they need their own inputs.
Test conversations with a simulated user
A thunk that talks to people stops at the first question unless something answers. Give the test work item a simulated user who plays the person, and assert on what the conversation leaves behind: a recorded value, a status, a flag set when the person is done. Choose a scenario for each test work item, such as the customer who answers fully, the one who is vague and the one who asks for something the thunk should refuse. See Test a conversation with a simulated user.
Turn every reported problem into a test
This is the highest-return habit in the whole practice. When someone reports that the thunk got something wrong, the case they are complaining about is a perfect test input, already known to be interesting. Capture it as a test work item, write the assertion that would have caught it, then fix the problem.
Do it the same day. A suite grown this way converges on exactly the cases your workflow finds hard.
Choosing assertions
Assert what matters, not everything
The temptation — especially when seeding assertions automatically from a good run — is to assert every property. That produces a brittle suite that fails whenever anything changes, including harmless things, and trains everyone to ignore failures.
For each work item, ask what would actually constitute a wrong answer, and assert that. Three sharp assertions beat fifteen incidental ones. But do not go to the other extreme: every test needs at least one.
Prefer a rule to an AI judgment
If an expectation can be written as a condition on a field, write it as a rule. Rules are free, instant, and deterministic — the same input gives the same verdict forever. AI-graded assertions cost a model call and carry judgment, which is exactly what you want for generated prose and exactly what you do not want for an amount.
A good rule of thumb: rules for structure and constraints (this field is not empty, this value is one of these three, this number is under the limit); AI assertions for content and meaning (the summary mentions the renewal date; the reply does not promise a delivery date).
Make the expected answer match what you told the thunk to produce
Before you save an expected value, check it against the instructions. If the step is configured to write status values from a fixed list, the assertion must use those exact values — not a synonym you happen to prefer. A test that demands something the thunk was never asked to do is not a failing thunk; it is a failing test, and it will burn hours before anyone notices.
Avoid asserting things that change on their own
Do not build assertions on live schedules, external web pages, or anything whose correct answer moves over time. Those tests go stale and start failing for reasons that have nothing to do with your change, which is worse than having no test — it is a test that teaches people to ignore red.
Where a case genuinely depends on the date, control the clock instead: set a Simulated current time in Test Settings so the case is evaluated against a fixed "now".
Making a suite you can actually re-run
A regression suite has to be runnable on demand, repeatedly, without consequences. That means dealing with side effects up front:
Email — set the test email policy to suppress, or redirect to one address you own.
Steps with external effects — a step that files a ticket or updates a record should be marked to be skipped for test work items.
Tools that write somewhere — replay captured tool outputs instead of calling them live, or point them at a sandbox.
If running the suite is scary, it will not be run — and a suite that is not run is not a suite.
Deciding when to ship
Before a thunk carries production work, be able to state three things:
Coverage — which of the expected kinds of input have a test, and which do not. Name the gaps explicitly; an unnamed gap is a surprise waiting to happen.
Pass rate — how the suite currently scores, from an actual run report rather than memory.
Risk and reward — whether the value of shipping now exceeds the cost of the gaps you just named.
Make that call with the people who own the business outcome, not alone. The run report is downloadable for exactly this reason.
Who does what
Testing slips when nobody owns it. Three roles need to be filled, and they are rarely the same person:
Business owners supply the cases and confirm that the expected answers are genuinely correct. They are the only people who can say what "right" means.
Thunk builders make the thunk pass those cases, and investigate every failure rather than adjusting the assertion until it goes green.
Whoever operates the thunk runs the suite on a regular cadence and circulates the results — including when nothing changed.
Name all three at the start of the project. A gap in any one of them shows up as declining quality a few weeks later.
For the broader engineering practices around shipping a thunk to production, see Production readiness and quality control.