Skip to main content

Analysis of AI behavior

Open a finished run to see every tool call, how long each part took, what it cost, and why it failed.

When a step produces the wrong answer, takes too long, or fails outright, the record of what it actually did is kept. This article covers where that record lives and how to read it.

Opening a run

Anywhere a step's agent has worked — on the step itself, on a work item, in an error report, in a tool's call history — its activity appears as a single line you can click to expand. Expanding it opens the run in full, in the order it happened:

  • What the agent was thinking before each action.

  • Every tool call, with the arguments the agent passed and the result that came back.

  • Approval requests, and who responded.

  • How the run ended — its conclusion, or the error that stopped it.

Nested runs expand too. When a step calls a custom AI tool, that tool's own agent run appears inside the parent run and can be opened the same way, so you can follow a decision down through several layers without leaving the page.

This is the single most useful diagnostic in the product. Most "why did it do that?" questions are answered by the arguments of one tool call — a search that returned nothing, a document the agent could not read, a value it filled in from the wrong property.

Reading the resources line

A finished run carries a summary line: when it started, its Total duration, the time it spent waiting on the AI model and on tools, and what it cost.

The split is the interesting part. A run that took four minutes and spent three of them in tools has a very different problem from one that spent three minutes in the model. The first is usually about which tools the step reaches for and how much data they return; the second is usually about how much you are asking the model to do in one step.

Why so long?

Next to the total, a bar-chart icon offers Why so long?. It walks the run, groups the work by what caused it, ranks it by duration, and explains what it found. The same icon sits beside a step's time in the workflow plan, where it answers the question for the step as a whole, across every run it made.

Why so long? is available to the thunk's owner and admins. On a run, it opens a conversation: after the breakdown, you can ask a follow-up, such as why one tool call was slow, and it looks further into that part of the run.

The breakdown is measured, not guessed: every duration and percentage is computed from the run itself. What the explanation adds is the part you cannot compute — why six calls to the same tool happened at all, what that says about how the step is written, and what to change. Typical findings are a tool called repeatedly where one call would do, a retry after a failure that cost the run twice, and a nested tool run that dominates its parent. For the full list of shapes and what each means, see Why a run takes the time it does.

A call that repeats an earlier one with exactly the same inputs is marked. Its group shows repeated calls, and in In order the call names the one it repeated, for example same arguments as call 1. Getting the same answer twice is rarely what a step meant to do. The explanation also reads the instructions the step or tool ran with and the tools it could reach, so its advice is about how the step is written. It runs on the thunk's planning model.

The top of the breakdown shows two figures side by side: Wall-clock, the time from start to finish, and Agent working time, what is left after taking out time spent waiting on a person and, for a step, time between runs. They can be far apart. A step that emails a supplier and hears back the next morning has a day of wall-clock and a couple of minutes of work.

Waiting on a person is its own band and its own rows, never counted as tool time or Platform overhead. That includes an approval, a reply typed into the run, and a tool that sent someone a message and paused until they answered, even when that happened inside a custom AI tool.

Some of the time belongs to the platform, not to your step. Queued is time a run waited for the platform to pick it up. Setting up is time spent preparing the step before the agent started. Wrapping up is time spent saving the step's results after a run finished. A step pays these each time a run starts, and again each time a paused run picks back up, such as after someone replies. They count toward agent working time, and nothing in your step needs to change to reduce them. Platform overhead is the rest of the platform's time between the agent's actions. Between runs is only time when nothing was running or waiting to run, such as a step waiting for its trigger.

Where the time went can be read two ways. Largest first groups the work by what caused it and ranks it, which answers what cost the most. In order lists every call in the order it happened, with a step's calls grouped under each of its runs, which answers what happened and when. Each model turn shows how many tokens it used — input (and how much of it was served from cache), output and, where reported, reasoning — because a slow turn is often a large one.

Each model turn also shows its output speed in tokens per second. The speed is measured over the whole turn, including the time the model spent reading its input, so the model wrote at least that fast. A turn much slower than the others around it is worth a closer look.

In In order, open a call to see where it sits among the calls like it: for example Call 3 of 6 in its group, the longest, or failed, then retried. Below that is the explanation Largest first gives for the whole group. It describes all the calls in the group, not only the one you opened.

What a work item records

Open a work item and you land on its detail pane, which organizes the same history around the item rather than around the agent's run. Its tabs are:

  • Input — the values the step was given. Expand Details inside it for the raw property values.

  • Result — the values it produced, with the same detail view.

  • Process — the item's steps, so you can walk the workflow in order and see where it stalled or went wrong.

  • Tests — its test assertions and their latest results, if it is a test work item.

Input and Result also surface extra values: things the agent produced that no property was declared for. A value showing up there repeatedly is a hint that your data model is missing a column.

A tool's call history

Every custom tool has a Call history tab on its detail page, under Custom-built tools. It lists that tool's past invocations, newest first, with the input each call received, the output it returned, and a status chip. Each entry expands into the full run.

This is where you go when a tool is the suspect rather than the step. Seeing twenty real calls side by side tells you quickly whether a tool is being called with the arguments you expected, whether it fails on a particular shape of input, and whether its output is stable enough for the step to rely on.

Call history is also the cheapest source of test cases you will find: any call in the list can be captured directly as a test case with Add to tests. See Automated tests for custom tools.

Error history

Open Monitor in the left nav and select the Errors tab. Monitor is available to thunk admins.

The tab has two halves.

The error-rate chart covers the last 30 days, in your local time zone. Each day shows how many agent runs succeeded and how many errored, with the daily error rate as a line, and a breakdown by error category. This is the view that answers "is this getting worse?" — a single failure is noise, a rising line is a problem.

The error list below it holds the most recent errors, newest first. Each record shows:

  • What failed and when, and which lifecycle phase the thunk was in at the time.

  • The error message.

  • A remediation note — what to do about this kind of error.

  • A link to the step it happened in, so you can go straight there. Errors from background work that has no step of its own link to the automation log instead.

  • The failed run itself, expandable inline, so you can read the tool calls that led up to it without leaving the page.

What went wrong?

Beside the remediation note on a failed run, the thunk's owner and admins see a What went wrong? button. The note tells you what kind of error this was and what generally helps. This reads the actual run and tells you what happened in this one — which is usually the difference between a generic retry and knowing which tool call broke and why. It opens a conversation: the explanation comes first, and you can then ask follow-up questions, such as which permission the connection needs. It runs on the thunk's planning model, the model that plans the workflow.

If a thunk is set to alert on errors, the email its owner and admins receive links straight here: following it opens the run that failed and its What went wrong? explanation together, so you start from the answer rather than from the error.

For how to read a failure yourself — where it started, what its shape says, and who can fix it — see Reading a failed run. For the full catalog of errors — what each one means, why it happens, and what to do next — see Troubleshooting Errors.

Looking across many runs

The views above examine one run at a time. For thunk-wide operational views — throughput, cost, and how the workflow is performing in aggregate — see Observability and Monitoring.

Did this answer your question?