Monitor AI model traffic at the gateway (private instances)
This article applies to private instances of Thunk.AI. On the public service at app.thunk.ai, the gateway is managed by Thunk.AI.
Every AI model call and every MCP tool call a thunk makes goes through the connection gateway — the routing layer between Thunk.AI and your model providers and MCP servers. Because all traffic passes through it, the gateway is the one place that sees everything your instance sends to AI providers, across every thunk, every environment, and every model. On a private instance, your administrators can watch that traffic in the gateway's admin console.
For what the gateway does and how environments are configured in it, see Configuring Environment Settings.
How traffic is organized
Each resource environment has its own access key in the gateway, and a key can only use the models and MCP servers assigned to its environment. Every call is recorded against the key that made it. That means the gateway can tell you:
how much traffic, and how much spend, each environment is generating;
which models are being called, and how often;
which calls failed, and what the provider said.
If each project or business unit has its own environment, the gateway's view per key is effectively a view per project.
Getting access
The gateway admin console is part of your private instance. If your administrators do not already have a sign-in, ask Thunk.AI support to set one up.
What you can see
Request logs. One entry per model call: when it happened, which key and model it used, the tokens it sent and received, what it cost, how long it took, and whether it succeeded. Use them to follow a specific failure back to the provider's error.
Usage and spend. Totals over time, broken down by key and by model. Use them to see which environments drive spend and whether that is changing.
Keys. Each environment's key, with its spend and any limits set on it.
Models and MCP servers. What is registered in the gateway, so you can confirm a model an environment needs is actually there.
Common questions it answers
Question | Where to look |
Which project is driving our AI spend? | Usage and spend, broken down by key (environment) |
Is a provider failing or rate-limiting us? | Request logs, filtered to failed calls |
Is the new model actually receiving traffic after a switch? | Usage by model, or request logs for that model |
Why did one work item's step fail with a model error? | Request logs around the time of the failure |
How this relates to Token Costs
Token Costs in a thunk's Monitor pane shows what one thunk spends, per work item and per step. The gateway shows what the whole instance spends, per environment and per model, including work that isn't tied to a single thunk. Use Token Costs to tune a thunk; use the gateway to understand and govern spend across the organization. See Token Costs.
Make changes in Thunk.AI, not in the console
Use the gateway console to monitor. Add and change models through Org Admin → Model Definitions in Thunk.AI, and environments through Org Admin → Environments — Thunk.AI keeps the gateway in step with them. See Orgs, Roles, and Environments. Editing keys, models, or MCP servers directly in the console can leave the gateway out of step with what Thunk.AI expects and break the thunks in that environment. For anything the Org Admin pages don't cover, contact Thunk.AI support.