Overview
Thunk.AI uses a Connection Gateway and Routing layer to manage access to AI models and tools. This layer sits between the Thunk.AI execution runtime and your chosen LLM providers and MCP servers. It handles model routing, provider authentication, MCP server configuration, rate limiting, and tool access.
The gateway technology currently used to implement this layer is LiteLLM.
As an enterprise IT administrator, you configure one or more environments within the connection gateway. Each environment is an isolated configuration that controls which AI models are available, which tools (via MCP servers) agents can access, and what operational limits apply. Thunk.AI users can then target a specific environment from the dropdown in the Thunk.AI UI when building or running a thunk.
Why Use Environments?
Environments let you mirror your organization’s software development lifecycle inside Thunk.AI. A thunk starts in a prototype stage and can be promoted to testing, then production. Each stage is associated with an environment enforcing the appropriate models, tools, and controls for that stage without any changes to the Thunk.AI logic itself.
| Prototype | Testing | Production |
Purpose | Build and explore thunks quickly usually against filesystem files | Validate against real-ish services before go-live | Live workloads using real production services |
Models | Range of full-capability models (e.g. GPT4.1-mini, GPT-5, Gemini Pro 3, etc.) set for exploring the capability needs. | Cheaper/faster models (e.g. GPT-41-mini) matching prod targets. | Cheaper/faster models (e.g. GPT-4o-mini) made to help keep costs low. |
MCP Tool Servers | Mock APIs, local filesystem, dev database | Sandbox versions of production services | Live services, CRM, internal services |
How It Works
Each environment is a separate configuration within the connection gateway routing layer. Internally this is implemented using LiteLLM namespaces from a list of LLM models and MCP servers.
When a Thunk.AI workflow runs, all model and tool calls route through the gateway for the selected environment. The gateway handles:
Translating requests to the correct provider API format (OpenAI, Anthropic, Google, AWS Bedrock, Azure, etc.)
Enforcing rate limits and spend caps defined for that environment
Routing requests to the configured model provider
Falling back to alternative models if the primary model is unavailable
Controlling which MCP tools are visible to the thunk
Model Setup Per Environment
Each environment exposes a named set of models. These names are what Thunk.AI users see when selecting a model within a thunk. The model names are aliases and the underlying provider routing is abstracted away from the user.
Model alias (user-visible) | Routes to (Dev) | Routes to (Prod) |
gpt-4.1-mini | openai/gpt-4.1-mini | Azure/gpt-4.1-mini |
claude-sonnet | anthropic/claude-sonnet-4-5 | Azure/claude-sonnet-4-5 |
default | openai/gpt-4.1 | Azure/gpt-4.1 |
This means a Thunk.AI built in Development that uses claude-sonnet will automatically use the Anthropic version of Sonnet. When the same Thunk.AI is promoted to Production, it uses the Azure-hosted version with no individual setup required by the Thunk.AI builder.
Assigning MCP Servers Per Environment
MCP (Model Context Protocol) servers expose tools that AI agents can call during thunk execution. The environment configuration defines which MCP servers are made available to the thunk. Environments typically share the same names across stages but connect to different server endpoints.
This allows a thunk that calls a search tool in Development to hit a different endpoint than the same thunk running in Production, without any changes to the thunk logic.
When planning MCP servers per environment, it helps to think in terms of three tool categories.
Read-only tools
Tools that fetch or retrieve information without modifying data. Examples include searching a knowledge base, reading documents, querying records, or fetching web content.
Read-only tools still require controls. AI agents can generate high request volumes if they enter tight loops. Apply rate limits in staging and production environments.
Write-based tools
Tools that create, update, or delete data. Examples include inserting records, updating a CRM contact, writing files, or modifying calendar entries.
In Development and Testing, write tools must point to isolated sandbox targets. They should never connect to production data stores.
In Production, apply rate limits and restrict credentials to the minimum required permissions.
External Actions
Tools that trigger real-world consequences outside the system. Examples include sending email, posting to Slack, triggering a webhook, initiating a payment, or calling a third-party API with side effects.
External actions carry the highest operational risk. In Development, these tools should point to mock MCP servers rather than live endpoints. High-risk actions should use human-in-the-loop (HITL) approval before execution.
Recommended controls by category and environment
Tool category | Prototype | Test | Production |
Read-only | Unrestricted. Point at dev/test data sources. | Rate-limited. Point at staging data sources. | Rate-limited. Scope credentials to read permissions only. Monitor query volume. |
Write-based | Sandboxed targets only. Never write to shared or production data. | Sandboxed targets. Rate-limited. Mirror production credential scope. | Live targets. Rate-limited. Minimum-permission credentials. Audit logging enabled. |
External Actions | Mock MCP servers strongly suggested. No connections to live action endpoints. | Live endpoints permitted on an explicit allowlist. HITL approval recommended. | Live endpoints. HITL approval required for high-risk or irreversible actions. |
Submitting an Environment Configuration
To keep environment setup simple, organizations typically submit an Environment Configuration Form to Thunk support. The information in this form is used to configure the environment in both Thunk.AI and the connection gateway.
Typical deployment pattern is to create three environments:
Prototype: development and experimentation
Test: validation against staging services
Production: live workloads
The models and MCP server endpoints can differ per environment while the thunk logic remains unchanged.
Environment Configuration Form
The form collects the information needed to configure models, MCP servers, and environment access.
Submit the following form to Thunk support to create a new environment. Fill in the Value column. Example values are shown in italics.
Project Details
Variable | Value |
Project Name | Unique name for your project (Project_Atlas) |
Description or Intended Use | Description of the intended use (For the Atlas team that is developing workflows for the finance business unit.) |
Allowed Users | Thunk.AI developers who will be using the environments ([email protected], [email protected]) |
LLM Models (repeat for each model required for the Project and Environment)
Variable | Value |
Project Name | Refer to the project from Project Details (Project_Atlas) |
Stages | The set of stages this should be available in (Prototype, Test, Production) |
Provider Model Alias | gpt-4.1 (user-visible name) |
Provider | OpenAI (OpenAI, Anthropic, Azure, etc.) |
Provider URL | URL to the provider’s endpoint (https://api.openai.com/v1) |
Authentication Method | API Key (API key, Azure auth, etc. Indicate if it is different for different environment.) |
MCP Servers (repeat for each set of MCP tools)
Variable | Value |
Project Name | Refer to the project from Project Details (Project_Atlas) |
Stages | The set of stages should be available in (Prototype, Test, Production). |
MCP Server Display Name | How you want the server to appear to Thunk.AI developers (company-doc-search) |
Description of Tools Provided | Short description string that will display to Thunk.AI developers (Internal knowledge base search) |
Optional Tool Filtering | The set of tools to expose (none or a given list) |
MCP Server URL for each environment | Prototype: MCP url (https://mcp-prototype.company.internal/actions) Test: MCP url (https://mcp-test.company.internal/actions) Production: MCP url (https://mcp.company.internal/actions) |
Transport Type | Streamable HTTP (usually Streamable HTTP) |
Authentication Type | Bearer Token (None, API Key, Bearer Token, OAuth2, etc. Indicate if it is different for different environment.) |
Once submitted, Thunk support will contact you to configure the environment secrets and confirm when it is available for use in the Thunk.AI UI.