Working with large APIs

Real business APIs return thousands of rows across dozens of pages. A model paging by hand is slow, expensive and silently incomplete: it fetches page one, sees twenty results, and reports “I found twenty”. This page covers the platform’s answer — the single most under-used capability in the product, and the fix for most “the agent only found six of them” incidents.

Two layers

Every tool an agent can call — API, MCP, connected app — gets a size guard unconditionally. A response that would not fit the conversation is cut at an item boundary, the full payload is kept aside for a few minutes, and the agent receives a note explaining what it is looking at and how to get the rest. On top of that, looping is opt-in per tool. Set loop_responses on the tool’s entry in an agent’s tool list and the tool gains a small set of reserved arguments that let the agent walk every page, fan out over a list of ids, and project only the fields it needs — inside one tool call.

{
  "agentTools": [
    { "tool_id": "TOOL_ID", "provider_id": "PROVIDER_ID", "loop_responses": true }
  ]
}

In the Studio the same switch is Allow response looping for this tool, on the tool card and in the agent’s tool list. Turning it on for any tool also attaches two retrieval tools, queryToolResult and fetchToolResult, that read a stored result without calling upstream again.

The flag lives on the agent’s binding, not on the tool. Two agents can hold the same tool with looping on for one and off for the other.

The reserved arguments

Looping adds these arguments to the tool’s input schema. They are stripped before the real request is built, so none of them ever reaches your API.

Argument	What it does
`_fields`	Project every item down to the listed dot-paths. Typically a 10–50× reduction in what enters the conversation.
`_all_pages`	Walk every page inside one step and return the merged list with a completeness verdict.
`_for_each`	Call once per value of a parameter and merge the rows, each tagged with the value it came from.
`_filter`	Filter the merged rows server-side before they reach the model.
`_refresh`	Bypass a cached result and fetch again.

`_fields`

{ "status": "open", "_fields": ["id", "customer.name", "total"] }

Paths map over arrays along the way, and the projection is always applied when set — not only when the result would overflow. The full, unprojected payload is still kept aside under a _resultId, so the agent can come back for a field it left out. Wrong paths don’t fail quietly. A path that resolves to nothing comes back as null, and a path that matched nothing on any item is reported in _unmatchedFields together with the items’ real top-level keys and a sample item. The agent corrects itself in one step instead of publishing twelve empty rows.

`_all_pages`

Offered only when the tool’s own schema declares a page-size parameter and a page-number or offset parameter. Recognised spellings:

Role	Parameter names
Page size	`itemsPerPage`, `items_per_page`, `perPage`, `per_page`, `pageSize`, `page_size`, `maxResults`, `max_results`, `limit`, `$top`
Page number	`page`, `pageNumber`, `page_number`, `pageIndex`, `page_index`
Offset	`offset`, `skip`, `$skip`, `startIndex`, `start_index`

The parameters may sit at the top level or one level down in a request body. There is nothing to configure: detection reads the schema you imported.

Cursor-based pagination (nextPageToken, Link headers) is not walked. A cursor-style endpoint still gets the size guard and _fields, but not _all_pages.

Pages are fetched sequentially so upstream rate limits are respected. The walk stops at the first short page, and on a bounded budget:

Bound	Default
Pages per collection	20
Requests per tool call	60
Items	5,000
Raw upstream bytes	24 MB
Wall-clock	60 s
Values per `_for_each`	50

A walk that hits a bound says so. The result carries _pagingComplete: false, _likelyMoreRows: true and a note naming the reason, so the agent reports a partial set as partial rather than passing it off as complete. Totals, counts and averages are only presented as sound when every page was fetched.

`_for_each`

{ "_for_each": { "key": "property_id", "values": ["p_1", "p_2", "p_3"] } }

One call per value, merged into a single list. Each row is tagged with the value it came from (an existing field of the same name is never overwritten — the API’s own answer wins). Values are de-duplicated and coerced to the parameter’s declared type. _for_each and _all_pages combine — every page of every value — under one shared budget. The key must be a parameter of this tool. Naming one that isn’t returns an invalid_for_each_key error that lists the tool’s real parameters. Per-value failures are reported in _forEachErrors alongside the rows that succeeded; only if every value fails does the call fail.

`_filter`

{ "_filter": [{ "field": "status", "op": "eq", "value": "open" }] }

Operators: eq, neq, gt, gte, lt, lte, contains, in, exists. Clauses are ANDed. _filter applies to the merged result of the current call — after the walk, after the fan-out — so the model never sees the rows it did not ask for. It is honoured but not advertised to the model on the source tool, because next to your API’s real filter parameters a model will reach for the reserved one and fetch everything to filter it afterwards. Use your API’s own filters first; _filter is for what they cannot express. To filter rows that were already fetched, with no upstream call at all, the agent uses queryToolResult on the stored _resultId — see below.

Stored results

When a response is cut, or a walk completes, the full payload is kept for about 15 minutes, scoped to the organization and the thread. The agent receives _resultId and two tools:

queryToolResult — fields, an optional filter, offset and limit over the stored items. Filtering runs on the full set before paging, so a clause may name a field that was not projected.
fetchToolResult — raw character slices for results that have no usable item boundary.

If a step ends with a truncated result and the agent has not yet queried it, the next step is restricted to the retrieval tools. That is what stops an agent from answering off a preview.

Caching

A complete walk can be cached so that the next identical call — same tool, same arguments, same fan-out values — is served without touching upstream. Cache entries are scoped by whose credentials fetched them: organization-level provider credentials share across the org, per-user runtime credentials are private to that user (or to a tenant-wide cache_scope your backend asserts at mint time). Defaults are 24 hours for org credentials and 1 hour for runtime credentials, with a per-binding override, dataset_ttl_hours. A cache hit is labelled: _cache: { hit: true, cachedAt, expiresAt } plus a note telling the agent that the user can get the very latest data by re-calling with _refresh: true. Re-calling with different _fields or _filter is still a hit — the stored set is unprojected and unfiltered.

Fixed arguments hit the cache every run. A workflow’s arguments are baked into its definition, so its cache key is identical on every scheduled run — exactly the runs that most need fresh data, with no human present to pass _refresh. In workflows the cache is therefore off unless the step opts in with a TTL, and it is worth opting in only where re-fetching is expensive and staleness up to the TTL is genuinely acceptable.

Caching is enabled per deployment; in shadow mode it records walks without serving them. Experiment runs never read from the cache.

Live progress

During a long walk the stream carries data-tool-loop-progress chunks, at most one every 750 ms, with the page and request counters (page, pagesMax, items, requests, requestsMax, elapsedMs) and, for a fan-out, the value currently being fetched. The same chunks flow through streamed workflow runs. Use them to render a progress line under the tool call; nothing in them is needed for correctness.

Tool discovery

Connected-app toolkits can be large — Gmail alone is more than sixty tools — and a model chooses worse from a long list. Above a budget of 15 directly injected tools, connected-app tools move behind two meta-tools:

search_tools takes a query and returns the best matches with short descriptions.
load_tool takes a toolName (or toolNames) and makes those tools callable for the rest of the conversation.

Toolkits are kept direct whole, smallest first, while they fit the budget; the first toolkit that would not fit moves entirely to the pool. That is how a three-tool WhatsApp toolkit stays directly callable next to a sixty-tool Gmail one. Override per entry with discovery: true (always pooled) or discovery: false (always direct, and counted against the budget). list_connections and connect_toolkit are always direct. Your own API and MCP tools are never pooled. Prune them at the agent instead — see Connecting your API.

In workflows

A tool step takes the same capabilities as a loop block, declared once rather than chosen per call:

{
  "id": "fetch",
  "type": "tool",
  "tool": "list_orders",
  "input": { "status": "open" },
  "loop": {
    "all_pages": true,
    "fields": ["id", "total"],
    "for_each": { "key": "property_id", "values": { "ref": "$.steps.properties.ids" } },
    "filter": [{ "field": "total", "op": "gt", "value": 100 }],
    "cache": { "ttl_minutes": 60 }
  }
}

The step returns the tool’s own payload shape with the verdict under a single _loop key — complete, requests, itemCount, stoppedBy, note — so a branch can gate on $.steps.fetch._loop.complete. There is no _resultId in workflows: nothing downstream of a step could redeem one, so the walk’s byte budget is the only limit. See Building a workflow.

Guidance the agent already has

An agent holding looping tools is given the rules above as a separate system message: walk before you count, project before you read, report a capped walk as partial, and never name the mechanism to the user. You do not need to repeat any of it in your own prompt. What is worth writing is which collections matter: “orders are paginated; always fetch every page before summarising” is a better instruction than an explanation of _all_pages.

Getting started

Build

Connect

Control

Automate

Operate

Working with large APIs

Two layers

The reserved arguments

`_fields`

`_all_pages`

`_for_each`

`_filter`

Stored results

Caching

Live progress

Tool discovery

In workflows

Guidance the agent already has

​Two layers

​The reserved arguments

​_fields

​_all_pages

​_for_each

​_filter

​Stored results

​Caching

​Live progress

​Tool discovery

​In workflows

​Guidance the agent already has

Two layers

The reserved arguments

`_fields`

`_all_pages`

`_for_each`

`_filter`

Stored results

Caching

Live progress

Tool discovery

In workflows

Guidance the agent already has