Skip to content

Agent.as_tool() nested subagent runs never hit prompt cache, even with byte-identical instructions/tools/prompt_cache_key #5085

Description

@prajwal-bik-ai

Setup: A top-level Agent with several subagents attached via sub_agent.as_tool(tool_name=..., tool_description=..., session=read_only_session_view), where read_only_session_view is a custom SessionABC implementation wrapping the parent's own session (read-only — add_items/pop_item/clear_session are no-ops). Model: gpt-5.6-luna. We explicitly set a stable session_id on the wrapper so the SDK's own cache-key grouping (resolve_run_grouping) resolves to ("session", <stable-hash>) instead of falling back to a random per-run key — confirmed this works (see below).

Observed: Across 42 real subagent calls that clear the 1,024-token minimum, 0 ever returned any cached_tokens — a flat 0% hit rate, including repeat calls to the same subagent, seconds apart, with verified byte-identical prefixes.

Reproduction — two consecutive calls to the same subagent, ~12 seconds apart:

Captured the literal kwargs passed to AsyncResponses.create() (patched at that layer to rule out any SDK-side transformation):

Call 1: instructions="Collect \n1. User first name\n2. User last name\n3. User email\n\n...
         ## Known conversation context\n\nToday's date: Fri, 18 Sep 2026 (UTC)."
        prompt_cache_key="agents-sdk:session:<same-hash-both-calls>"
        input: [54 items]
        usage: input_tokens=695, cached_tokens=0

Call 2: instructions=<byte-identical to Call 1 — diff returns nothing>
        prompt_cache_key="agents-sdk:session:<same-hash-both-calls>"
        input: [58 items — first 53 byte-equal to Call 1's, remainder is
                the new turn appended, i.e. a normal growing conversation]
        usage: input_tokens=838, cached_tokens=0

Also confirmed identical between the two calls: tools (empty in this pair), model, and every other kwarg we captured (store, previous_response_id, conversation, truncation, parallel_tool_calls, text, reasoning, extra_body, extra_headers — all consistently omitted/default on both).

What we ruled out before filing:

  • Unstable prompt_cache_key (confirmed identical both calls, via a raw capture at the responses.create() call site)
  • Tool definition drift
  • Instructions drift (confirmed byte-identical via diff, not just visual inspection)
  • Input prefix drift (53/54 leading items are dict-equal)
  • Cache TTL expiry (still 0% at gaps as short as 8.5s)
  • Being under the 1,024-token minimum (42/42 tested calls are above it)

For comparison: the parent agent in the same run, calling the same model via a normal (non-nested) Runner.run() with its own stable session, gets 78-90% cache hit rates over the same kind of growing conversation. The only structural difference we can identify is that these are nested calls originating from Agent.as_tool(), using a custom read-only Session implementation rather than the SDK's own session type.

Question: is there a known limitation where nested/as_tool()-originated Runner.run() calls are excluded from prompt caching regardless of session/prompt_cache_key, or is there something about a custom SessionABC subclass that would prevent this (e.g. is session_id alone insufficient, and something else internal is expected)?

Related but distinct: #2784 (base64 file inputs breaking cache) — different root cause, but confirms prompt_cache_key having "zero effect" on unexpected 0% hit rates is a recurring pattern in this SDK.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions