Commerce Prompt Caching

Cache-stable request assembly

commerce_common/ is commerce-common/commerce_common/, shopping_agent/ is shopping-agent/core/shopping_agent/, and merchant_agent/ is merchant-agent/core/merchant_agent/.

The three breakpoints

A request's cacheable prefix runs tools, then system, then the messages; a breakpoint ends a span the next call reads back. The reference agents place three, all from commerce_common/prompt_assembly.py:

1. The last tool: with_tool_cache_control(tools).
2. The static system text: build_system_blocks(static, context) marks it and appends the context block unmarked.
3. The newest persisted message: build_request_messages(messages, rolling_breakpoint=...) marks the last persisted block on the outgoing request only; the next call then reads the earlier rounds, search payloads included, from cache.

Everything per request goes in the context block, which the role's build_dynamic_context(...) renders once per turn. In the static block or the tool list, those bytes would break the system or tool span on every request; in the context block they cost one re-read of the conversation on a turn whose cart, page, or facts moved, and the rolling marker holds from that turn's next round. The clock renders to the hour (context_clock), so a new minute moves nothing.

The split as implemented

• build_static_system(config, skills) in each role's prompt.py renders identity, the most-turns rules, the fence notice, and the skill index from the config and the installed skills: the same bytes for the life of a deployment.
• build_dynamic_context(...) in the same module renders the per-request material inside the role's data fence: shopping takes preferences, memory facts, the cart, the page, the account block, and the time; merchant takes the store context, memory facts, and the time. Backend context blocks have their own size cap.
• ShoppingAgent.__init__ and MerchantAgent.__init__ (each role's runtime-messages-api orchestrator.py) build _static_system and _tools once per process; stream_turn renders the context block per turn and calls build_request_messages per model call.
• build_tools in each role's tools/registry.py emits the built-ins in a fixed order (the merchant builder adds run_analysis after them when enabled), the load_skill enum sorted, then extensions in the order given, then web_search when enabled; every registered tool ships on every request, and the executor decides on arrival whether a call can be served.
• The Agent SDK runtimes pass the same static text (plus SKILL_TOOL_ADAPTER) and the same contracts, and the SDK caches them; the hosted manifests carry the text as system.md, which scripts/check.py compares with the builder.

When the rolling marker is skipped

Two rounds carry no marker. A bare first call (one message): a one-shot session would pay the write without a read, and the second call's marker covers the first message anyway. A round whose tool_choice is other than auto (the grounding-forced first iteration and the forced-text last one): tool_choice keys the messages span, so an entry written under a forced round is unreadable by the auto rounds after it; the system and tool spans still hit. Both live in build_request_messages and the loops' rolling_breakpoint= argument; rolling_conversation_cache turns the marker off for debugging.

When the conversation is compacted

When a turn's last call was given compact_history_above_tokens or more (its usage says; the default is the platform's own tool-result-clearing default, a tenth of the window), the turn ends with compact_history in commerce_common/turn.py, which replaces the oldest tool results in the stored conversation with a one-line marker until the conversation is half its previous size. The next turn's first round rewrites the messages span once and later rounds read the shorter one; turn_complete.results_cleared tells a host that appends its transcript to rewrite it. The system and tool spans, the messages, and the write gates are unaffected; provenance lives on the session state.