Prompt Caching Becomes the Control Plane for Persistent AI Agents

GPT-6’s Caching Reset Reframes Persistent Agent Economics

OpenAI has shipped a revamped prompt caching system for GPT-6 that raises default cache hit rates and applies discounts to eligible shared prefixes reused within a 30-minute window.

The move turns caching from an invisible inference optimization into a first-class observability layer for persistent agents, which can run for hours on complex tasks and chain large numbers of API calls.

For teams operating those workloads, the immediate promise is faster responses and input token discounts of up to 90% on cached context.

Inside the Caching Toolchain: Dashboard, Diagnostics, Breakpoints

GPT-6 is designed around agents that handle long-running codebase refactors, research-heavy documents, and production-grade presentations.

Each request from those agents often carries the same instructions, tool definitions, and context from prior turns.

The updated caching layer reuses shared computation across requests, reducing response time and cutting the cost of repeat input tokens when a prefix remains eligible.

The new Prompt Caching Dashboard shows how much of an application’s input is served from cache and tracks hit rates over time.

An input composition chart separates cached and uncached tokens, giving developers a direct view of how application changes affect reuse.

For unexpected misses, a diagnostics tool compares a request with a recent response to identify what changed: the model, tools, settings, or input itself.

The sample diagnostic output returns a reason such as tools_changed and estimates both reusable and missed tokens in the same object.

Mario Rodriguez, Chief Product Officer at GitHub, frames the operational stakes in production systems.

OpenAI’s prompt caching plays a critical role in helping GitHub Copilot deliver fast, efficient experiences at scale. Over the past several months, we’ve reduced by more than 50% the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models, relative to our previous baseline. The result is a more efficient inference stack and faster time to first response for developers.

Explicit Breakpoints and Reasoning Effort

Developers can now choose which prompt prefixes to reuse through explicit cache breakpoints.

This shifts the control model away from fully automatic prefix selection, allowing stable shared context to be pinned while frequently changing content stays outside the cache boundary.

GPT-6 also supports changing reasoning effort between responses without breaking cache.

By appending a configuration_update while leaving request-level reasoning effort unchanged, a developer can raise or lower reasoning for a task without invalidating reusable context.

The refreshed prompt caching guide explains how long cached prefixes remain eligible and how tool or input changes affect reuse.

Keeping Tools and Context Stable

Cache reuse is sensitive to structural shifts in tool definitions, schemas, and ordering.

To preserve earlier context, developers can keep those definitions stable and use allowed_tools to expose only the relevant subset.

Setting tool_choice to none avoids removing definitions entirely and keeps the cache intact when no tools are needed.

New developer messages allow fresh instructions to be appended near the end of the context, overriding older instructions without invalidating the cached prefix.

Prewarming prepares known context in advance so the model can start responding sooner when a request arrives.

Shared instructions, tool definitions, and reference material can be loaded during startup, moving preprocessing out of the user’s waiting time.

Cache Control Turns Into a Cloud Infrastructure Contest

AWS Machine Learning Blog documents a comparable savings curve for Amazon Bedrock prompt caching, with cached reads cutting input token costs by up to 90% and cache writes priced at a premium.

That structure makes cache placement a tactical decision: a stable 10,000-token prefix can be cheap to read repeatedly but costly to write if the strategy is wrong.

The platform’s previous automatic caching did not charge for cache writes and exposed cached token counts in usage details, but it did not give developers direct control over which prefix to cache.

Explicit GPT-6 breakpoints narrow that gap, bringing the platform closer to the cache point and cache control mechanisms already used across other foundation model services.

Why Cache Hits Still Break

Several common application choices silently reduce cache hit rates.

• Timestamps inside system prompts
• Request IDs that change per call
• Per-user system prompts that fragment shared context
• Randomized tool order that breaks exact byte matching
• A/B prompt edits that shift prefixes between turns

The new diagnostics are built for precisely that problem: they reveal which variable changed and estimate how many tokens were lost from the cache.

Strawberry Browser said the diagnostics and dashboard helped raise cache hit rates by a few percentage points and cut costs by 20%.

Eugene Mikhantyev, AI Engineer at Strawberry Browser, noted that explicit breakpoints reduced cache writes by roughly two-thirds and inference costs by 36% for the same workload.

For Manus, the improvement path was concrete: its team worked with the GPT-6 engineering group to refine breakpoints and use production requests for miss diagnosis.

For long-running agents like Manus, reliable caching is fundamental to the economics. Working with OpenAI’s engineering team, we refined cache breakpoint placement, combined explicit and automatic caching, and used real requests to pinpoint unexpected cache misses. In less than a week, our OpenAI model cache hit rate went from roughly 85% to consistently above 90%, further lowering inference costs in production. The progress came through a series of targeted improvements, with both teams validating the results along the way.

Cross-provider failover adds another risk.

Cache entries do not follow traffic when a workload shifts between providers, so a cached read on one platform can become a full-price input on another.

That turns caching from a pure cost lever into an availability and resilience consideration for multi-provider agent stacks.

Cache Strategy Becomes an Infrastructure Discipline

Persistent agents only scale when inference cost and latency stay predictable across thousands of long-running turns; GPT-6’s update makes cache behavior inspectable, tunable, and production-grade rather than an opaque model property. For AI teams building cost-controlled agent orchestration systems, programmatic SEO and AI automation is how Andres SEO Expert approaches the same operational discipline — tell us about your stack.