GPT-6’s new cache tools help trace the cost of repeated context

An AI agent that works through a long task makes a series of API requests. Each request can carry forward the same instructions, tool definitions, reference material and conversation history. Reprocessing that shared context adds latency and input-token charges even when little has changed.

OpenAI's September 22 release gives developers more control over that repeated work. The company says GPT-6 now gets higher prompt-cache hit rates by default and keeps eligible shared prefixes reusable for at least 30 minutes after their latest write or reuse. It also launched a cache dashboard, request-level diagnostics and explicit cache breakpoints. OpenAI's announcement

The big change

• What changed: Prompt caching is becoming an observable part of an agent's runtime. A dashboard shows cached and uncached input, while diagnostics can identify the setting or content change behind a miss.
• Why it matters: Repeated context is an operating cost. Developers can now separate stable instructions from changing work and measure whether the intended prefix was actually reused.
• What to do next: Compare a recent response with the request that should reuse it, then check reported usage for the cached-token count. Diagnostics can explain the first classified miss; the usage record determines reuse and billing.

A cache hit begins with an unchanged prefix

The cache holds computed key-value tensors for an unchanged prompt prefix. Matching includes rendered instructions, tools and history. Changing an earlier part prevents reuse beyond it. GPT-5.6 and later require at least 1,024 visible input tokens; each write or reuse renews the default 30-minute eligibility window. The prompt-caching guide

A small application change can therefore make a large block of context expensive again. Switching models or service tiers can cause a miss. So can changing a tool's name, schema or order; changing the response format, verbosity or request-level reasoning effort; compacting the conversation; or editing an earlier message.

OpenAI's new diagnostics compare the current request with a recent completed response from the same organization. They return a reason such as tools_changed, along with an estimate of the reusable tokens that were missed. The current response's usage fields remain the record for actual cache reuse and billing. The diagnostics documentation

Breakpoints put a price boundary inside the prompt

GPT-5.6 and later offer automatic breakpoints at eligible message boundaries or developer-selected ones. In explicit-only mode, content after the final breakpoint incurs ordinary input charges without a cache-write charge. Place stable content before it and changing material after it.

Keep tool definitions stable while allowed_tools restricts their use. In GPT-6's standard single-agent mode, append a configuration_update to change reasoning effort between responses while keeping the request-level setting unchanged. That preserves the earlier reusable prefix. Reasoning configuration updates

The 90% discount is only one part of the bill

GPT-6 Sol’s Standard rates through 272,000 input tokens are $2 for ordinary input, $2.50 for cache writes and $0.20 for cache reads per million tokens. For a 100,000-token prefix across ten requests that each stay within that band, one full write and nine full reads would cost $0.43, compared with $2 for ten uncached processings. This is a calculation from the published rates. OpenAI's current API pricing

That calculation applies to the reused input prefix. New input after the breakpoint is still processed. Output tokens, tool charges and any additional requests remain separate. A 90% cached-input discount therefore does not mean a 90% reduction in the whole API bill. Current model rates also differ; our GPT-6 and Claude tier comparison covers those unit prices separately.

In OpenAI's announcement, GitHub Copilot reports more than halving its share of freshly processed prompt tokens across billions of requests. Wordsmith reports evaluation cache hits rising from 83% to 91%, roughly two-thirds fewer cache writes and 36% lower inference costs on the same workload.

The dashboard shows when reuse falls. Diagnostics identify the first classified reason, and explicit breakpoints mark which context is worth writing. Prompt stability, reuse frequency and the rest of the workload still determine the final cost, but the repeated-context portion can now be inspected directly.