# Prompt Caching 201
> **Note:** This guide applies only to models before GPT-5.6. For GPT-5.6 and later, see the [Prompt Caching guide](https://developers.openai.com/api/docs/guides/prompt-caching).
A practical guide to prompt caching: fundamentals, performance impact, measurement, and optimization strategies.
## 1. Prompt Caching Basics
Model prompts often include repeated content - such as system instructions, tools, and messages. When a request contains a prefix the system has recently processed, OpenAI can route it to a server that already computed that prefix, allowing the model to reuse prior work instead of recomputing it from scratch.
Prompt Caching can reduce time-to-first-token latency by up to 80% and input token costs by up to 90%. It works automatically on all API requests and has no additional fees.
The goal of this cookbook is to go deeper on optimizing for cache hits. Review our [API docs](https://developers.openai.com/api/docs/guides/prompt-caching) and [Prompt Caching 101](https://developers.openai.com/cookbook/examples/prompt_caching101/) for a great overview - let’s dive in!
### 1.1 Basics
- Cache hits require an exact, repeated prefix match and works automatically for prompts containing 1024 tokens or more, with cache hits occurring in increments of 128 tokens.
- The entire request prefix is cacheable: messages, images, audio, tool definitions, and structured output schemas.
- In-memory prompt caching works automatically on all your API requests. Extended prompt caching increases that to 24hrs.
- Caching only works if two requests share the same prefix and land on the same machine. Take advantage of the optional parameter `prompt_cache_key` for traffic that shares common prefixes to improve that routing.
- Carefully consider the impact of caching from context engineering techniques like compaction.
- Monitor caching, cost and latency via request logs or the Usage dashboard while iterating.
## 2. Why caching matters
### 2.1 Core Technical Reason: Skipping Prefill Compute
The forward pass through transformer layers over the input tokens is the main driver of inference cost and latency. In the transformer stack, prompt caching applies specifically to the key and value projections inside the attention layers. Attention is the mechanism that allows models to weigh the importance of different parts of an input sequence when processing a specific token, enabling them to capture context and long-range dependencies.
When you make a request to OpenAI, we'll create token embeddings that are then transformed into three vectors: a query (Q) that represents what that token would need to self-contextualize, a key (K) that encodes what each token represents, and a value (V) that contains the information that can be incorporated if it were to be relevant. The model compares the current token’s query to all tokens’ keys (via dot products) to produce attention scores. After a softmax, these scores become weights that determine how much each token’s value contributes to the updated representation. For example, in the phrase “she sat by the river bank,” the word “river” provides important context to “bank,” pushing its meaning toward the aquatic rather than the financial. The KV cache stores the key and value tensors for that prefix across all layers and heads.
If you then make another request — “she sat by the river bank on a cloudy day” — the first part of the prefix is identical. When processing the new, unseen suffix, the model reuses the cached tensors and only computes attention for the new tokens.
Crucially, the strong semantic relationship between “river” and “bank” has already been encoded into those cached key and value representations during the earlier forward pass. The model does not need to recompute how “bank” aligns with “river.” Instead, new tokens like “cloudy” and “day” generate their own queries and attend over the cached keys from the prefix. In other words, the semantic groundwork has already been laid; the model simply builds on top of that existing attention state rather than recalculating it from scratch.
### 2.2 Cost Impact
Cache discounts can be significant. Discount magnitude varies by model family - as our inference stack has become more efficient, our newest models have been able to offer steeper cache discounts. Here are some examples, but see our pricing page for all models. Prompt Caching is enabled for all recent models, gpt-4o and newer.
| Model | Input <br>(per 1M tokens) | Cached input <br>(per 1M tokens) | Caching Discount |
| --- | --- | --- | --- |
| gpt-4o | $2.50 | $1.25 | 50.00% |
| gpt-4.1 | $2.00 | $0.50 | 75.00% |
| gpt-5-nano | $0.05 | $0.005 | 90.00% |
| gpt-5.2 | $1.75 | $0.175 | 90.00% |
| gpt-realtime (audio) | $32.00 | $0.40 | 98.75% |
### 2.3 Latency Impact
Reducing time-to-first-token (TTFT) is a primary motivation for improving cache rates. Cached tokens can reduce latency by up to ~80%.
Caching keeps latency roughly proportional to generated output length rather than total conversation length because the full historical context is not re-prefilled. When we get cache hits, sampling the model is linear rather than quadratic.
I ran a series of prompts 2300 times and plotted the cached vs uncached requests. For the shortest prompts (1024 tokens), cached requests are 7% faster, but at the longer end (150k+ tokens) we’re seeing 67% faster TTFT. The longer the input, the bigger the impact of caching on that first-token latency.
## 3. Measure caching first (so you can iterate)
### 3.1 Per-request: `cached_tokens`
Responses include usage fields indicating how many prompt tokens were served from the cache. All requests will display a `cached_tokens` field of the `usage.prompt_tokens_details` [Response](https://developers.openai.com/api/docs/api-reference/responses/object) or [Chat](https://developers.openai.com/api/docs/api-reference/chat/object) object indicating how many of the prompt tokens were cached.