Prompt caching
Storing the unchanging part of a conversation so the model does not pay full price to re-read it every turn.
Models have no memory between steps. Every time you send a message, the whole conversation so far gets fed back in, because that is the only way the model knows what you were talking about. In a long session that means re-reading the same text over and over. Prompt caching stores that repeated chunk on the provider’s side so it can be loaded again cheaply instead of processed from scratch. Cached tokens usually cost a fraction of fresh ones.
This matters enormously for agents, which work in many small steps and re-read their accumulated context at each one. In one measured example, 96 percent of all tokens an agent processed over eight weeks were cache reads, and the text the user actually saw was under half a percent. So when you see a huge token count attached to an agent, most of it is the system reminding itself what it was doing, not new thinking.