Prompt Caching in LLMs!

Prompt caching is a critical cost-optimization technique for LLM-based agents. Every agent turn resends the full conversation history, causing redundant computation. By separating static context (system prompt, tool definitions) from dynamic context (conversation history), the KV cache stores computed Key/Value tensors and serves them at a 90% discount on subsequent reads. Using Claude Code as a case study, a 30-minute coding session achieves a 92% cache hit rate and 81% cost reduction. Key rules: never modify tool definitions mid-session, never switch models, never mutate the static prefix. Prompt structure should place stable content at the top and dynamic content at the bottom. Cache efficiency can be monitored via API response fields.