Key Info
Engineers from Together and Ollama published notes on what the harness and the inference engine each need to get right so prefix caching keeps long coding-agent sessions fast in tools like Claude Code, Codex, and Pi.
Highlights
- Coding agents resend the entire conversation on every turn; prefix caching is what keeps long sessions responsive.
- A single timestamp in the system prompt can break the cache.
- Changing reasoning effort can wipe the whole cache.
- Telling the agent its output was cut off prompts it to read what it missed.
- Both the harness and the inference engine must cooperate for caching to work.