Key Info

Engineers from Together and Ollama published notes on what the harness and the inference engine each need to get right so prefix caching keeps long coding-agent sessions fast in tools like Claude Code, Codex, and Pi.

Highlights

  • Coding agents resend the entire conversation on every turn; prefix caching is what keeps long sessions responsive.
  • A single timestamp in the system prompt can break the cache.
  • Changing reasoning effort can wipe the whole cache.
  • Telling the agent its output was cut off prompts it to read what it missed.
  • Both the harness and the inference engine must cooperate for caching to work.