Mistral Large 4 Technical Report Details KV-Cache Trade-offs and Sparse Attention

Key Info

The technical report for Mistral Large 4, published exclusively on alphaXiv, shows that aggressively reducing KV-cache memory can degrade an LLM's agentic capabilities even when its language modeling performance looks unchanged.

Highlights

• Mistral initially cut KV-cache memory by 16x, but found this hurt long-context agentic tasks that require retrieving information from much earlier context.
• To fix it, Mistral restored full-context attention to more layers, then adopted DeepSeek Sparse Attention to select the 2,048 most relevant tokens across the context instead of attending to everything.
• The approach feeds into their resulting 1T-parameter MoE model with 52B active parameters (original post truncated at this point).