LiteLLM Merges Fixes for Bedrock Structured Output, Gemini Continuation, Prompt Caching, and Web Search Routing

LiteLLM (YC W23) 1 hour ago

Key Info

LiteLLM merged several gateway improvements covering structured output, long-response continuation, prompt caching, and web search routing.

Bedrock Opus 5 and Opus 4.8 now use native JSON schema for structured output (#50222)
Gemini Interactions can resume long responses with continuation_token (#50151)
Prompt caching keeps a conversation on one deployment even with message logging off (#50276)
Web search requests skip deployments marked as not supporting web search (#50275)

Compiled from 1 post

#LiteLLM#Bedrock#Gemini