LiteLLM Applies Per-Model TPM/RPM Limits to Fallback Models

Key Info

LiteLLM now applies per-model TPM and RPM limits to fallback models, with token attribution following the model that actually serves the request.

Highlights

• Scenario 1: if Opus fails and the key is under the fallback model's limit, LiteLLM routes to the fallback, attributes tokens to it, and the fallback serves the request.
• Scenario 2: if Opus fails and the key is at the fallback model's limit, LiteLLM does not route to the fallback and the caller receives the original Opus error.
• The change is tracked in PRs #51082 and #50994.
• Per-model limits now govern fallback behavior rather than only the primary model.