Key Info
LiteLLM announced Lens Profiles, a feature that turns production traffic into task-level model evaluations by automatically discovering what customers use agents for and attributing gateway-recorded spend to those tasks.
Highlights
- Automatically discovers agent tasks without manually labeling every run
- Attributes gateway-recorded spend to specific tasks to surface cost drivers
- Compares model deployments by request volume, average cost, and p95 latency per task
- Builds evals from task categories and real production runs against actual customer workloads
- Implementation PRs are referenced for gateway cost attribution