Key Info
LiteLLM Lens now supports datasets, creating a workflow where the tool identifies failing traces, users add them to a dataset, and then run evaluations on their agents.
Highlights
- Lens automatically finds failing traces in agent runs
- Failing traces can be added to a dataset for reuse
- Datasets enable running evaluations directly against agents
- Completes a loop from detection to evaluation