Key Info

LiteLLM Lens now integrates Typesafe's JEV to automatically evaluate every agent run and tag the ones that failed directly in logs. JEV completes each judgment in ~130 ms, fast enough to keep up with workloads of 200K+ traces.

Highlights

  • JEV judges each agent run automatically and tags failures in the logs
  • Judgments take about 130 ms per run
  • Designed to handle 200K+ traces without slowing down
  • Helps developers find failing runs without reading traces one by one
  • Targets production agent builders who need run-level observability