Key Info

Requesty published a 30-day median latency comparison showing its gateway adds 2.3 ms versus 241 ms for a direct model call, with gateway overhead under 1% of the request.

Highlights

  • Median over the last 30 days: direct model call 241 ms, Requesty 2.3 ms
  • The 2.3 ms covers key auth, spend limits, approved-model and compliance checks, and rate limits
  • Gateway overhead is stated to be under 1% of the request
  • Example model referenced: typesafe/jev-latest