Key Info

GMI Cloud ran EAGLE-4 speculative decoding on NVIDIA B300 and benchmarked it against other speculators for Qwen3.8-27B, publishing throughput results.

Highlights

  • 703 tok/s at batch
  • 2.7x faster than plain decoding
  • 27,600 tok/s on one 4xB300 replica at 512 concurrent requests
  • Full breakdown available via the linked post