Key Info

A first test run of a new GPU rack running a DeepSeek inference setup processed 6.77T tokens in 24 hours, with p50 TTFT of 1.16s and p50 TPS of 143, according to the author.

Highlights

  • The rack handled 6.77T tokens over the last 24 hours.
  • p50 time-to-first-token was 1.16 seconds; p50 tokens per second was 143.
  • p95 figures were degraded because one tray is flaky and needs replacement.
  • The author claims the price/performance beats every vendor they have found, with more optimization still to come.