Key Info

Hugging Face worked with Thinky Machines to make Inkling 2 Mini run fast locally on Apple Silicon, achieving 210 tok/s decode and 3620 tok/s prefill on a 96GB M5 Ultra Mac Studio, and 125 tok/s decode and 1750 tok/s prefill on a 48GB M5 Max MacBook Pro.

Highlights

  • On an M5 Ultra Mac Studio, Inkling 2 Mini built a native weather app 1.8x faster than Claude Sonnet 5.5.
  • The model is described as 11x cheaper than Claude Sonnet 5.5 based on API pricing.
  • Decode speed reaches 210 tok/s on 96GB M5 Ultra and 125 tok/s on 48GB M5 Max.
  • Prefill throughput is 3620 tok/s and 1750 tok/s respectively.
  • The result is framed as evidence that local and open source AI can compete on speed and cost.