Key Info
Hugging Face worked with Thinky Machines to make Inkling 2 Mini run fast locally on Apple Silicon, achieving 210 tok/s decode and 3620 tok/s prefill on a 96GB M5 Ultra Mac Studio, and 125 tok/s decode and 1750 tok/s prefill on a 48GB M5 Max MacBook Pro.
Highlights
- On an M5 Ultra Mac Studio, Inkling 2 Mini built a native weather app 1.8x faster than Claude Sonnet 5.5.
- The model is described as 11x cheaper than Claude Sonnet 5.5 based on API pricing.
- Decode speed reaches 210 tok/s on 96GB M5 Ultra and 125 tok/s on 48GB M5 Max.
- Prefill throughput is 3620 tok/s and 1750 tok/s respectively.
- The result is framed as evidence that local and open source AI can compete on speed and cost.