Key Info
Since this summer Hugging Face has been running its own CDN, shipping almost 1 exabyte per month — roughly 420 GB every second across 80,000 requests per second — because AI downloads do not fit the traditional CDN model.
Highlights
- Traditional CDNs were designed to ship many small web files to many people
- AI downloads are a few enormous files, a significant part pulled by machines
- The "users" are largely GPU clusters in a handful of cloud regions, not people in hundreds of cities
- This invalidates assumptions about caching small objects close to end users
- Scale: ~1 EB/month, ~420 GB/s, 80,000 requests per second