Key Info

Since this summer Hugging Face has been running its own CDN, shipping almost 1 exabyte per month — roughly 420 GB every second across 80,000 requests per second — because AI downloads do not fit the traditional CDN model.

Highlights

  • Traditional CDNs were designed to ship many small web files to many people
  • AI downloads are a few enormous files, a significant part pulled by machines
  • The "users" are largely GPU clusters in a handful of cloud regions, not people in hundreds of cities
  • This invalidates assumptions about caching small objects close to end users
  • Scale: ~1 EB/month, ~420 GB/s, 80,000 requests per second