Key Info

Meta described Prometheus, its gigawatt-scale supercluster in New Albany, Ohio, where it trained Muse foundational models including Muse Wonder, and shared reliability metrics for large-scale training.

Highlights

  • Prometheus spans hundreds of thousands of GPUs, where a single underperforming host can slow an entire training run.
  • Meta engineered the cluster for resilience at scale, reaching 85%+ goodput on pretraining and 90%+ on reinforcement learning.
  • Mean time between interruptions during Muse Wonder training was about 47K GPU hours.
  • Meta linked to a technical write-up on the infrastructure powering Prometheus.