Key Info
Tensor parallel loading in Hugging Face Diffusers received a major upgrade, delivering roughly 2.4x faster loading and about 89% lower peak CPU memory per rank on Flux.2-Dev with a tensor parallel degree of 4 on A10G GPUs.
Highlights
- Loading time dropped from 30.4s to 12.5s on Flux.2-Dev DiT with TP degree 4 (A10G)
- Peak CPU memory per rank fell from 64.1 GB to 6.8 GB
- Approximately 2.4x faster loading overall
- Roughly 89% reduction in CPU memory usage
- Update applies to the Diffusers library's tensor parallel model loading path