Key Info
AsyncGRPOTrainer is now stable in TRL v1.18.0, enabling generation and training to run at the same time on separate GPUs rather than taking turns.
Highlights
- AsyncGRPOTrainer reaches stable status in TRL v1.18.0
- Generation and training execute concurrently on separate GPUs
- Documentation covers how to run it and what it can train
- Includes a comparison with the standard GRPOTrainer