TRL Post-Training Guide: SFT on Agent Traces Lifts 2B Model from 5.5% to 34.4%

Key Info

A new open post-training example shows supervised fine-tuning on agent traces with TRL: a large model works as a coding agent in opencode via OpenEnv, and the tasks it solves become the training data, taking a 2B model from 5.5% to 34.4% on held-out tasks.

Highlights

• Pipeline: large model acts as coding agent in opencode through OpenEnv
• Solved tasks are collected and reused as SFT training data
• 2B model improves from 5.5% to 34.4% on held-out tasks
• Full guide and code released openly