Key Info
The author notes that the industry is again benchmarking new models on 3D games (referencing their own earlier "flappy bench"), but argues the metric that matters is tokens burned per real fix in a real repo — which is why they built TEF bench.
Highlights
- Community trend: 3D game benchmarks are back in fashion for evaluating new models
- Author admits previous participation with "flappy bench" last month
- Core proposal: measure tokens consumed per genuine fix in a real repository
- The benchmark is called TEF bench and is linked in the thread