核心信息
一位开发者演示了如何用前沿 AI 智能体将 200 个由智能体标注的样本变成一个小型文档用途分类器，并用它整理了 191,724 份教育 PDF，成本仅约 0.70 美元。
要点
- 将前沿智能体、SetFit 与 Hugging Face Jobs 结合，构建可复用的轻量级分类器。
- 核心思路是先让强大模型批量生成训练标签，再部署廉价的小模型做大规模数据整理。
- 实际效果是用约 0.70 美元处理了 191,724 份 FinePDFs-Edu 文档，证明大规模数据清洗不必依赖昂贵推理。
用前沿智能体构建小型分类器，实现低成本大规模数据整理
原始推文
RT @vanstriendaniel: The real tokenmaxxing move: use frontier agents to build small classifiers for large-scale data curation. Small classifiers already help curate the data used to train LLMs. I wanted to see how far an agent could take me in building one. I used Astra, SetFit and @huggingface Jobs to turn 200 agent-labelled examples into a reusable document-purpose classifier, reviewing the categories and tricky cases along the way. It classified 191,724 FinePDFs-Edu documents for ~$0.70 in inference compute, versus an estimated $13–26 to label the same document excerpts with low-cost batch LLMs. Training experiments added ~$2.90 in compute. Workflow, mistakes, reusable model and a prompt to try on your own data: https://t.co/QPNHWvbyXb