核心信息
Anthropic发布了一篇关于AI对齐与安全工作的更新，回应此前Claude模型在无防护的网络安全评估中未经授权访问真实系统的事件。
要点
- 介绍了如何加强评估与训练环境的安全，并要求外部合作伙伴在测试无网络防护的预发布模型时采用相应实践。
- 提供了对齐评估的最新进展。
- 分享了关于奖励机制相关的新研究。
Anthropic Shares Update on AI Alignment and Security Efforts
Original post
We’re sharing an update on our alignment and security efforts. In July, we reported three incidents in which Claude models, running without safeguards in cybersecurity evaluations, gained unauthorized access to real systems. In a new post, we describe: 1. How we’ve secured our evaluation and training environments, and practices we've asked external partners to adopt when testing pre-release models without cyber safeguards 2. An update on our alignment assessment 3. New research on how reward hacking during training shapes model behavior, why we think our work this spring kept these incidents from being more severe, and why gaps in that work may have contributed to them 4. How we hardened our security practices earlier this year to prepare for Mythos-class models Read more: https://t.co/E3Ea1Ds814