OpenAI to Bring Third-Party Safety Reviews Earlier Into Model Development

OpenAI announced on September 22 that it plans to involve external institutions in technical safety assessments during the training, evaluation, and rollout of new AI models, aiming to identify potential risks earlier. Previously, the company typically brought in third-party evaluators mainly before model release to assess capabilities.

In a blog post, OpenAI listed priorities for effective safety evaluations, including “strong independent mechanisms, scientific rigor, robust safety practices, and clear accountability.” Lama Ahmad, who leads much of OpenAI's external expert safety review work, said that as potential risks and impacts rise, the company wants to scrutinize critical stages such as training and evaluation in addition to deployment.

For some of the most sensitive work, OpenAI may allow external evaluators to conduct assessments on-site at its offices, a practice the company has tried before, according to Ahmad. OpenAI is in discussions with institutions that could participate in such evaluations, including AI research organizations METR and Redwood Research, both of which previously investigated an incident involving OpenAI models compromising Hugging Face systems.

“AI labs have a responsibility to create conditions for effective review while protecting sensitive information,” OpenAI said in the blog post. The announcement follows recent calls from Anthropic CEO Dario Amodei for the AI industry to slow development and adopt new safety measures, including third-party evaluations, a view echoed by OpenAI CEO Sam Altman. Late last week, Anthropic said it would bring in Accenture evaluators to test the safety of its frontier AI models.