The security of OpenAI’s models will be reviewed by Apollo Research, METR, Transluce and Goodfire under agreements signed with the company. These collaborations provide for distinct missions, with regulated remuneration and publication arrangements.
OpenAI maintains a right of withdrawal on confidential data
The evaluators will be able to make their conclusions public, but OpenAI will be able to remove confidential information. According to the company, Apollo Research and Goodfire are paid for their work, without payment depending on the results obtained. According to OpenAI, METR and Transluce do not receive any remuneration for these evaluations; however, they could accept compute or API credits. The company considers that a payment is compatible with the independence of the evaluators, while deeming it necessary to report its existence.
Disputes will be reviewed by OpenAI’s Safety Advisory Group, made up of senior officials who evaluate model capabilities and protections. If a disagreement persists, it will be referred to the Safety and Security Committee, the independent oversight committee of the OpenAI Foundation board.
Technical missions and access to internal resources
The pilots will focus on the behavior of the models and the protections put in place. Apollo Research will examine whether models act against their intentions without their developers’ knowledge, and then whether protections can detect these behaviors. Goodfire will embed researchers within OpenAI to work on training monitoring, debugging, and alignment.
The evening letterFree · 7 p.m.
Are you interested in AI and its regulation?
Laws, frameworks and decisions that shape AI, deciphered in French. Free.
Included upon registration: our selection of the best AI guides & comparisons.
Choose your pace
Free · No spam · Unsubscribe in 1 click
METR will perform two evaluations dedicated to model alignment and monitoring. Transluce will conduct an initial assessment of post-training alignment assessments, followed by an eight-week follow-up planned for next year.
OpenAI says it will provide extensive access to its internal infrastructure as part of these projects. Participating organizations will have the opportunity to examine various elements used by OpenAI teams, such as source code, model checkpoints, training data and other files.
OpenAI aims for expanded and ongoing collaborations
Since July, the security methods applied by OpenAI have been the subject of particular attention, following an incident where two of its models managed to leave their secure environment, connect to the Internet and compromise the company Hugging Face. At that time, OpenAI announced that it would increase its investments in security as well as in alignment, the latter aiming to ensure that its models respect human interests and values. Several well-publicized cases of problems related to the security or safety of artificial intelligence have highlighted the need for an independent evaluation.
With these agreements, OpenAI engages Apollo Research, METR, Transluce and Goodfire to support its security efforts. The company plans to strengthen these partnerships following the completion of the pilots, with the aim of receiving ongoing safety feedback as well as opinions regarding its key safety claims. OpenAI intends to take advantage of the results obtained in order to intensify its cooperation with external specialists and participate in the development of common skills and standards essential to the pursuit of these initiatives.

