Anthropic grants METR researchers employee-level access for independent AI safety evaluation

Anthropic announced on October 15 that it is giving six researchers from METR, a nonprofit that evaluates frontier AI models, employee-level access to its systems. The researchers will receive Anthropic-issued laptops, office space, and internal tools comparable to those used by Anthropic's own risk assessment teams, allowing them to independently assess the company's models, safety practices, and risk management.

In a blog post, Anthropic said it wants to be publicly accountable for its decisions and to close the gap between what external evaluators understand about a frontier AI company's systems and what the company itself understands. It compared the arrangement to regulators placing resident examiners inside large banks, and encouraged employees to share information with the researchers as they would with a colleague working on alignment risk.

METR can publish its own assessments of Anthropic's risks, incidents, and practices, and Anthropic has given up the right to block publication. However, the company can require redaction of information falling into narrow categories, including customer data, attorney-client privileged material, and model architecture details. METR may publicly characterize the redactions and state whether it considers them justified, as long as it does not reveal the redacted information.

The move fulfills a proposal Anthropic CEO Dario Amodei made in a September essay advocating for embedding outside evaluators in AI labs. OpenAI subsequently made a similar pledge, and Anthropic, OpenAI, Google, Meta, Nvidia, and xAI have signed a White House accord committing to independent review. METR is currently the only organization at the top level of a three-tier evaluation system Anthropic is building.