Anthropic says it has begun giving six employees of the nonprofit METR access to its internal systems to assess AI risks. The arrangement gives a small outside team a closer view of the company's work than a public model release or risk report would provide. Anthropic still controls which information the team receives and can require specified material to be removed from METR's published analysis.

Anthropic announced the engagement on October 15 and updated its statement on October 16 to clarify the current publication terms. The announcement describes an access arrangement, not a completed METR assessment of Anthropic's risks. Its value to readers will depend on what METR can examine and what its eventual report can explain about any limits.

The big change

  • What changed: Anthropic says six METR employees have company hardware, office space and internal-tool access roughly comparable to its own risk assessment teams. METR is expected to publish analyses of Anthropic's risk levels, incidents or practices.
  • Why it matters: An outside evaluator may be able to examine internal evidence that the public cannot see. METR's eventual account can also describe material limits on its access and redactions, helping readers weigh the conclusions.
  • What to watch: Anthropic says it cannot veto METR's publication, but it can require redactions under preset categories and ultimately decides what falls within them. No assessment result from this embedded arrangement is established by the announcement.

What the six evaluators can access

Anthropic says it has onboarded six METR employees with company hardware, office space and access to internal tools and systems “roughly on par” with its risk assessment teams. It encourages staff to share relevant information with them as they would with a colleague working on alignment risk. The company names customer data, attorney-client privileged information and model architectures as examples of excluded categories. It says architecture information could sometimes be shared when highly relevant, subject to explicit internal approvals.

That is a substantial description of access, but not a list of every dataset, system or interview the team has actually used. Anthropic's statement says the team operates under enforceable agreements and that other METR employees, including researchers on other Anthropic projects, are siloed from its work. BIG CHANGE has reviewed the public statements; we have not inspected the agreement or the internal systems.

METR's October 14 proposal for loss-of-control assessments explains the kind of evidence it wants from frontier developers. It identifies three questions: whether internal AI agents might take seriously harmful actions, whether monitoring would catch and stop them, and whether AI-driven research could accelerate capabilities faster than safeguards keep pace. For those questions, METR describes potential access to model checkpoints, internal transcripts, monitoring systems and measures of research progress. That document is a proposal across developers, not an inventory of what Anthropic has supplied under this engagement.

Publication rights and their limits

Anthropic says METR can publish analysis of risk levels, incidents or company practices without an Anthropic veto. The company can require redaction of information in predefined categories. Its October 16 clarification adds that Anthropic ultimately decides whether information falls within those categories, while METR reserves the right to make certain statements about the access and information it received. A right to publish findings therefore coexists with company control over sensitive details.

METR's policy on working with AI companies says its reports should disclose the scope and duration of work, access limits that affect conclusions, important redactions and pressures that may shape the result. METR says it aims to make a redaction summary when a company removes information important to its conclusions. Those are METR's stated practices and aims; the public cannot yet check how they will operate in a report from this new embedded team.

METR also acknowledges that voluntary assessments depend on companies granting access and allowing publication. It says companies can withhold access or redact material in ways that affect the evidence the public receives. The October 14 proposal says the exercise would focus on technical loss-of-control questions and would not amount to an audit of security, organizational governance or compliance. METR says the underlying science cannot yet give confident assurance that development is safe.

Anthropic describes a three-tier approach for other evaluators. Its METR arrangement is the third tier, with employee-like access and publication rights. Earlier tiers offer narrower investigations or more limited publication rights. Those tier descriptions are Anthropic's plans and current engagement terms, not evidence that every partner has METR's access.

The public test of this arrangement will be METR's reporting: which risk questions it investigated, what internal evidence it could examine, and which limits or redactions it can describe. The current statements establish the terms Anthropic has announced and METR's published approach. They do not establish a finding about Anthropic's safety or the effectiveness of the embedded evaluation.

Sources & further reading