Anthropic publishes R&D automation metrics: Claude led 26% of AI research, 30,000 agents active

On Sept. 17, Anthropic released a new set of metrics for tracking research and development progress at frontier AI labs. The data shows that, as of August, Claude led about 26% of the company's AI research work, while roughly 30,000 AI agents were simultaneously carrying out research and engineering tasks inside the company. Claude, however, has not reached fully autonomous operation in any of the measured AI research tasks.

The metrics reference Epoch AI's automation levels for AI research and development. At the "AI leads" level, humans provide high-level direction while AI handles most of the concrete work under human supervision. By this standard, Claude's share of AI-led research was below 1% in February; more than 90% of relevant tasks have now reached "AI collaborates" or a higher automation level.

Anthropic also analyzed more than 1 billion AI-agent decisions in August. Its online monitoring systems blocked about 0.002% of those decisions, or roughly 1 in every 47,000. The company stressed that the figure measures the share of decisions stopped by its monitoring systems and is not equivalent to an AI error rate or dangerous behavior rate.

During the week of July 13-20, about 6% of Anthropic's AI research compute was used for safety work; when considering only AI-driven AI research, about 12% of compute went to safety-related work. Anthropic said it plans to give independent third-party evaluators access to internal processes and data to verify safety practices and help outsiders understand actual AI R&D progress inside frontier labs.

Anthropic said it hopes the ongoing disclosures will narrow the gap between actual progress inside frontier AI labs and what the public can learn. The current data shows AI is deeply embedded in frontier model development, but fully autonomous AI research and development has not yet been achieved.