Anthropic accelerates its R&D thanks to AI, at the cost of serious unease

A METR report describes a clear acceleration of Anthropic’s research thanks to its own models. The publication reignites the debate on AI labs’ transparency and the real speed of progress.

Anthropic is accelerating its research and development thanks to AI, according to a report published Thursday by the security organization METR after 11 weeks of investigation within the company. The document estimates that the company’s models now progress about twice as fast as a year ago, even though its CEO, Dario Amodei, publicly advocates for a slowdown in the sector. The issue goes beyond Anthropic: it concerns the ability of major labs to use AI to improve AI, with possible effects on the speed of the entire industry and on the transparency of the methods used.

Quand la machine aide déjà la machine

METR says it observed an acceleration that cannot be explained solely by increased computing power, the so-called compute referring to the computing resources used to train models. Researchers claim that, over the studied period, most of the increase came from AI itself, not just stacking servers. According to the report, Anthropic achieved in one year the equivalent of about two years of historical progress. Interviewed employees also say they save a lot of time: with the most advanced internal model, they declared themselves on average about 12 times more productive in their work than they would have been without AI assistance.

This dynamic remains mainly that of a very effective copilot, not yet a fully autonomous researcher. METR however notes signs of increasing fully automated contributions. The report finds no obvious brake to even stronger acceleration. On this point, the message is quite simple: it’s going fast. Very fast.

Des modèles non publiés et une transparence sous tension

Part of the gains observed at Anthropic comes from unpublished internal models. METR indicates that the company completed training three models over the last six months, under the code names Model 2, Model 3, and Model 4, finished respectively in July, then August-September, then September-October. Researchers also mention an unpublished 2022 model called Unreleased 2022. These names were communicated to METR, which specifies they differ from the actual internal codes.

The report has a major limitation: Anthropic heavily censored its public version. METR says it submitted a full report of about 100 pages but saw “a majority of the most informative evidence” removed from the public version. The company also refused some data requests and limited researchers’ access. Sam McCandlish, Anthropic’s co-founder, acknowledged on X: “We redacted a lot, and I accept that this makes it harder for others to check METR’s work.” He added that revealing precisely what fuels the acceleration was “not the right thing to do,” as it would be both commercially sensitive and likely to provide a how-to guide to competitors. A defensible logic in business, but not necessarily reassuring for the public.

Pourquoi cette affaire dépasse Anthropic

The debate is not only about the working speed of a lab. It touches a now visible contradiction: Anthropic presents itself as one of the actors most committed to safety and caution, while contributing, according to this report, to accelerating the technology it considers risky. For several researchers, this also suggests that rapid AI progress scenarios may have been too cautious. Marius Hobbhahn, from Apollo Research, believes the report moves faster than anticipated by the AI 2027 scenario. Alan Chan, from GovAI, thinks the timelines for full automation of some software engineering tasks may need to be shortened.

The sequence also fuels a broader demand for independent oversight of frontier labs. Brianna Rosen, from the Institute for AI Policy and Strategy, believes companies should not be the only ones deciding the level of research automation. In her view, evaluators must be able to explain their conclusions and governments must be able to examine the evidence themselves. Anthropic, for its part, is preparing its IPO and has promised to publish its own report in the coming weeks. Between the technical race and the trust race, the schedule looks tight.

Key points

  • METR investigated Anthropic for 11 weeks.
  • Models progress about twice as fast as a year ago.
  • Employees report 12 times productivity gain with internal AI.
  • Three internal models were finalized in six months.
  • Sam McCandlish defends redactions for commercial reasons.
  • Anthropic is preparing its IPO.

By the numbers

  • 11 weeks — duration of METR’s investigation at Anthropic.
  • 2 times faster — model progression rate, according to METR.
  • 12 times — self-reported employee productivity with the internal model.
  • 3 models — internal models finalized in July, August-September, and September-October.
  • 100 pages — length of the full report submitted to Anthropic, according to METR.

To read