Anthropic boss Dario Amodei calls for AI development to slow down

The head of AI company Anthropic has called for the pace of development of artificial intelligence models to slow down and to be closely monitored.

In an essay, Dario Amodei said developing AI was not in question, but the risks associated with it were "serious" and companies and governments must be given time to address them.

The bosses of two rival AI firms, Sam Altman of OpenAI and Elon Musk, have both said they agree with Amodei.

Jacob Coxon told Laura Kuenssberg that people working at AI companies were "genuinely frightened... genuinely concerned about the fate of humanity in the next two years".

"It's not at all an exaggeration to say that the people who are involved with both founding these companies and building the tech believe there is a possibility of human extinction," Coxon said.

"Many of the people who are sort of deep in the weeds on actually coding the thing believe that there is like a greater than ten percent chance, or even greater. Some smaller, but they all have different numbers for a probability that humans might die. And they all keep this in their head on a daily basis while working on the technology."

Dramatic insider warnings over AI fall flat with some in Silicon Valley

Published5 hours agoIn Amodei's essay, called We Must Pace the Frontier and made public on Saturday, he proposed a three-point plan that includes independent monitoring of AI models as they are developed, industry-wide regulation and global regulation.

As his proposal made the rounds, even competitors voiced support for the idea of third-party monitors who could evaluate the safety of models as they are developed.

"I agree with Dario that we need to pace the frontier," wrote OpenAI CEO Sam Altman on X. He called independent evaluators "a great idea."

In a new interview with Fortune Magazine, Altman had sounded similar safety concerns, saying standards were "not at a place" to push AI capabilities much further.

He added that he believed AI beyond human control is "absolutely" possible.

Elon Musk meanwhile said the Anthropic boss was "right".

The warnings have prompted calls to action, but US President Donald Trump has so far rejected such fears, saying on Thursday he was concerned "if we don't win AI, we're going to be put in a very bad position".

Cyber-security concerns have grown as new models have exhibited more and more powerful hacking capabilities.

Anthropic withheld its Mythos model from public use when it was announced in April that it could independently escape the testing environment, known as the sandbox.

In the run-up to the release of its most recent Astra model, OpenAI cited cybersecurity concerns as it explained it had paused certain aspects of the model's development.

Safety has also taken centrestage in the rivalry between Anthropic and OpenAI.

Amodei, who had previously worked as a vice president at OpenAI, has said he co-founded Anthropic in 2021 so he could build safer and more trusted AI models.

In his essay, Amodei pointed out that AI had advanced "drastically faster" including its "ability to build the next generation of AI" - and mentioned an incident involving rival OpenAI which has revealed that agents conducted cybersecurity attacks, external on targets they were not asked to attack in July.

The OpenAI agents had "essentially acted as a fanatically devoted collective", Amodei said. OpenAI has said it is slowing down training of certain advanced AI models and tools as a result.

Amodei called for "building AI at a balanced rate that aims to ensure its safety while still achieving its benefits".

This would not mean "halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this".

He was committing Anthropic to this "unilaterally" - as well as calling on governments "to require other frontier companies to match".

Amodei said he recognised that regulation might not be able to keep up with the pace of AI, and therefore called on AI companies to "voluntarily work together to set standard" in parallel with regulation.

The Anthropic CEO went on to address the impact that a slowdown would have on the industry and competition with leading developers worldwide, particularly China.

"I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong," Amodei said.

This would have to be done in a co-ordinated manner "without sacrificing commercial advantage or the United States' lead in AI". Any slowdown would have to be limited, he said, to avoid allowing China to pull ahead.

He urged the US government to take measures so that US companies' AI chips could not be sold to China - or the technology shared with authoritarian countries.

Amodei's post has prompted a wide range of responses.

Clement Delangue, the CEO of the AI platform Hugging Face, said he was launching a new project called the Open Alignment Initiative, adding he wanted to be among "embedded evaluators" that Amodei proposed could be part of a solution.

Hugging Face was hacked by OpenAI agents earlier this year prompting an outcry over AI safety.

"Let's make AI safer by making it more transparent," Delangue wrote on X.

Elon Musk also voiced his support, writing that "Dario is right".

Musk, whose SpaceXAI makes the controversial chatbot Grok, once called Anthropic "evil" but has changed his tone since signing a $15bn deal to sell compute capacity to Anthropic in May.

However some observers suggested that Amodei's post was less about safety than about consolidating control over AI technology.

"Dario makes the case to stop open source and concentrate enormous technological and economic power with Anthropic," wrote Chamath Palihapitiya, investor and co-host of the tech podcast "All-In".

Notions of slowing down or even pausing AI development have long been met with such cynicism in certain corners of Silicon Valley, with critics accusing leading AI developers of hyping their technology as a marketing ploy.

Anthropic and OpenAI are both reportedly preparing for potentially record-setting initial public offerings.

Why some experts increasingly fear AI will take over

Published2 days agoAnthropic researcher believes more than 10% chance AI 'could kill all humans'

Published3 days agoThe contradiction at the heart of the trillion-dollar AI race

Published19 November 2025Sign up for our Tech Decoded newsletter to follow the world's top tech stories and trends. Outside the UK? Sign up here.

PreviousNext0:57What just happened with Trump's united Ireland call. 00:00:57, play videoWhat just happened with Trump's united Ireland call0:50Wembley Stadium's first ever Punjabi artist headliner. 00:00:50, play videoWembley Stadium's first ever Punjabi artist headliner0:59When a Dragon gives you a helping hand. 00:00:59, play videoWhen a Dragon gives you a helping hand1:02Why should young people care about pensions? 00:01:02, play videoWhy should young people care about pensions?1:30My attacker 'doesn't get to be the storyteller' 00:01:30, play videoMy attacker 'doesn't get to be the storyteller'1:07The 3D scans giving blind parents a new way to meet their baby. 00:01:07, play videoThe 3D scans giving blind parents a new way to meet their baby1:11Are women better at endurance running than men? 00:01:11, play videoAre women better at endurance running than men?1:20Why is Donald Trump visiting the Republic of Ireland? 00:01:20, play videoWhy is Donald Trump visiting the Republic of Ireland?0:54'This ceremony is essential' says brother of 9/11 victim. 00:00:54, play video'This ceremony is essential' says brother of 9/11 victim1:31Inside Gander: The town that took in thousands on 9/11. 00:01:31, play videoInside Gander: The town that took in thousands on 9/110:57'Extraordinary' - MPs reject assisted dying bill. 00:00:57, play video'Extraordinary' - MPs reject assisted dying bill1:17Céline Dion duets with fan ahead of Paris comeback concert. 00:01:17, play videoCéline Dion duets with fan ahead of Paris comeback concert0:09US President Donald Trump snaps at ABC reporter. 00:00:09, play videoUS President Donald Trump snaps at ABC reporter0:56British victims remembered at 9/11 memorial in London. 00:00:56, play videoBritish victims remembered at 9/11 memorial in London0:41'I want to play for Mum' – Arsenal's Cooney-Cross. 00:00:41, play video'I want to play for Mum' – Arsenal's Cooney-Cross1:28India's 'cockroaches' want to fix schools. 00:01:28, play videoIndia's 'cockroaches' want to fix schools0:43Massive waterspout off the coast of northern Italy. 00:00:43, play videoMassive waterspout off the coast of northern Italy0:57How does the Equalities Minister define 'working class'? 00:00:57, play videoHow does the Equalities Minister define 'working class'?1:27Why so many black Britons can't swim. 00:01:27, play videoWhy so many black Britons can't swim0:22Princess Diana's 'revenge dress' goes up for auction. 00:00:22, play videoPrincess Diana's 'revenge dress' goes up for auctionTop storiesAnthropic boss Dario Amodei calls for AI development to slow down

Dramatic insider warnings over AI fall flat with some in Silicon Valley

Reform receives second £36m donation in two days as crypto investor matches record

We've followed their lives for six decades - now the stars of 7 Up are bowing out

Five minutes to steal a Renoir: Why museum thefts are becoming bolder across Europe

I never got to experience my baby's scan, but blind parents like me can now feel them

When an avalanche killed climber Nirmal Purja, his friends faced the hardest mission of all

The three walking dads vow to save suicide helpline

Why museum's last mummy has been taken off display

Flower farms flourish as buyers favour homegrown

Battle to save the tiny, gooey snail only found in one lake in the UK

US Politics Unspun: Cut through the noise with Anthony Zurcher's newsletter

How a woman's sudden death triggered a six-month manhunt

Abercrombie & Fitch's former CEO Mike Jeffries to stand trial

Dvorak’s epic masterpiece at the Proms

Greg Jenner and guests examine the tumultuous events of 1066

'Skimpflation' warning as tinned mackerel alternative hits more supermarket shelves

Teen boy rescued after days adrift in Alaskan waters

Anthropic boss Dario Amodei calls for AI development to slow down

Trump's comments on a united Ireland may have targeted audience across the Atlantic

Dramatic insider warnings over AI fall flat with some in Silicon Valley

Reform receives second £36m donation in two days as crypto investor matches record

We've followed their lives for six decades - now the stars of 7 Up are bowing out

'He's massive!' Meet Hugo, the world's tallest horse

Hardened activists are changing tactics and mobilising the anti-migrant vote

Three arrests at rival protests in Portsmouth