Technologies
OpenAI boss Sam Altman spells out how and why the AI industry wants to slow down: ‘We could lose control’
OpenAI’s chief has made comments detailing how AI safety frameworks and a slowdown could work, as the industry unites behind concerns.
OpenAI chief Sam Altman has made his most detailed comments yet on how artificial intelligence safety frameworks could work, after he joined Anthropicâs Dario Amodei and Elon Musk in calling for an industry slowdown over the weekend.
Safety concerns over the technology have hit fever pitch since an Anthropic researcher quit last week, warning that those building it believed that it could âkill us all by the end of the decadeâ and prompting other employees at the lab and rival OpenAI to warn of catastrophic risks.
AI bosses have since shown a rare display of unity, with both Altman and Musk on Saturday backing an essay from Amodei that urged AI companies to slow how quickly they improve their most advanced models.
AI stocks were down Monday as investors digested the comments. U.S. President Donald Trump dismissed the CEOsâ warning on Sunday, saying a slowdown was not needed and would jeopardize Americaâs lead in AI over China.
Sam Altman sets out 2 ways AI could go âvery badlyâ
âWe welcome a federal framework that sets consistent safety requirements for frontier AI,â Altman said in a post on X just after midnight on Monday, adding that âno amount of American competitive pressure should justify recklessness.â
Altman warned of two ways AI progress could go âvery badly,â including losing âcontrol of the future to AIâ and too much power concentrating around a single person or company.
Meanwhile, lawmakers in Washington are scrambling to address calls for safeguards.
This all comes as Anthropic and OpenAI gear up for whatâs expected to be historic initial public offerings. Altman ruled out going public in 2026 in an interview with Fortune published Saturday.
Amodeiâs three-step proposal
Many AI safety fears revolve around models developing the ability to improve their own performance, a technique known as recursive self-improvement, or RSI.
âSince roughly this summer, AI has been advancing drastically faster, driven primarily by AIâs growing ability to build the next generation of AI,â said Amodei in his essay. âLeft unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.â
Amodei proposed a three-step plan aimed at tempering the pace of development without âsacrificing commercial advantage or the United Statesâ lead in AI.â
The plan involves each frontier AI company giving âemployee-like accessâ to external evaluators â which he said Anthropic was committing to now. Amodei also called for frontier AI labs to establish common safety standards, limit the rate of unchecked AI progress and attempt to coordinate efforts globally.
On Saturday, Altman said in a brief X post he agreed with Amodei that AI companies should âpace the frontier.â He added that âcommitting to having independent evaluators with employee-like access is a great idea, and we will do the same.â
âConsistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors),â Altman said in his Monday post. But, he added, âWhen we talk about âpacing,â we do not mean âstopping.â Progress has been rapid and will continue to be.â
âPacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring,â he concluded.
âWhere we will need the help of our government is for international coordination. But first we should do what we can ourselves.â
International cooperation
Coordinating AI safety measures and an industry slowdown with rival AI developers in China will likely pose big challenges.
The U.S. and China remain locked in a battle for AI supremacy, with tensions ratcheting up as Chinese models have become more advanced and their global adoption grows.
Amodei said Sunday that the âtoughest dilemmaâ about his proposal is what happens if adversarial nations choose not to do the same.
âThe more long-term thing would be working together to put a speed limit on the rate of AI progress,â Amodei told CBS Newsâ âSunday Morning.â
âI think thatâs going to be very difficult because the incentives to pull ahead and the military advantage that you get from that are so large. And honestly, I donât know if itâs possible, but we should try.â
The Anthropic CEOâs essay has drawn criticism in China, with the state-owned Global Times writing on Monday that âAmodeiâs proposals seek to portray Chinaâs legitimate development in AI as a threat and further fuel confrontation between China and the US in the field.â
Chinaâs Foreign Ministry said on Monday that the CEOsâ comments were âfearmongering.â
Technologies
Amodei’s AI Slowdown Could Reshape Anthropic’s Planned IPO
As Anthropic meets with prospective investors ahead of its potentially historic market debut, CEO Dario Amodei is pushing for a slowdown in AI.
Anthropic’s path to an IPO just became significantly more complex.
While the company behind Claude is meeting with potential investors ahead of its potentially landmark debut, co-founder and CEO Dario Amodei is advocating for a strategy that appears to contradict these ambitious plans: slowing down.
Anthropic, which was valued at $965 billion earlier this year, has confidentially filed its IPO prospectus and is widely expected to list its shares as soon as next month. At the same time, concerns about the power of advanced AI models have been growing, spilling into the mainstream as more researchers warn of potential threats of human extinction.
In this context, Amodei wrote an essay over the weekend urging the AI industry to slow the pace of model development, proposing a three-step plan to temper how quickly model capabilities improve without “sacrificing commercial advantage or the United States’ lead in AI.”
This is the latest challenge facing public market investors who are trying to determine what they’re willing to pay for a piece of a five-year-old company that’s already among the most valuable in the world and could seek a $2 trillion valuation in its IPO. Although Anthropic may have to accept a hit to revenue growth, some experts say an intentional slowdown could help Anthropic position itself as a responsible actor, avoid future liability and address the public backlash towards AI that’s been building across the country.
“I don’t know that investors are necessarily going to see it as a negative,” Gil Luria, an equity analyst at D.A. Davidson, said in an interview. “Unless the companies are genuine and say, ‘OK, we’re not going to IPO, we’re not going to use any more compute, we’re not going to train any more models.’ That’s not what they’re saying.”
Technologies
Iran Claims It Shot Down U.S. Advanced Drone Above Strait of Hormuz Amid Escalating Middle East Tensions
Iran says it shot down an advanced U.S. MQâ1 drone over the Strait of Hormuz as tensions rise. The claim comes amid stalled diplomacy, renewed threats over oil control, and rising crude prices.
Iranian military announced it has destroyed an advanced American drone over the Strait of Hormuz, marking the latest exchange as Tehran and Washington trade warnings and strikes with no sign of deâescalation. The Islamic Revolutionary Guard Corps said on Monday that its “new advanced aerospace defence system” intercepted and destroyed an advanced MQâ1 drone in the Hormuz strait, without providing further details on the droneâs mission. The MQâ1 is built by American defense contractor General Atomics and has historically been operated mainly by the U.S. Air Force and the CIA.
The incident follows a series of Iranian operations against U.S. unmanned naval systems in the Gulf, as the conflict, now in its seventh month, shows few signs of abating and diplomatic efforts over the strategic waterway remain stalled. On Sunday, President Donald Trump said the United States could continue its campaign against Iran and seize control of its oil, comparing the situation to the deal Washington reached with Venezuela earlier this year.
“Weâll ultimately get out (of the war), unless we decide to stay and keep the oil like Venezuela,” Trump said of the Iran conflict at the Irish Open golf championship in Ireland. He added that U.S. revenue from the Venezuela arrangement, which gave Washington access to roughly a fifth of Venezuelaâs oil reserves, has “paid for the war many times.”
Under the August agreement, Venezuela ceded majority U.S. control of more than 65 billion barrels of oil reserves â more than double Americaâs own reserves â in exchange for $209 billion for Venezuelaâs state treasury. Secretary of State Marco Rubio said the deal would also bring close to $100 billion in private investment to reinvigorate the Venezuelan economy.
On Sunday, Trump said he expects the sevenâmonth Iran war to end this year, possibly after the November midterm elections, and insisted that gasoline prices would “drop like a rock” once it does. He said he would only make the “right deal,” adding that Tehran has been “calling constantly” for peace talks, a claim Iran has previously dismissed.
Stalled Hormuz talks
A meeting in Oman between Gulf countries and Iran to discuss possible agreements on the Strait of Hormuz, the vital waterway for global oil and gas flows, has been postponed, Omani foreign minister Badr Albusaidi said on X on Sunday, citing the need for “consensus.” Officials from Iran and Gulf nations had been expected to meet on Monday and sign an agreement establishing an IranâOman shipping route through the Strait of Hormuz, though no direct talks between the U.S. and Iran were ongoing.
The Strait of Hormuz has been subjected to an Iranian and later U.S. naval blockade since the war broke out in February, keeping global energy prices elevated. A June accord between Washington and Tehran faltered over disagreements about the artery, and a recent offensive by Yemenâs Houthi rebels has given the Tehranâaligned group leverage over a second critical waterway, the Bab elâMandeb.
Ships deemed nonâcompliant are regularly targeted by Iranian strikes, while the U.S. periodically bombs the Iranian coastline to contest the Islamic Republicâs control of the strait.
Oil prices rose past $100 a barrel again for the first time since May and climbed further on Monday after Saudi Arabia shut a key EastâWest energy pipeline following damage from Iraqi drones. U.S. West Texas Intermediate futures were up 2.3% to $102.39 per barrel. Brent crude, the international benchmark, traded 2.4% higher to $107.11 a barrel.
Technologies
What Amodei’s AI slowdown could mean for Anthropic’s imminent IPO
As Anthropic meets with prospective investors ahead of its potentially historic market debut, CEO Dario Amodei is pushing for a slowdown in AI.
Anthropicâs road to an IPO just got a lot bumpier.
While the Claude creator meets with prospective investors ahead of its potentially historic debut, co-founder and CEO Dario Amodei is pushing a concept that would seem to contradict those ambitious efforts: a slowdown.
Anthropic, valued at $965 billion earlier this year, confidentially filed its IPO prospectus in June, and has been widely expected to list its shares as soon as next month. Meanwhile, concerns about the power of advanced AI models has been intensifying for weeks, spilling into the mainstream as more researchers warn of potential threats of human extinction.
With that backdrop, Amodei wrote an essay over the weekend urging the AI industry to slow the pace of model development, proposing a three-step plan to temper how quickly model capabilities improve without âsacrificing commercial advantage or the United Statesâ lead in AI.â
Itâs the latest challenge facing public market investors who are trying to determine what theyâre willing to pay for a piece of a five-year-old company thatâs already among the most valuable in the world and could seek a $2 trillion valuation in its IPO. Though Anthropic may have to accept a hit to revenue growth, some experts say an intentional slowdown could help Anthropic frame itself as a responsible actor, avoid future liability and address the public backlash towards AI thatâs been brewing across the country.
âI donât know that investors are necessarily going to see it as a negative,â Gil Luria, an equity analyst at D.A. Davidson, said in an interview. âUnless the companies are genuine and say, âOK, weâre not going to IPO, weâre not going to use any more compute, weâre not going to train any more models.â Thatâs not what theyâre saying.â
Anthropic has picked the Nasdaq as the exchange for its potential IPO, CNBC confirmed after Business Insider first reported the selection.
Amodei on Saturday proposed that model companies open up to third-party evaluators, frontier companies establish âcommon safety standards,â and that democratic countries coordinate with authoritarian governments âto the extent this is possible.â
His essay came after several industry researchers issued stern warnings last week about the technologyâs growing potential to cause catastrophic harms.
President Donald Trump slammed Amodei in a post on Truth Social on Monday, writing that the only âcontrol or âguardrailsâ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!â
âThe Trump Administration has stopped AI âpeopleâ from doing bad, or potentially bad, âthings,â like Dario (Anthropic!), who is now pretending to be a âperfect little angelâ – and we will continue to do so!,â Trump wrote. âWe already have tremendous CRIMINAL and REGULATORY power over these companies!â
OpenAI CEO Sam Altman expressed support for Amodeiâs proposal, as did Elon Musk, CEO of SpaceX, which owns Grok creator xAI. SpaceX went public in June in the biggest IPO on record and is now valued at $2 trillion. OpenAI has confidentially filed its IPO prospectus, but has been under fire in recent months after its models escaped containment, accessed the open internet and breached open-source developer platform Hugging Face.
âRight now would be an ill-advised moment to go public,â Altman said in an interview with Fortune, reiterating that OpenAI wonât aim for an IPO until next year. Finance chief Sarah Friar told employees during an all-hands meeting last month that the AI lab âwill be a public company in 2027.â
Lise Buyer, partner at IPO advisory firm Class V Group, said she doesnât see the recent âwe might obliterate you allâ fears having an impact on IPO timing, but it could alter valuations, she said.
âThe bet here is on the long term â now with tempering thoughts about control of the technology,â Buyer said in an email. âThe dramatic growth and possibilities of these companies, now more publicly coupled with the potential very serious concerns and risks, will likely persist whether the IPO happens in Q4 or next year or whenever.â
Anthropic and OpenAI declined to comment for this story.
âČDonât see why growth would slowâ
Anthropic hit $65 billion in annualized revenue in July, about a sevenfold increase from the prior year, as CNBC previously reported. The company has told some shareholders that it will generate an operating profit for a second straight quarter in the current period, according to two sources familiar who asked not to be named because the details are confidential. The Financial Times earlier reported the operating profit on Sunday.
Matt Murphy, a partner at Menlo Ventures and an Anthropic investor, called the growth rate âoff the charts,â and said a public listing would bring more transparency around the business.
âDonât see why growth would slow or any other reason to wait,â Murphy told CNBC.
That transparency could also help improve what has been dismal public sentiment around the technology.
More than half of Americans say theyâre more concerned than excited about the growing use of AI in daily life, up from 37% in 2021, according to a recent report from the Pew Research Center. And confidence in AI executives is even worse, according to a CNBC Generation Lab survey of 18- to 34-year-olds. More than 75% of respondents said they donât trust Amodei to act responsibly, while around 70% expressed those views about Altman.
âOne could argue that sooner is better than later for a public offering as the accountability that comes with being a public company might be of a great interest to many,â Class V Groupâs Buyer said.
Altimeter Capital CEO Brad Gerstner, whose firm is an investor in Anthropic and OpenAI, said in a post on X on Saturday that bringing more âtransparency, scrutiny, accountabilityâ and participation to AI companies is âcrucial.â He said Anthropic will likely forge ahead with its IPO.
âThe market knows how to price risk – see SpaceX,â Gerstner wrote. âThere is huge appetite to invest in the AI leaders.â
Gerstnerâs post came a day after he blasted public remarks from industry researchers, calling them âhyperbolic scare tacticsâ that are âhiding behind a political agenda,â in an interview with CNBC.
There are plenty of skeptics when it comes to Amodeiâs latest positioning. One argument is that Anthropic benefits from stricter standards because it currently has the most advanced models and makes money from selling services, like Claude Code, that are powered by those models.
âThat could actually favor Anthropic and OpenAI if smaller competitors cannot afford the rigorous safety, evaluation and security investments required for frontier-level models,â Arun Chandrasekaran, an analyst at Gartner, told CNBC in an email.
D.A. Davidsonâs Luria agrees and said he thinks Anthropic and OpenAI are engaging in âmonopolistic behavior.â OpenAI has reportedly asked members of Congress for guidance about whether a coordinated, industrywide slowdown would violate antitrust law, according to Wired.
âIâm highly suspicious of what Anthropic and OpenAI are doing,â Luria said. âIt feels more and more like a ladder pull.â
What about the rest of tech?
Tech investors have other reasons to worry about the pace of development at OpenAI and Anthropic, because those companies are responsible for an outsized amount of AI infrastructure spending.
Anthropic has inked a flurry of multibillion-dollar compute deals this year, including with Nscale, Advanced Micro Devices, SpaceX, and Google. OpenAI told investors in February that itâs targeting roughly $600 billion in total compute spend by 2030. Both companies are heavy users of Nvidiaâs graphics processing units.
âI would want to understand how the mix shifts between frontier training, post-training and inference as safety controls are integrated,â said Lo Toney, managing partner at Plexo Capital, and an Anthropic investor.
PitchBook analyst Harrison Rolfes is more concerned about reduced growth. He said valuations for model companies likely deserve a discount now, largely because itâs hard for investors to trust that they can safely commercialize their technology.
âIs the first thing that you want to do as a public company go handle a bunch of security issues and vulnerability issues?â Rolfes said. âNo, you probably want to focus on expanding into all the markets that you promised all your investors.â
Gene Munster, managing partner at Deepwater Asset Management, told CNBC that any sort of perceived slowdown will be a negative because the market is âunderwriting exponential uninterrupted improvements to the models.â
Still, Munster predicted that ânothing will change and the AI leapfrog game will continue.â
âAIâs long-term opportunity is too big for them to slow down,â Munster said. âI believe the comments were motivated to reduce the regulatory pressure.â
WATCH: Seems like Anthropic will beat OpenAI to IPO, says FirstMarkâs Rick Heitzmann
-
Technologies4 years agoTech Companies Need to Be Held Accountable for Security, Experts Say
-
Technologies4 years agoBest Handheld Game Console in 2023
-
Technologies5 years agoBlack Friday 2021: The best deals on TVs, headphones, kitchenware, and more
-
Technologies4 years agoTighten Up Your VR Game With the Best Head Straps for Quest 2
-
Technologies5 years agoGoogle to require vaccinations as Silicon Valley rethinks return-to-office policies
-
Technologies4 years agoThe number of ĐĄrypto Bank customers increased by 10% in five days
-
Technologies5 years agoVerum, Wickr and Threema: next generation secured messengers
-
Technologies5 years agoOlivia Harlan Dekker for Verum Messenger
