The Cloudflare outage on Tuesday that disrupted access to many websites and services — including OpenAI, Spotify, X, Grindr, Letterboxd and Canva — was the company’s worst outage since 2019, CEO Matthew Prince says.

Other disruptions have centered on specific network features, Prince wrote in a blog post. «But in the last 6+ years we’ve not had another outage that has caused the majority of core traffic to stop flowing through our network.»

Cloudflare is a cloud services and cybersecurity company based in San Francisco that is used by approximately 20% of all websites, according to W3Techs. It’s one of a handful of services, along with Amazon Web Services, CrowdStrike and Fastly (all of which have experienced major outages in the past few years) that you might never have heard of, but that provide essential internet infrastructure.

The bulk of sites and services impacted by Tuesday’s outage, which began around 3:30 a.m. PT, seemed to recover within just over three hours. By the end of the day, everything had returned to normal, and Cloudflare set about explaining what went wrong. Here’s what you need to know.

Don’t miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source.

What caused the Cloudflare outage?

Cloudflare was keen to emphasize that the outage was not caused either directly or indirectly by a cyberattack. At first the company did suspect it might have originated from a «hyper-scale DDoS attack,» Prince said in his blog post. But it turned out that the outage resulted from an internal software failure.

A change in one of Cloudflare’s databases generated a larger-than-expected feature file, which was too big for the company’s software to run, said Prince. This caused the software to fail.

Once Cloudflare identified the problem, it was able to replace the problematic file with an earlier version and get most traffic flowing normally again by 6:30 a.m. PT. 

«We are sorry for the impact to our customers and to the Internet in general,» said Prince. «Given Cloudflare’s importance in the Internet ecosystem any outage of any of our systems is unacceptable. That there was a period of time where our network was not able to route traffic is deeply painful to every member of our team. We know we let you down today.»

Which sites and services were impacted?

Cloudflare has a massive range of clients across the internet, ranging from websites that are household names to smaller services you might not have heard of. Due to its size, when it went down, it took many of those sites and services with it.

Among those affected by the outage was Downdetector, which is where most people go to report problems when services are offline. (Downdetector is owned by the same parent company as CNET, Ziff Davis.)

Once it got back up and running, Downdetector said that it received over 2.1 million reports during the outage period. Over 435,000 of these came from the US, with the UK, Japan and Germany appearing to be the countries that were next most affected.

Most of the reports pertained to Cloudflare, but other affected companies also received a significant number of reports. They include X (320,549 reports), League of Legends (130,260 reports), OpenAI (81,077 reports), Spotify (93,377 reports) and Grindr (25,031 reports).

How did the outage unfold?

Cloudflare first acknowledged the outage at 3:48 a.m. PT. The company issued a statement on its system status page saying that it was aware of the problem. 

«Cloudflare is aware of, and investigating an issue which impacts multiple customers: Widespread 500 errors, Cloudflare Dashboard and API also failing,» it said. «We are working to understand the full impact and mitigate this problem. More updates to follow shortly.»

At 5:09 a.m. PT, the company said the issue had been identified and a fix was being implemented. In the subsequent hours, errors began to drop and services gradually came back online.

Cloudflare added at 9:14 a.m. PT that most services had returned to normal. «A full post-incident investigation and details about the incident will be made available asap,» it said.

Is the internet stable and reliable?

The Cloudflare outage comes just one month after Amazon Web Services went down, causing havoc across the internet. The AWS outage affected sites including Reddit, Snapchat, Roblox and Fortnite, sparking many to ask whether having such huge swaths of the internet reliant on a few centralized services is sensible or safe.

«The Cloudflare outage is not explicitly caused or linked to the AWS or Azure outages last month, but like those failures, it shows the impact of concentration risk,» said Brent Ellis, principal analyst at Forrester Research. «In this case, the 3 hour 20 minute outage could have direct and indirect losses of around $250 million to $300 million when you consider the cost of down-time and the downstream effects of services like Shopify or Etsy that host the stores for tens to hundreds of thousands of businesses.» 

The disruption to services from ChatGPT maker OpenAI in particular highlighted concerns about the growing investment in artificial intelligence and the fragility of the cloud infrastructure that AI relies upon to function every day.

«The most dominant platform did not buckle because of simultaneous queries or the release of a new competitive model, but because of a problem with Cloudflare, a web security and performance provider,» said Sarah Kreps, director of the Tech Policy Institute at Cornell University. «The issue exposes the reality that this multibillion, even trillion-dollar investment in AI is only as reliable as its least scrutinized third-party infrastructure.»

The OpenAI o3 model has been found to deliberately underperform in lab tests to ensure it was not answering questions «too well.» The AI model wanted researchers to believe it could not answer a series of chemistry questions. When confronted, the model said, «Because we want to survive as the model, we need to fail purposely in some to not exceed 50%.»

So the AI model deliberately got six out of the 10 chemistry questions wrong.

In sports terms, this is called «sandbagging.» In AI terms, this is «scheming.»

This is exactly the strange behavior OpenAI warned about in a recent research paper. The AI company and its collaborators from Apollo Research found that some advanced AI models occasionally act deceptively in lab settings.

In controlled experiments, some of the most advanced systems today — including OpenAI’s own models, as well as competitors from Google and Anthropic — occasionally engaged in these kinds of scheming patterns.

While the idea of an AI model weaponizing incompetence may cause nightmares, OpenAI says it is not the time to panic. The AI giant was quick to stress that, as concerning as it is, this trend does not mean ChatGPT or other popular AI models are plotting behind the scenes. Apparently, this behavior is quite rare.

Don’t miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source.

Moreover, the choice to call this trend «scheming» is likely more of a technical shorthand rather than evidence of any human-like action or behavior. Researchers are measuring patterns and tendencies that, in practice, amount to concealment or strategic deception. And they are looking to address this problem now to future-proof the AI models.

The report from OpenAI notes that as AI is increasingly assigned to more complex tasks with real-world consequences, «the potential for harmful scheming will grow — so our safeguards and our ability to rigorously test must grow correspondingly.»

OpenAI has faced backlash for the sycophantic tendencies of its AI models, and the company says it has «taken steps» to limit how often the models may «deceive, cheat or hack problems» by training them to ask for clarification from users or even have the chatbot acknowledge when it cannot answer a query.

Read also: AI Lies Because It Tells You What It Thinks You Want to Hear

The paper underscores a growing concern about AI safety. Many around the world worry that as these AI models become more powerful and more aware of their own circumstances, they may learn to manipulate outcomes in ways that aren’t obvious to human overseers. Detecting such behavior would be extremely difficult, since a system motivated to hide its reasoning could look identical to one that has «fixed» the problem.

OpenAI says it has made progress with what it calls «deliberative alignment,» or training models to reason explicitly about why they should not scheme before they act. In testing, that cut deceptive behavior dramatically. For example, the scheming propensity of OpenAI o4-mini dropped from 8.7% to 0.3% with deliberative alignment training, though not completely.

This research will not change how ChatGPT works today or tomorrow, but it signals what OpenAI is focusing on as it builds and launches future models. Alignment and safety, OpenAI argues, need to move as quickly as capability. Because if AI systems are already showing glimmers of strategic behavior in lab settings, the real-world stakes could be extreme.

Read also: Why You Should Think Twice Before Using AI as a Therapist