The Cloudflare outage on Tuesday that disrupted access to many websites and services — including OpenAI, Spotify, X, Grindr, Letterboxd and Canva — was the company’s worst outage since 2019, CEO Matthew Prince says.

Other disruptions have centered on specific network features, Prince wrote in a blog post. «But in the last 6+ years we’ve not had another outage that has caused the majority of core traffic to stop flowing through our network.»

Cloudflare is a cloud services and cybersecurity company based in San Francisco that is used by approximately 20% of all websites, according to W3Techs. It’s one of a handful of services, along with Amazon Web Services, CrowdStrike and Fastly (all of which have experienced major outages in the past few years) that you might never have heard of, but that provide essential internet infrastructure.

The bulk of sites and services impacted by Tuesday’s outage, which began around 3:30 a.m. PT, seemed to recover within just over three hours. By the end of the day, everything had returned to normal, and Cloudflare set about explaining what went wrong. Here’s what you need to know.

Don’t miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source.

What caused the Cloudflare outage?

Cloudflare was keen to emphasize that the outage was not caused either directly or indirectly by a cyberattack. At first the company did suspect it might have originated from a «hyper-scale DDoS attack,» Prince said in his blog post. But it turned out that the outage resulted from an internal software failure.

A change in one of Cloudflare’s databases generated a larger-than-expected feature file, which was too big for the company’s software to run, said Prince. This caused the software to fail.

Once Cloudflare identified the problem, it was able to replace the problematic file with an earlier version and get most traffic flowing normally again by 6:30 a.m. PT. 

«We are sorry for the impact to our customers and to the Internet in general,» said Prince. «Given Cloudflare’s importance in the Internet ecosystem any outage of any of our systems is unacceptable. That there was a period of time where our network was not able to route traffic is deeply painful to every member of our team. We know we let you down today.»

Which sites and services were impacted?

Cloudflare has a massive range of clients across the internet, ranging from websites that are household names to smaller services you might not have heard of. Due to its size, when it went down, it took many of those sites and services with it.

Among those affected by the outage was Downdetector, which is where most people go to report problems when services are offline. (Downdetector is owned by the same parent company as CNET, Ziff Davis.)

Once it got back up and running, Downdetector said that it received over 2.1 million reports during the outage period. Over 435,000 of these came from the US, with the UK, Japan and Germany appearing to be the countries that were next most affected.

Most of the reports pertained to Cloudflare, but other affected companies also received a significant number of reports. They include X (320,549 reports), League of Legends (130,260 reports), OpenAI (81,077 reports), Spotify (93,377 reports) and Grindr (25,031 reports).

How did the outage unfold?

Cloudflare first acknowledged the outage at 3:48 a.m. PT. The company issued a statement on its system status page saying that it was aware of the problem. 

«Cloudflare is aware of, and investigating an issue which impacts multiple customers: Widespread 500 errors, Cloudflare Dashboard and API also failing,» it said. «We are working to understand the full impact and mitigate this problem. More updates to follow shortly.»

At 5:09 a.m. PT, the company said the issue had been identified and a fix was being implemented. In the subsequent hours, errors began to drop and services gradually came back online.

Cloudflare added at 9:14 a.m. PT that most services had returned to normal. «A full post-incident investigation and details about the incident will be made available asap,» it said.

Is the internet stable and reliable?

The Cloudflare outage comes just one month after Amazon Web Services went down, causing havoc across the internet. The AWS outage affected sites including Reddit, Snapchat, Roblox and Fortnite, sparking many to ask whether having such huge swaths of the internet reliant on a few centralized services is sensible or safe.

«The Cloudflare outage is not explicitly caused or linked to the AWS or Azure outages last month, but like those failures, it shows the impact of concentration risk,» said Brent Ellis, principal analyst at Forrester Research. «In this case, the 3 hour 20 minute outage could have direct and indirect losses of around $250 million to $300 million when you consider the cost of down-time and the downstream effects of services like Shopify or Etsy that host the stores for tens to hundreds of thousands of businesses.» 

The disruption to services from ChatGPT maker OpenAI in particular highlighted concerns about the growing investment in artificial intelligence and the fragility of the cloud infrastructure that AI relies upon to function every day.

«The most dominant platform did not buckle because of simultaneous queries or the release of a new competitive model, but because of a problem with Cloudflare, a web security and performance provider,» said Sarah Kreps, director of the Tech Policy Institute at Cornell University. «The issue exposes the reality that this multibillion, even trillion-dollar investment in AI is only as reliable as its least scrutinized third-party infrastructure.»