The problem traced to an update issued for Windows computers by security firm CrowdStrike, the company said in a blog post. The company said at 6 a.m. Friday the problem had been identified and fixed, but by that point the effects had rippled around the world.
The effects of the outage were a reminder of how the global economy is dependent on computer systems that are vulnerable to everything from attacks by sophisticated hackers to bungled software updates. But while the problems were widespread, there was an element of randomness at play, too, depending on which companies in any given industry made use of the defective system.
GET CAUGHT UP
Stories to keep you informed
CrowdStrike chief executive George Kurtz said he was “deeply sorry” to anyone affected by the global outage during an appearance Friday on NBC’s “Today” show and vowed to “make sure every customer is fully recovered.”
Many CrowdStrike systems are recovering and will soon be operational, Kurtz said, but “it could take some time for some systems that won’t automatically recover.”
The company’s problems follow a string of computer security incidents and service outages in recent years that have disrupted online services. Bruce Schneier, a security technologist who teaches at the Harvard Kennedy School, said the latest problem shows how brittle parts of the online world have become as companies have chased efficiency while sacrificing resiliency.
“This is one of hundreds of companies you’ve never heard of that are essential to the functioning internet,” Schneier said. He compared the situation to a house built in such a way that nailing a picture to the wall puts it at risk of collapsing.
CrowdStrike’s software is widely used around the world by businesses seeking to defend themselves from hackers. The problems Friday affected only computers running Microsoft’s Windows operating system, CrowdStrike said, leaving Apple computers and those using Linux unaffected.
While Windows is the operating system for hundreds of millions of personal PCs, it also runs on the computers behind the scenes that are vital to the operation of airlines, digital payment systems, emergency services call centers and other organizations.
Microsoft chief executive Satya Nadella said in a post on X that his company was working with CrowdStrike and Windows users “to provide customers technical guidance and support to safely bring their systems back online.”
CrowdStrike said in the blog post that the faulty file implicated in the more widespread outages had a timestamp equivalent to 12:09 a.m. Eastern time Friday. An unrelated Microsoft outage Thursday also caused problems.
Some of the most visible early effects of the outage were at airlines. Budget U.S. carriers Frontier and Sun Country disclosed in the early hours of Friday that they were experiencing disruptions, and the problems spread to major airlines and affected airlines overseas.
By 6:30 p.m. more than 4,400 flights had been canceled worldwide. Of that, more than 2,700 were flights operating into, within or out of the United States, according to FlightAware.com, an online flight-tracking website. More than 40,000 flights were delayed worldwide, with delays affecting more than 10,000 flights originating or ending in the United States.
United Airlines said the outage impacted several key systems: those used for checking in customers, calculating aircraft weight and operating call centers.
In the United States alone the impacts were similar to what might be expected during a major snowstorm. And even as airlines reported getting operations back up and running Friday morning, problems in the aviation system can take hours to fully resolve as crews and aircraft are left in the wrong cities.
John Cox and his wife found themselves spending more than a day at Reagan National Airport as they tried to get a flight home to North Carolina.
A technical issue Thursday evening prompted major airlines including Delta to delay service and reroute passengers. Then their 10:17 a.m. Friday flight to Charlotte was pushed to Saturday at noon. “It completely derailed our plans,” Cox said, slumped on a seat by the entrance of the airport.
“We’re going to end up spending a lot more money hanging around in D.C. than if we were back at home,” Cox said, before reclining in his seat and sighing. “There should be some compensation for the immense inconvenience that so many people are enduring.”
By Friday morning it had become clear that the effects were reaching a large number of industries and affecting government agencies.
The effects on medical providers stretched from the United States to Israel, with doctors losing access to electronic medical records and some hospitals postponing elective procedures.
Mass General Brigham, a nonprofit that operates one of the largest hospital systems in Massachusetts, said on its website that it was canceling all “non-urgent visits” at its hospitals and clinics Friday. The organization said its emergency departments remained open and were providing care for urgent appointments and procedures.
Alison Baulos said her father, Gary Baulos, was scheduled to have heart surgery Friday at Baptist Health Hospital in Paducah, Ky., but got a call at 3:30 a.m. that the procedure was canceled due to the outage.
Her father, a 73-year-old former Navy Seabee and AT&T technician, had just been diagnosed with eight blockages and an aneurysm, she said, and had spent Thursday night at a hotel close to the hospital.
“The doctor said he can’t be too active, he can’t raise his hands above his head,” Alison Baulos said. She said her father is taking it in stride. His reaction to the cancellation, she said, was, “all right, since we’re already out here I guess we can go to IHOP for breakfast.”
Danielle Carzell, a 35-year-old Atlanta resident, was scheduled to have her hip replaced Friday, a procedure she’s eagerly anticipated for months. Born with a genetic condition — her hips are “like mismatched puzzle pieces,” she said — she’s in constant pain and hoped the surgery would relieve it. But after waiting for hours at Emory St. Joseph’s Hospital, she was told the procedure would have to be rescheduled due to the outage.
“I have no idea when I’m going to be able to get my surgery now,” she said. “I have a 5-year-old that I just want to play with, and this outage is just ruining everything.”
Even in a sector that has suffered debilitating blows this year — notably the hacks of claims processor Change Healthcare and health system Ascension — experts were surprised by the scale of the impacts Friday.
“I was stunned to see this kind of ripple effect,” said Chris Cummiskey, a cybersecurity expert and former Department of Homeland Security official. While CrowdStrike is often seen as the gold standard in cyber protection, he said this event may call for reviewing their internal practices.
“You don’t want to hit one button and have this kind of worldwide meltdown occur,” Cummiskey said.
Television news stations in several countries were knocked off the air. In Australia, the outage left ABC News unable to deliver its usual evening news update. Instead it showed a special report on the disruption at Sydney airports.
Some emergency services also reported problems. The Alaska State Troopers reported that, starting at 9 p.m. Thursday local time, 911 call centers statewide were not working correctly. Service was restored around 4 a.m. Friday when dispatch centers switched to analog phone systems or partnered with dispatch centers that had not been impacted, according to the Alaska Department of Public Safety.
Federal officials scrambled to assess the situation and provide assistance. Anne Neuberger, a security and technology adviser to President Biden, said her day began on a call with the White House situation room at 4 a.m. as the government sought to determine the effects on its own systems and critical parts of the economy.
“It highlights both the degree to which our economies, our national security are now digital and interconnected in a fundamental way,” said Neuberger, speaking on a panel at the Aspen Security Forum in Colorado.
A White House official said that Biden himself was briefed on the outage and that the administration was in touch with CrowdStrike and affected entities. The White House will get “sector by sector updates throughout the day and is standing by to provide assistance as needed,” the official added.
Transportation Secretary Pete Buttigieg said in an interview on CNBC Friday that he expected airlines, ports and freight companies to be largely back on their feet by Friday. But he said the episode should prompt some soul searching about the vulnerability of the country to even seemingly minor technical problems.
“We’re certainly in a new era in terms of these risks,” Buttigieg said. “I think there will be a huge amount of after action assessment, after the dust settles and things get back to normal.”
Hannah Ziegler, Jeff Stein, Aaron Gregg, Adela Suliman and Julian Mark contributed to this report.
Read More
