It’s natural to ask how the heck this could have happened, and how we can make sure it never does again. But another question needs to come first: How much might we be willing to sacrifice to protect against the risk?
At the moment, this might seem absurd. To people stranded by canceled flights or missing doctor’s appointments, it’s worth spending almost anything to avert the chaos that has beset airlines, hospitals, first responders and other critical services. But this is like saying it’s worth insuring your car to the absolute maximum — after you’ve gotten into a big accident. Many car owners decide not to buy the most generous possible coverage because that coverage also carries higher premiums.
So, too, with maximally insuring against meltdowns like the CrowdStrike mess, because doing so would require paying a price in economic efficiency. So we have to decide what trade-offs we’re willing to make.
Averting this particular disaster might not have cost a ton of money, to be clear. But CrowdStrike is only one of approximately one zillion points of possible failure in our thoroughly networked and globalized economy. Over the past fifty years, the market’s relentless drive for efficiency and reach has made such mass failure nodes more numerous, more potentially catastrophic and harder to see before they fail — while also giving us instant access to all the world’s culture and most of its information, plus more, cheaper and better goods and services, and a global economy that every year lifts tens of millions more people out of poverty.
I came of age in the 1990s, when this transformation was still in its comparatively early days. The internet was just beginning, and global trade was shifting manufacturing from the United States to countries that had cost advantages (such as China and Mexico) and countries that had special technical expertise (such as Germany and Japan).
At the same time, Americans were importing foreign techniques such as “just-in-time” manufacturing, which replaced inefficient local suppliers and mountains of spare parts with far-flung but streamlined supply chains that delivered inputs precisely when they were needed. This was often hard on people who worked for the local industries. But it was a boon to workers in poor countries and to American consumers. Compare any consumer product today with its 1990 equivalent, and you’ll often find that either the quality has risen dramatically (cars) or the price has fallen precipitously (clothes). In the case of televisions and many other products, it’s both.
The internet intensified this trend, because its network effects and economies of scale often drive markets toward a handful of players. This newspaper, for example, used to be one of thousands of local newspapers, each competing for readership within a geographically limited market. Now we’re vying to be among a handful of daily news publications that serve the entire country, maybe even the entire English-speaking world.
Enlarging markets, and putting more eggs in fewer baskets has real benefits: A few big news publications can cover more beats, more deeply, than a lot of small ones can. But it also has real downsides, such as a higher risk that those few players will all miss some stories — notably, local news — or get one of them badly wrong. Similarly, as became obvious during the pandemic, cost-efficient just-in-time global supply chains can leave companies and their customers vulnerable when borders slam shut and governments hoard critical resources for their own citizens.
In the case of software companies, it’s quite efficient for one firm to serve a large number of important customers, as CrowdStrike does — or even practically all the customers, as is the case with online search. In some ways, these concentrated players might provide greater reliability, because they develop a lot of expertise by serving many users, and they can invest more in R&D and security than Bob’s Friendly Local Software Co. can. But when outages happen, they happen to seemingly everyone, everywhere, all at once, leaving users no alternatives.
How best to try to manage the trade-off between efficiency and redundancy is a hard question for another day. For the moment, the important thing is to recognize that it exists, and that there’s no easy way around it. We probably should have thought more about such trade-offs when the Great Efficiency Drive was underway. We’ll have to think even harder about them now.
Read More
