Once again, the digital economy was caught off guard when Amazon Web Services suffered its second major outage this year on October 20, crippling exchanges such as Coinbase and Robinhood and analytics service Coinmarketcap. This was followed by a second, smaller outage just 10 days later.
According to Amazon’s initial report, the October 20 outage was caused by a malfunction that affected one of its internal subsystems that manages its domain name service, causing connectivity issues across multiple services. It was the result of a faulty update and ended up deleting Amazon’s critical US East-1 region. It is a huge server center that powers many of the country’s main Internet services. For two hours, numerous commercial platforms, streaming services, payment providers and gaming networks were inaccessible to users around the world.
Amazon engineers were no doubt working overtime to try to fix the outage, and to the company’s credit, the vast majority of services that reported problems were back online within a couple of hours. But the incident once again highlights the dangers of relying on centralized infrastructure, and came just months after a similar outage in Amazon’s eu-north-1 region. Going offline causes pain in almost every company, but for the crypto industry, where billions of dollars are traded every hour, such events are unacceptable.
Incalculable losses for merchants
Although it is quite rare for centralized cloud platforms like Amazon to go down, it does happen from time to time. And when this happens, the impact is often monumental, affecting millions, if not billions, of people around the world. A case in point is that just six months earlier, Amazon suffered a similar outage, taking down two of the world’s largest crypto platforms, Binance and Kucoin, for several hours. Amazon isn’t alone in this either, as rival clouds like Google and Microsoft Azure have suffered crippling outages of their own. In fact, Azure was down for several hours on October 29, with reports suggesting it took numerous websites and online services offline.
The problem with centralized infrastructure is that, well, it’s centralized. The weakness of these platforms is that they introduce single points of failure, due to the way they depend on critical components that, if taken offline, cause the entire system to collapse. It could be as simple as something like a computer server or database that contains vital configuration parameters, or a solitary network connection that lacks redundancy. These vulnerabilities exist in all clouds and, no matter how diligent operators are, they will always present a risk.
Coinbase was one of the first services to report problems in the wake of the Amazon mishap, and quickly stepped up to assure its users that their funds are safe and secure. But this clarity does not address the underlying problems of frozen trades and delayed market orders, which occur when systems go offline without warning. The longer the delay, the more the price of an asset could move, meaning the trader cannot profit from this. They may even lose money if the price of an asset falls shortly after they have entered a position and they are unable to sell it.
While it is impossible to calculate the exact impact, the paralysis inflicted on traders is likely to have caused them pain and financial losses.
It’s time to decentralize
One possible way around this is for crypto exchanges to at least partially shift to a more resilient decentralized infrastructure that eliminates these single points of failure. By operating some key modules of the trading system on a distributed network of servers, exchanges will virtually eliminate the possibility of such calamities occurring.
For an industry that prides itself on decentralization and constantly praises its benefits, relying so heavily on centralized and vulnerable cloud platforms for its own infrastructure seems hypocritical. While blockchain networks are spread across hundreds of nodes, very few exchange platforms can say the same and instead choose to host their entire infrastructure on one cloud provider or another.
Fortunately, Monday’s outage was not as serious as previous incidents, because Amazon had most services back up and running within a couple of hours, but it should still serve as a wake-up call for the crypto industry to get its act together. Decentralized cloud infrastructure still has teething issues related to latency, network coordination, and scalability, but is maturing quickly to support at least a hybrid cloud strategy. By distributing their data and systems across an expansive network, exchanges can become virtually immune to total outages caused by such an outage.
Centralized clouds will always have their place due to their immense scale, high performance, enterprise-grade security, and the specialized services they offer, which decentralized alternatives cannot match. They will likely remain the backbone of the Internet for many years, but they will never be able to replicate the resilience of decentralized alternatives. Since crypto exchanges control billions of dollars of customer funds in a market where every second counts, they must step up and ensure this episode does not happen again.
