SUBSCRIBER+ EXCLUSIVE REPORTING - As Chinese government hackers have burrowed into U.S. critical infrastructure, they have avoided detection by targeting low-profile devices: obsolete internet routers long ago forgotten in office closets and home basements.
The Department of Justice says that in a recent cyberattack known as “Volt Typhoon,” Chinese hackers were seeking ways to sabotage vital equipment, and did so by breaching outdated U.S. routers at American businesses. Investigators said the hackers used those routers’ IP addresses to disguise intrusions into far more sensitive critical infrastructure networks. The operation has alarmed U.S. intelligence officials because it suggests that in the event of a U.S.-China war, Beijing could use this access to cripple American power plants, hospitals and communications networks. But Volt Typhoon is only the latest high-profile reminder of the dangers posed by hardware and software that have reached the end of their supported lifespans - meaning they no longer receive security updates from manufacturers.
Of all the cultural and technical weaknesses that hackers routinely exploit, experts say the failure of American citizens and businesses to keep track of their technology’s lifespans — and to update their equipment while they still can — ranks as one of the most dangerous and least appreciated threats to the U.S. As the Chinese cyberattacks have demonstrated, digital intruders can achieve dramatic results by targeting obsolete devices with flaws that their manufacturers no longer bother fixing.
“This is something that's actually being targeted all the time, and we need to take it seriously and … make sure that people understand it,” said Ari Schwartz, the managing director of cybersecurity services at the Venable law firm and the coordinator of the Center for Cybersecurity Policy and Law.
In July 2023, Schwartz and several colleagues worked with a group of leading networking equipment vendors and some of their biggest customers to launch the Network Resilience Coalition, in an effort to raise awareness of the threat and search for solutions. The focus is on what’s known as the “end-of-life” problem — not for human beings, in this case, but for tech products — and finding ways to close security loopholes before disaster strikes.
The “end-of-life” dilemma
Roughly two years ago, critical infrastructure operators — executives at companies supplying vital services such as IT, electricity, water and telecommunications — began observing a marked change in how hackers were breaching their computer networks.
Time after time, when the U.S. Cybersecurity and Infrastructure Security Agency (CISA) or the U.K. National Cyber Security Center (NCSC) would alert telecom companies and other infrastructure owners that hackers had infiltrated their systems, they found that the initial intrusion points involved obsolete, unsupported technology.
“We were seeing a major uptick in attacks on end-of-life hardware and software,” Schwartz told The Cipher Brief. “That really alarmed those that were getting notified.”
Government and industry cyber experts agree that China in particular remains laser-focused on breaching obsolete routers as a springboard to more dangerous intrusions. One firm, SecurityScorecard, recently monitored more than 1,100 obsolete Cisco routers over a one-month period and watched as Chinese hackers breached 30 percent of them.
End-of-life devices represent a serious vulnerability for many industries, as China’s Volt Typhoon campaign made clear. Security for these devices is weak — not only because they are old and more likely to accumulate vulnerabilities, but also because their owners often fail to apply necessary patches or upgrades. In some cases such updates are no longer available.
Telecom companies are among the firms most worried about Chinese hackers’ focus on end-of-life equipment, given the sector’s heavy reliance on networking devices and software. This helps explain the current telecom-heavy makeup of the Network Resilience Coalition.. But NRC members say all forms of critical infrastructure are vulnerable, and that both vendors and customers must act to lessen the risks.
“We need to have honest conversations about the responsibilities of all the different parties in this issue,” Matt Fussa, the chief trust officer at networking giant and NRC member Cisco, told The Cipher Brief.
Who’s responsible for what?
Solving the end-of-life security problems means wading into a sensitive debate over patching and answering a basic question: Who is ultimately responsible for a product’s security, especially in the last years of a product’s “life”?
Vendors are often critical of customers — businesses and individuals alike — who fail to quickly apply new software patches, arguing that their negligence is opening the door to cyberattacks that quickly metastasize across entire industries. For their part, customers sometimes say that patching requires too much manual work or technical know-how.
“There's lots of reasons [why] people don’t patch or don’t take [obsolete] stuff offline,” Schwartz said.
Even after a product reaches the end of its supported lifespan, it may still be eligible for patches that the vendor releases for more modern products; it just won’t receive those patches automatically. Instead, companies and individuals have to hunt for the right file to download and install. Vendors often bemoan the fact that so few customers bother to do this work for obsolete products they continue to use.
Whichever side is to blame, Schwartz said he has worked with companies that were “hundreds of thousands of patches behind” — and every one of those presents an inviting target for would-be hackers. With the Chinese hacks, Fussa said, “it felt to us like attackers were able to exploit that gap” between what vendors and customers each consider their roles.
NRC leaders say they are trying to mediate disagreements between vendors and customers; Schwartz said he has heard “valid complaints from both sides.” Customers gripe about how vendors’ product longevity claims compare to their support windows, while vendors grouse that customers are giving up on patching too quickly when official support ends, not realizing that they can still manually apply some updates. Schwartz and others are quick to acknowledge that there are no simple answers.
“There’s a feeling that there's a cost to patching,” Schwartz said, “but there's a cost to not patching, too … Understanding what those risks are, and the costs on both sides of it, is really key.”
Beyond patching, Schwartz and others said, customers and vendors share responsibility when it comes to all kinds of planning for “end-of-life” issues — in particular, how to maintain a product and knowing when to replace it.
Vendors “do have some responsibility [for] helping their customers understand the lifecycle better than they have in the past,” Schwartz said. “But it's also the responsibility of the users to have a plan in place of when they're going to update things, and how they're going to do it. … We don't expect any product manufacturer to maintain any product forever under all circumstances.”
Kathryn Condello, the senior director for national security and emergency preparedness at NRC member Lumen Technologies, says infrastructure operators like hers must design their systems so that components can be swapped out as they reach their end-of-life dates. But that can be a tough pill to swallow, particularly for companies that are used to leaving core systems mostly untouched for decades.
“There's now a sense of, just because you bought it and invested it in year one, doesn't mean you'll be able to amortize it over 50 years,” Condello said. “Maybe at year seven, you're going to have to redo equipment. … You start building your product development lifecycle in a different way than you did before.”
Closing the loopholes
In late January, the NRC published its initial report on end-of-life planning and related security issues for tech products, along with a series of security-related recommendations for vendors and customers.
The coalition urged vendors to provide “firmly defined” end-of-life dates and other information to customers, distinguish between critical patches and less essential updates, and to follow the Biden administration’s “secure by design” guidance for incorporating cybersecurity considerations into all phases of product development. For customers, the report suggested favoring vendors that follow its recommendations, paying attention to vendors’ security notifications, and applying extra security scrutiny to products that are approaching their end-of-life dates.
To help vendors and customers accomplish some of these goals, the NRC says it is studying ways to automate the sharing of end-of-life details and information about important patches. Despite all the recent advances in artificial intelligence, machine learning, and other automation technologies, there have been only tepid efforts to automatically distribute this particular kind of product information. Remarkably, much of the data that customers need to determine when their product will “age out” of support isn’t currently shared with those customers on a rapid basis.
The report specifically highlights one project, the Open Product Lifecycle Framework, as a promising example of automated information sharing. The framework has quickly garnered broad support across several industries.
Vendors “need to do a better job pushing the relevant information to that network manager,” said Fussa. One example he cited: providing better context about new security patches so that customers know which ones to apply first. Many vendors, he said, aren’t adequately harnessing machine learning to automatically analyze data and share insights with customers.
The coalition also wants policymakers and businesses to pay more attention to end-of-life concerns generally. Members “want to see the idea of unsupported and end-of-life stuff being taken seriously,” Schwartz said. “It was targeted by the Chinese, it has been targeted by the Russians, but yet, when [federal agencies] come up with these [security warnings], there's a lot more on other issues that probably are not as frequent targets.”
Some vendors have brought their own ideas to the NRC. Fussa said the government could force critical infrastructure companies to replace obsolete routers by a certain date — and then offer them grants to buy new equipment.
But first, Fussa said, the NRC should convene technical experts from government and industry to make sure they see eye to eye. “Until we understand what the right approaches are by bringing our tech leaders into these discussions,” he said, “there's a risk that we'll jump into policymaking mode with all the wrong solutions.”
Finally, there’s the general public. “I don't know yet if the average American citizen realizes that it might be a good idea to trade in your end-of-life Best Buy router for maybe something newer,” Condello said.
That’s a lesson that the average American and many American companies need to learn — and fast — if the gaping holes in the U.S.’s critical infrastructure are to be closed.
Read more expert-driven national security insights, perspective and analysis in The Cipher Brief
