The SolarWinds megahack underscores what security mavens have been warning about for years: The software supply chain is complex, vulnerable, somewhat invisible and insufficiently protected.
For example, on Dec. 2, 2020, eleven days before the government’s announcement that it had been hacked, we quoted sources warning that the software supply chain is extremely vulnerable to cyberattacks, primarily because of the many links in the chain that are potentially invisible or unknown to design engineers. Our sources said attacks are especially likely during and after firmware updates, which is precisely how the SolarWinds hack occurred: during updates of Orion software that was trojan-ized to deliver malware.
Weaknesses in the software supply chain were a major finding of Trend Micro’s report, Attacks on Smart Manufacturing Systems, published a year ago (and mentioned in our December story). The report analyzed the security of simulated goods production in Italy’s Industry 4.0 Lab.
The SolarWinds hack highlights the vulnerabilities and lack of visibility into what could be called the external supply chain—stuff used by companies but designed elsewhere. Implementation in a company workflow might include writing code or at least patching known vulnerabilities.
The mirror image could be dubbed the internal—others call it the embedded component, supply chain—code used in designing and developing products. It’s a useful distinction when thinking about the different processes to use in securing these devices and their software. So far, more attention has been paid to the dangers lurking in third-party device software than software as it is being developed.
Even a relatively isolated smart manufacturing system likely contains industrial IoT devices custom-designed by internal employees and external consultants, including custom-designed software that includes third-party components. All this creates weak security links in what is often a complex supply chain. (Source: Trend Micro)
How bad the threat?
Pretty bad, both in terms of what can happen, determining how it happened, what to do about it and preventing it from happening again.
Aside from highly-publicized hacks, downright scary supply-chain hazards are everywhere. For instance, a zombie-style Trojan was discovered in April that opens browser windows, lets in additional malicious apps and sends texts to spread malware, yet can’t be destroyed even after a factory reset. It was injected into Android smartphones during a software update by phone maker Gigaset’s server. The malware can only be uninstalled via a complicated procedure.
Also in April, the National Counterintelligence and Security Center reported foreign hackers are increasingly attacking suppliers to the federal government, compromising their products in order to spy and steal intellectual property.
A March report from Microsoft found that 80% of businesses have experienced at least one firmware attack in the last two years, but only 30% have budgeted for firmware protection.
Earlier this year, Nozomi Networks’ OT/IoT Security Report called software supply chain vulnerability research and exploitation “the most important trend of recent months” in cybersecurity. For external supply chain threats, “This attack highlights the risks to end users who have limited agency over the software or services used within their networks, yet are directly impacted by their compromise.”
Software components can be relatively invisible because code is created “by stacking up one component after the other until you have a deliverable,” Ivan Speziale, security researcher with Nozomi Networks Labs, told EE Times . “Each of those components may be made of others, so once you get this deliverable, it’s hard to understand what’s hidden under all these layers or components.
“You must try to understand how these layers are composed, what’s inside them, who’s responsible for them, and you must repeat this process for each and every service,” Speziale added.
Detailed analysis is especially critical given the prevalence of vulnerabilities hackers exploit. The Verve Industrial Protection ICS Advisory Report, released in February, analyzed 248 security advisories issued in 2020 by ICS-CERT. Industrial control system (ICS) vulnerabilities were evenly split between OEM application software and embedded devices. About 15% of that total affected the supply chain.
Of supply chain-related advisories, the report said: “This is an underlying threat to legacy devices with hidden vulnerabilities. The reality is that these underlying components—vulnerabilities relating to CodeMeter, or InstallShield—are present in many devices and the impact cannot even begin to be measured in the number of downloads, asset owners affected, or any useful metric.”
Moreover, not all vendors are reporting vulnerabilities, and those within projects hosted on public repositories such as GitHub or FOSS may not be reported at all, according to the study. And much of ICS equipment consists of legacy machines, running old components that may no longer be updated.
All this makes it very difficult to detect supply-chain vulnerabilities in an organization’s ICS devices.
A month after the Verve Industrial report surfaced, hackers attacked the official PHP repository on GitHub, adding remote code execution backdoors to PHP source code, according to Bleeping Computer . Although the changes were caught and reverted in time, “The incident is alarming considering PHP remains the server-side programming language to power over 79% of the websites on the Internet,” said the article.
According to Speziale, “Many organizations knew they had SolarWinds’ software, but not how far it reached into their network, how privileged its use was or how useful it could be for an attacker.”
Industry, government response
In response to recent hacks, a White House executive order was finalized on May 12. It covers federal agencies only. Nevertheless, some say the order’s effects will cascade throughout industry, since federal agencies are an enormous buyers of technology goods and services, as we’ve discussed.
The order calls for improvements in federal agencies’ cybersecurity practices, as well as in how business is done with software suppliers. Steps include: the use of multi-factor authorization, cloud computing and zero-trust principles in networks and infrastructure; mandatory breach reporting; requiring suppliers provide software bills-of-materials (SBOMs) in the purchasing process; and establishing a Cybersecurity Safety Review Board to analyze major hacks and recommend post-attack actions.
Separately, the U.S. Department of Homeland Security took steps to improve supply chain security. Announced at the RSA Conference 2021 last month, DHS’ Cybersecurity and Infrastructure Security Agency (CISA) proposed a program for fixing firmware vulnerabilities among “worst offender” vendors in critical infrastructure devices.
Actions include requiring SBOMs from vendors and requesting vendors provide the intent of software components and disclose what, if any, vulnerability mitigation techniques are included in the software. Also included: developing a specification for verifying vendor code claims; developing a system of public risk scoring; and promoting policies that encourage reducing purchases of products that don’t measure up.
Boyden Rohner, CISA’s associate director, and Thomas Ruoff, methodology branch chief, said they want to integrate those controls with those in the executive order, according to SC Magazine .
Rohner and Ruoff said CISA will begin determining firmware risks in critical infrastructure sectors through discussions with stakeholders. The initial focus on firmware vulnerabilities would probably focus on “purpose-built, self-contained products” such as programmable logic controllers (PLCs).
Other federal actions include the formation by the House Armed Services Committee of a task force to investigate vulnerabilities in the defense supply chain, among other concerns.
Industry hasn’t been ignoring the problem, either. In March, for example, the Linux Foundation and several partners created a free cryptographic software signing service, Sigstore, to improve the security of open-source programs.
In April, the Automated Source Code Quality Measures addressing software reliability, security, performance efficiency and maintainability became an ISO standard: ISO/IEC 5055:2021. The measures were originally developed by sponsors of the Consortium for Information & Software Quality.
Speziale said, “There’s no single solution to make the software supply chain problem go away. It’s a process. When we use third-party services and software we’re taking on a certain amount of risk.”
What also counts in reducing risk is how third-party services and software are deployed. “For example, a PLC in a network only needs to access a segment of that network, so we should attach it only to that segment,” he said.
“It shouldn’t be talking to a workstation located somewhere else.”
Chris Grove, Nozomi Networks’ technology evangelist, put the subject in a larger perspective. “SolarWinds isn’t just about supply chain attacks becoming more prevalent—that was the opportunity the attacker used in this particular case,” he said.
“It has more to do with the actions of adversaries and their willingness to be more aggressive with their approach that we didn’t see in the past. Once an attack trail has been blazed, there will be secondary attackers taking advantage of the opportunity—other attackers now see that supply chain attacks are possible.”
>> This article was originally published on our sister site, EE Times Europe.
- TCP/IP stacks vulnerabilities are a wake-up call for embedded software
- Device security: when ‘how’ becomes just as important as ‘what’
- Understanding NIST Framework security controls
- Designing security into the industrial IoT
- IoT Security – Physical and hardware security
- Automating IoT security threat/risk analysis and compliance
For more Embedded, subscribe to Embedded’s weekly email newsletter.