SolarWinds breach highlights threats to the county technology supply chain

Key Takeaways

As many of you have heard, there was a significant national cyber security breach in December 2020 that affected federal and local government, as well as private industry. If you have heard the name SolarWinds, then you know by now that it is not referring to solar energy! But rather, a cyber breach on one of the nation’s largest cyber security monitoring solution providers. This breach was not ransomware and a demand for a large ransom. Nor was it an effort to damage an organization’s infrastructure or access to the internet. Rather, it was espionage and the theft of critical information that can later be used to jeopardize the security of our nation’s supply chain and defenses.

One may be thinking, “Well why is my technology supply chain important; and frankly, what is a technology supply chain?”

Before we dive into a review of the SolarWinds breach, let me further explain “supply chain.” According to the Cybersecurity and Infrastructure security Agency (CISA), a part of Department of Homeland Security (DHS), the technology supply chain is composed of hardware, software, and managed services from third-party vendors, suppliers, service providers, and contractors. If any component of this supply chain is exploited, the consequences can affect all users of that technology or service, including local government.

Learn More

For more information and guidance, you can read more here. And be sure to save the date for the NACo Spring CIO Forum to be held on March 31 and April 1, 2021. Registration will open soon. 

With that background in mind, the SolarWinds breach was only just exposed in late December. Another security provider that many counties use, FireEye, discovered the malware in their instance of the SolarWinds monitoring application (Orion) and subsequently notified SolarWinds. In turn, SolarWinds quickly notified their 18,000 customers and provided updated patches and remediation steps to help their customers determine if they were compromised. Those affected quickly discovered that remediation was not going to be a one-day task, but weeks of clean-up efforts.

Recognizing the seriousness of the breach and that it affected numerous federal agencies, the National Security Council staff stood up a task force construct known as the Cyber Unified Coordination Group (UCG), composed of the FBI, CISA, and, to coordinate the investigation and remediation of this significant cyber incident involving federal government networks. Fortunately, for county government, we have a strong working relationship with CISA. CISA has been holding national calls and providing a plethora of resources (located at the end of this blog) to local government to ascertain if they are vulnerable to this and other similar attacks.

As remediation by affected customers continues, it has been determined that espionage was the goal, not ransomware or theft of security tools (as in FireEye). To be more specific, the bad actors, once they gained access to the affected network, culled email, business applications and SharePoint sites for information. Thousands of confidential emails and other classified information were stolen by the bad actors! While the intent appears to be only espionage, this could very well be just the tip of the iceberg, or it could be a prelude by the bad actors of a worse attack yet to come.

How did this happen:

It is how the bad actors accomplished that is disconcerting. The current industry standard for maintaining third party provided applications is through pushing out patches to their customers. Think of it like the current Sirius XM Satellite Radio technology in cars. When you turn your car on, you will be notified on the console that there is an update waiting for you to install. Just click “ok” and the updated will install on your console. Or think of all those phone apps that you have on your iPhone or Android devices. It is the same premise. You see a little icon on the app and when you click on it, you can install the update immediately or in some cases overnight while you sleep. Do you trust that the update is virus free? Of course, you do, especially if it is from a reliable provider of services like Apple or Microsoft. Well, that is how the bad actors were able to infiltrate the SolarWinds customers, which just happen to include a multitude of Fortune 500 companies as well as a myriad of federal customers including the US Treasury, Commerce and Department of Justice.

The bad actors, which the federal government is identifying as Russia, had been at this for quite a while, before the damaging breach came to light in late December. There are various timelines that are in the news (Appendix A). If you want to see the technical progression. Here is the simplified timeline.

• Fall 2019 – A threat Actor accessed SolarWinds development environment (nothing yet on how) and started testing their malicious code in the SolarWinds product “Orion”.
• March 2020 – Going undetected, the actual malicious code was included in a hotfix or update and made available to customers.
• December 2020 – The breach is discovered by another security vendor, FireEye. SolarWinds quickly issued a software fix, but the damage had been done!

As of now, one last mystery remains, and that is how did the SolarWinds hackers manage to breach the company's network in the first place and install the Sunspot malware. It may take months before we know for sure.

Why should local governments be concerned?

The reality is that the method that the bad actors most likely used to infiltrate the SolarWinds development environment was a commonly used method. While we do not know for sure, it could have been an unpatched Virtual Private Network (VPN), an email spear-phishing attack, or a server that was left exposed online with a guessable password. The investigation is on-going, and it will be some time before we know the how. What is important to understand is that it could happen to any county and it does not have to be a county that uses the SolarWinds tool directly. It could be another vital provider in the supply chain, like a third-party provider of county services.

What should counties be doing?

This is a great question and one that I have been asked several times in the past month. The short answer is yes, you should be concerned. First, even if your county does not use their products, you may have a third-party vendor that you use that does. The supply chain is what you should be concerned about. CISA and others have reiterated that first and foremost it all starts with best practices. If nothing else, here are positive steps that every county can take to validate and potentially identify weaknesses that need remediated. To read more

• Limit Credential Access: Review and verify all regular, guest, system, privileged and admin user accounts.
• Implement Multi-Factor Authentication (MFA) for all your users, as well as vendors that access your systems through a Virtual Private Network (VPN). The bad actors are looking to find those one or two accounts that are not using MFA.
• Update your application (in this case SolarWinds) as quickly as possible when updates come out.
• Disconnect those devices from the internet that do not need 24/7 access. Ask yourself the question, does this device or application or service need to be connected to the internet?
• If devices require internet access, use strong firewall rules to limit access to the minimal services required to perform only the necessary functions.
• Review all technical indicators of compromise. As just one example, the Cybersecurity and Infrastructure Security Agency has provided guidance for organizations that integrate the Microsoft 365 productivity suite with the Azure user-authentication service.
• Review all vendor/supplier/third-party contracts to ensure that you have identified specific security control requirements.

Steps like those are crucial, but they are just a start. Lessons learned from the SolarWinds and FireEye breaches are just beginning to unfold!