Your fridge may be part of an underground network of evil, bent on destroying America’s electric grid, and you don’t even know it.
The Washington Post was roundly criticized for, and forced to retract, a story claiming that Burlington (Vt.) Electric Department was targeted by the Russians for hacking, and that the malware penetrated the U.S. electric grid. It turned out that the story was wildly exaggerated, and not properly vetted by Post editors or reporters.
In a followup, the Post admitted:
An employee at Burlington Electric Department was checking his Yahoo email account Friday and triggered an alert indicating that his computer had connected to a suspicious IP address associated by authorities with the Russian hacking operation that infiltrated the Democratic Party. Officials told the company that traffic with this particular address is found elsewhere in the country and is not unique to Burlington Electric, suggesting the company wasn’t being targeted by the Russians. Indeed, officials say it is possible that the traffic is benign, since this particular IP address is not always connected to malicious activity.
An employee of an electric utility checked their Yahoo email from a company computer. As a computer geek, I know if I ran a utility, I’d block all of Yahoo from the work network, period. Yahoo is a sieve begging users to infect their PC’s. It’s no wonder the computer had malware.
Scary? A little, but not much. So we have nothing to worry about, right? What’s this have to do with my fridge anyway? Should we all smash our smart TV’s, fridges, and stoves because an electric company employee checked Yahoo mail? Seeking a real answer, I decided to find out for myself.
I started with the the electric utility that serves my home. It’s an electric cooperative, not one of the large power companies that serve most cities. I’m not just a customer of my electric utility–I’m a member. I’ve known and done business with this company for over 20 years, so I shot an email over to my friend Jimmy Autry, Senior VP of Member and Community Relations at Flint Energies. We caught up over some nice Mexican at lunch.
Co-ops are an artifact of the Rural Electrification Act of 1936, during FDR’s administration. While the rules for co-ops are a bit different than for larger companies like Georgia Power, that also run power plants and large scale distribution networks, in general, home distribution of power in America is pretty standardized.
First, we discussed some misconceptions people have about “the grid.” It’s not one monolithic control system, in a bunker under Manhattan, that controls the entire country’s power. That’s the stuff of movies. Even if hackers penetrated Burlington Electric, it wouldn’t affect anyone else. But the fear factor gives rise to “preppers” doom predictions and a heightened awareness of vulnerability.
When the power goes out for even 10 minutes, people panic. No lights, no electric stoves, heaters, computers, televisions–it’s frightening. When large scale power outages happen, like the one that killed power from Cleveland to New York City in 2003, people are more than inconvenienced. Life and safety are threatened.
After that blackout, Congress tightened regulations, giving the North American Electric Reliability Corp. (NERC) the ability to fine utilities for failure to comply with standards. But standards are only as good as those who implement them.
The good news
The good news is that each utility operates its own, independent Supervisory Control and Data Acquisition (SCADA) network. SCADA is the equipment and software that monitor and control power grid reliability. From opening a circuit switch at a transformer, to preventing a cascading failure such as the one that caused the 2003 blackout (and one in Europe in 2006), SCADA systems are integral to electric grid reliability.
Should a hacker penetrate a SCADA system and take control, the results could be very damaging. But the damage would be local. There’s no “national SCADA system” for hackers to take over. Utilities connect to each other at various switch points and yards where high voltage power lines come in to local distribution grids.
But, as Autry told me, the system is much more hands-on than people imagine. The failsafe is in manual control. If the SCADA system were disconnected, the grid would still function. Linemen and operators would have to revert to manual procedures, which they’re trained to do. Standardization helps in emergencies. Crews from the smallest and largest utilities all know how to work with other company’s grids because they all use the same standardized ways of building the network.
It’s far more important to grid security and reliability that utilities maintain pre-signed mutual aid agreements and crossover points, Autry explained. These allow crews to quickly move and anticipate natural disasters, storms, and other contingencies. It’s a very manpower-centered process.
SCADA systems are also isolated, as much as possible, from public networks. Generally, that means there’s no contact between a SCADA system and outside computers. Policies at electric utilities prevent portable storage devices, thumb drives and even DVDs from connecting to SCADA systems. This means any attempt to control SCADA from the outside would require a breach of those policies.
There are exceptions.
The payment system that Flint uses allows some unbanked customers to pay in advance for power–even for a day’s worth–at special kiosks placed around the served areas. These kiosks can turn on power at any address in the system. The payment and accounting systems can also disconnect power from a desktop computer.
If a hacker penetrated the payment system (operated by a contractor), they could theoretically turn off, or on, power for the entire 70,000-plus served meters. “Theoretically” is a long shot. The issue isn’t that it can’t happen, it’s that the network itself is the limiting factor. Some utilities use “meshed wireless” networks to transmit signals from meters to and from SCADA. Some use fiber-optic connections. Flint’s system uses the power grid itself to transmit data, but it’s slow.
In this case, slowness isn’t a bug, it’s a feature. The slowness means that the system would take hours to turn off everyone’s power, house by house. There would be more than enough time for operators (and overloaded customer service lines) to catch the problem and disconnect the offending system. Maybe one home could get free power, but that’s more a fraud issue than a grid security vulnerability.
A double-edged sword
The Rural Utilities Service, part of the U.S. Department of Agriculture, helps fund and develop infrastructure for rural communities. Of the 900 or so electric co-ops in America, only around 180 are what the industry called “independent borrowers,” Autry explained. Borrowing from the RUS requires utilities to meet the agency’s standards, including grid security.
The RUS website agrees with these figures.
The RUS Electric Program helps nearly 700 borrowers in 46 states finance safe, modern, and efficient infrastructure. The resulting loan portfolio of approximately $46 billion is managed by the Electric Program.
As an aside, the RUS also helps implement President Obama’s Climate Action Plan by forcing it down member-owned rural utilities’ throats–ironically the very people who voted for Trump. Hopefully that will be one of the things Trump undoes “on day one.”
But the RUS standards act as a double-edged sword. A relatively small group of analysts and engineers determine technical standards for RUS-funded utilities, and the utilities must comply with these standards. They literally determine which vendors utilities can buy from, for everything from “Arresters, surge” to “Y-clevis ball” (whatever that is), down to the catalog part number.
Again, that’s useful for interoperability, but such tight standards create a very focused market for industrial control systems (ICS), which make up a big chunk of the SCADA network.
These systems are vetted and standardized by NERC, for large utilities and power producers. Smaller utilities rely on the National Rural Electric Cooperative Association (NRECA). Steven Bell, NRECA’s media relations director, wrote this in an email to me.
Recognizing cybersecurity supply chain issues, electric co-ops and the electric utility industry use cybersecurity supply chain best practices and, through the North American Electric Reliability Corporation (NERC), are developing mandatory standards in this area for the bulk electric system.
On the same day (Jan. 6th), the organization’s CEO, Jim Matheson issued this statement.
We concur wholeheartedly with the view that both electricity and broadband are vital to securing the future of rural communities. We therefore echo the call to strengthen the rural electric grid and extend broadband to the thousands of communities still lacking access to high-speed internet.
Where the tight standards and purchasing guidelines of the grid meet the freewheeling cost-reducing imperatives of broadband is the point at which the double-edged sword comes into play.
Utilities are big attractive targets for hackers. Just seeing a utility’s website defaced used to be enough (this happened to Flint Energies in 2007), but now hackers are more sophisticated and their goals are more nefarious. Autry told me that Flint is probed every single day. And that has led to increasing investment in cybersecurity.
Policies can only go so far, because employees must follow them to be effective. Autry agreed with me over an enchilada that employees are really the weakest link (as John Podesta found out when his aide clicked on a spear-phishing malware link). And that’s where the bad news starts.
The bad news
An electric utility engineer’s biggest fear of cyber vulnerabilities is when new equipment shows up in the box.
That plus an employee’s careless mistake can compromise a grid. And the hackers who are capable of actually breaking into a grid aren’t doing it for yucks, or to see some social message plastered on the public website of the company.
As an extreme example, Iran ran one of the most protected, secret uranium enrichment programs in the world. Yet someone (speculation is the U.S., possibly working with Israel) managed to get Stuxnet into their computer network. Stuxnet was more than just a computer virus. It contained a payload that infected the programmable logic controllers (PLC’s) in the Siemens high speed centrifuges Iran used, and another payload that caused the ICS system controlling those PLC’s to register false information to mislead the operators.
PLC’s run everything from 3D printers to automobile painting robotic arms, to ICS systems used in SCADA networks. And many of them are mass-produced in China, where hacking is a state-sponsored sport.
Vulnerabilities are found every day for PLC’s “in the wild.” The government’s Computer Emergency Response Team (CERT) publishes them. One, just a month old, is for the Siemens S7-300/400, a part that the company said is “used worldwide.”
Another, from 2015, is the Rockwell-owned Allen-Bradley MicroLogix PLC.
The so-called FrostyURL vulnerability affects the Allen-Bradley MicroLogix 1100 PLC used to control industrial processes in a number of critical industries. CyberX, a security vendor operating in the industrial control system and SCADA markets, said that a single click of a maliciously crafted URL could affect an operational network.
“It blew our minds how simple it is,” said Nir Giller, CyberX CTO.
Your home as a hacker’s paradise
This isn’t a new problem. In 2011, security writer Dale G. Peterson wrote that PLC’s are “insecure by design” in applications like SCADA. With the arrival of the Internet of Things (IoT), vulnerabilities that affected only a handful of industrial processes can now find their way into everyday devices like refrigerators, ovens, garage door controllers, door locks, security systems, you-name-it in the ever-widening world of home automation.
I’m not so worried about the large manufacturers like Google (who owns Nest), or Apple, or Amazon, or even Microsoft (if they ever get in gear). Those companies have the resources and a commitment to security–in general–to handle IoT issues. They also offer robust application program interfaces (APIs) and software development kits (SDKs) to smaller companies wishing to use their back ends.
That’s good and it’s bad. The smaller companies might use devices procured from shady companies in China constructed with built-in backdoors, unbeknownst to the developers who used the part. Then they connect these devices to Apple’s HomeKit, or Google Nest, or Amazon Alexa API.
Since many companies use shared authentication schemes like OAuth2 to allow users to log in through Facebook or Google, they might use identical passwords on “Brand X” devices, or even a Frigidaire Smart Appliance. I mean, who would hack your fridge? Well, nobody, if the goal was to spoil your milk.
But if the goal was to enlist your fridge without your permission in a pernicious evil army of cyber bots to attack your electric company, well that’s a different story. On October 21, 2016, a series of large-scale distributed denial of service (DDOS) attacks disrupted U.S. internet service.
The 10/21 attacks were made possible by the large number of unsecured internet-connected digital devices, such as home routers and surveillance cameras. The attackers employed thousands of such devices that had been infected with malicious code to form a botnet. The software used to crawl the internet to find unsecured devices is freely available. Even though some of these devices are not powerful computers, they can generate massive amounts of bogus traffic to swamp targeted servers, especially if you abuse a large numbers of them at once.
These DDOS attacks are trouble for the companies targeted by them, but really more of a nuisance for the rest of us who can’t binge-watch Netflix. But what if a savvy attacker combined insecure IoT and home router gear to target an as-yet undiscovered PLC vulnerability, and used that to craft an attack against the completely standardized and vendor-limited SCADA systems?
How do we know that hasn’t already happened?
The grid is pretty secure, but the threat is real
Actually, we don’t. It’s the reason why engineers are scared of equipment new out-of-the-box. It’s the reason NRECA and NERC invest time and money to work with vendors and set cybersecurity standards. It’s the reason larger utilities spend enormous resources training employees in cybersecurity. It’s the reason small co-ops like Flint Energies invest in upgrading their cybersecurity systems.
Constant vigilance is the answer. The so-called scare of a single virus penetrating the electric grid in Burlington, Vermont may have been overhyped, but that doesn’t make the threat any less real.