WARNING: Zombie Policy Ahead

Maybe it's just my dark and twisted sense of humor, but I find it slightly funny when someone hacks a traffic control sign with the message "ZOMBIES AHEAD." It's probably not so funny to the person stuck at the back of the line in the resulting traffic jam though. Less humorous (because of potential panic and such), but along the same lines is the recent emergency broadcast system hack(s) warning of zombie attacks.

What I don't find funny is the seemingly endless night-of-the-living-dead cybersecurity legislation. Apparently, we are doomed to repeat our insane history by doing the same thing over and over again expecting different results. The Congressional graveyard of dead cybersecurity bills is coming back to life. Zombie bills are breaking through the soil and slowly ambling their way around the beltway seeking brains.

I'm sure someone somewhere has a comprehensive list of these dead cybersecurity bills, but from my own rough estimate it's at least a hundred of them in the past few years. What troubles me more is that we already have a rather significant foundation of cybersecurity legislation, regulation or otherwise legally oriented mumbling. I ask a simple question… Do we need new legislation/regulation or can we more effectively enforce (or maybe even refine) what we already have?

My hunch is that we won't make it out of 2013 without seeing at least one new cybersecurity law passing. Another prediction is that whatever we get will focus squarely on critical infrastructure.

If you've got skin in this game, now is the time to grab your virtual pitchfork and torch and storm the legislative castle (figuratively, not for real of course). By this, I mean use the power of your voice to create your future. If we're going to get zombie policy whether we like it or not, we should at least do our best to get something we can live with. That won't happen unless smart, informed people get involved and actually talk to Congressional staff. I know, I know… you're afraid that what you say can and will be used against you. Yep. It probably will. But we can't solve this problem with the same thinking that got us to where we are. The only way out is through. Together.

Here's the counterargument. If you don't engage you'll be labeled as "disinterested," "unresponsive," "apathetic," etc. If you do engage, and all you say is the same thing you've always said, you are no different than they are and you're essentially a zombie too. Something about a pot and a kettle should come to mind. Simply put, if you don't participate - in a meaningful and newly creative way - then you can't complain about what you get.

Image credit

Not Fast Enough

The problem with SCADA and ICS security isn't going to rapidly change for the better, no matter how bad the situation gets. Let's go ahead and accept the fact that Operational Technology (OT) environments are not able to withstand the degree of constant update enjoyed by traditional and future Information Technology (IT) environments. Complexity, cost, and culture are such formidable barriers at this time that our best short-term approach looks no better than a frantic game of "Whac-a-mole." Sure, long-term we can buy and implement better products - assuming the vendors provide us with better options - but for now, it would probably be best to assume your OT environments are not as secure as you think they are.

So what do you do?

To have any hope of "keeping the lights on," SCADA/ICS security should balance prevention, detection and response. You've heard it before. You've seen this portrayed on too many slide decks. So, has it truly sunk in? Simply put, you need to know that you can operate in a degraded capacity under duress with unknown system integrity. This usually means equipping experienced and engaged people with powerful and effective tools. It usually means changing architectures. It usually means changing culture. All of this means cost. Possibly BIG cost. Will the cost vs. benefit balance reach equilibrium or even tip in favor of benefit? In my opinion, yes. But not fast enough for many counting the coins.

[Photo by Maggie Smith]

Take Your Pick

In the recent TechNewsWorld article by Richard Adhikari "Study: Electric Grid Needs Full-Time Cyberguard," I was quoted as saying "We would welcome a single authority." At the end of that sentence, I also said "whomever that may be: DOE, DHS, etc." Note that I wasn't favoring any agency over another. The discussion was in reference to a recent report released from MIT, "The Future of the Electric Grid."

It is certainly true that the legislative, regulatory and overall policy sands are shifting. The existing bulk power system security regulations (NERC CIP) are changing. Multiple federal agencies are competing for control over the authority for grid security. The Distribution system, with its deep relationship to the consumer side of the grid modernization (smart grid) push, is hotly contested between the State Commissions and the feds.

All of this confusion has a numbing effect the utility executives. They are traditionally risk averse to begin with, and the policy forecast essentially indicates a 70% chance of storms ahead. The unintended consequence is that many organizations will only do the bare minimum required to be compliant with today's regulation. It is difficult to justify dedicating resources to future efforts with the significant possibility that things could change and that money may have been wasted.

With all of this churn, yes, I do think that some focus and harmonization in the policy landscape would be a good thing. Who can do this best? Well, the jury is still out on that one. I think all of the proposed agencies have their pros and cons. Ask me again in a year or so.

14 Seconds

I'm still a nobody (so I don't really qualify for a full 15 seconds) but I've been getting some media hits lately. Everyone told me, and I believed them to a certain extent, that everything you say can be twisted. I've dealt with many people who do just that and I have always felt that the truth will surface no matter what. Truth is like data. It wants to be free.

I sincerely make every effort to be balanced in my statements/positions and always say the good with the bad. My personal belief is that there's always good to be found. At times when I can't see it immediately, I try to drop the ego or emotion and look a little deeper. Invariably, it's there. With this in mind, I decided to dust off my personal blog and use it as the future platform to correct any misstatements, quotes taken out of context or just add the "whole story" where necessary.

So, let me start with the recent set of articles about various SCADA security topics (http://goo.gl/Kty17, http://goo.gl/KcvCh and http://goo.gl/txIDp). It is true, those statements are mine, and I did provide them in email interviews. They're actually pretty close to the mark, but I'd like to add a few of my other statements that were omitted, just for context...

"All of the above (and more) lead to a state where many are forced to operate with aging infrastructure extended beyond its lifespan. Note however, that many staff at municipal utilities are actually remarkably dedicated and resourceful people. They have to be, given the circumstances." 

"The threat is somewhat exaggerated, but it is still very real. The vulnerabilities are underestimated."

Please understand that I'm not casting aspersions on the Municipal Utilities of the world. Some are further along the security maturity path than others, but I have worked with many of them and I find them to be amazing people and amazing organizations.

Beyond The Bullets

It’s been a while since I’ve posted. I’ve been busy and various other pseudo-legitimate excuses, but something happened at a meeting recently that caused me enough pause to actually carve out time for a quick blog post.

I was discussing some trends from recent meetings when a participant began venting his frustration with a few of the bullets in one of my past presentations. I made a genuine attempt to explain that I don’t just read the bullets on the screen and go away. Rather, I provide context and backstory to each bullet while presenting. I gave him the context and backstory to the bullets of his concern but that didn’t seem to satisfy his frustration. Which is ok. I know I can’t please everyone. I’m certainly open to constructive criticism (as anyone who knows me from my WECC CIP Audits and Investigations days can attest). The fact that someone was frustrated with me and my content wasn’t the issue that spurred me.

The rub is that people actually think a PowerPoint presentation stands alone by itself. It doesn’t. Bullets are (or should be) used to cue the presenter’s thought process to provide valuable descriptive elements surrounding the bumper-sticker-bulletized blurb on the screen. You might get a fraction of the intended substance from the bullets in the slide deck but, assuming the presenter is worth a dime, then the presentation itself – given by the presenter – is where the real value resides. Whether webcast or in-person, the presenter should make an attempt to go beyond the bullets.

If you read something in a presentation that sets you sideways – or if you read something that really resonates with you – take either emotion with a grain of salt. After all, you’re only getting part of the story. If you can’t make it to the presentation then consider emailing or even calling the presenter to get the complete and intended message.

2010 EnergySec Summit 9/21-9/22, Denver

For six years running, the EnergySec Summit has proven to deliver the highest value when it comes to Energy/Electric Sector security events. And no, I’m not just saying that because I’m on the Board of Directors [FD]. I say it because I have yet to see another conference where such a diverse body of professionals meet under one roof and actually talk to each other. The variety of attendees ranges from all organizational strata of electric asset owners spanning the operational and business groups, State Regulators, Feds of all agencies, national labs, consulting firms big and small, security product vendors, industrial control systems vendors, research firms, and universities.

But it’s not enough to just get great minds to the table. The magic is when those great minds begin sharing in ways they hadn’t before, thus generating a result that is greater than the sum of its parts. The EnergySec culture draws this out of you. The collaborative spirit is simply pervasive and infectious. There is no singular agenda or axe to grind. The audience isn’t just there to listen and blink at powerpoint slides from the so-called “experts” all day. Instead, all attendees are recognized as experts and they are active participants in the event. You can’t help but leave empowered with new connections, new knowledge and an invigorated desire to do more for your company and your industry. Sure, this happens at many conferences, but there is something unique about the EnergySec vibe that takes it to a different level.

The information sharing grass roots effort known as EnergySec today has matured from its humble beginnings back in 2004 as Energy Security Northwest (E-Sec NW). Those original meetings of adjacent asset owners over lunch and the mailing list were the seeds of something great. The membership started as an all-volunteer group with no real organizational structure other than a loose “board” of moderators who maintained the vision of the EnergySec culture. They fostered it through technological advancements and beyond into non-profit status as a 501(c)(3) company. Today, it boasts 380+ members, 102+ organizations, 50%+ North American Generation, 65.3%+ North American Distribution, etc.

Ok, so back to the Summit... ICF International [FD: my current employer] kicked things off with a networking breakfast on Day One. I followed with my “Energy Sector Crystal Ball” spiel to kick off the event and set the mood. I really tried to keep it FUD-free and just talk about trends I’m seeing within the industry. Carol Hawk of the Department of Energy then walked through some of the projects they are currently working/funding - one of which is the National Electric Sector Cybersecurity Organization (NESCO). More on this again later. Mike Mertz of PNM then presented on security policy. His subtitle was “The most important element for Security and Compliance.” By the time he was done, he had me convinced. Mike always has such a great way of getting to the core of the issue and providing actionable take-aways. Bill Hunteman of DOE was up next to tell the group about the department’s strategic direction for smart grid security, an area in which nearly everyone is interested. Bill was followed by Brian Girardi of NetWitness presenting on network forensics & advanced threat analysis for critical infrastructure - an area where NetWitness is recognized as a practice leader.

Then came lunch, sponsored by NetWitness and more networking. I eavesdropped on several conversations as I usually do and joined in as many as I could. I am always surprised to see how much fusion happens when the walls come down and the silos are broken.

Lunch was followed by an entertaining and insightful presentation from Mike Assante (now with NBISE, formerly NERC Chief Security Officer). I particularly enjoyed the parallels between suicidal squirrels and the threat actors, but this portion of his talk was curiously left out of the presentation he provided to EnergySec for posting (I’ll see if he’d be willing to furnish the un-redacted deck). Suicidal squirrels aside, Mike had some solid and sobering wisdom to share about how regulatory compliance has affected cybersecurity for the industry. Jack Whitsitt, Sr Critical Infrastructure Cyber SME supporting the Transportation SSA followed Mike with a technologist’s admission of inadequacy, the executive’s role in national cybersecurity. Though Jack doesn’t come directly from the power biz, he knows industrial control systems security and his subtle-but-wry humor and acute technical references really fit the audience well in my opinion. Tim Erlin of nCircle was next with a talk on configuration auditing and vulnerability management in the NERC CIP era. Short of the free tool Nessus, nCircle probably has the largest presence in the vulnerability and configuration management space within the electric sector, based on what I’ve seen as an auditor and consultant.

Day One closed with a panel on the Summit’s primary topic, the intersection of security and compliance. The panelists were: Josh Axelrod, WECC CIP Audits and Investigations Team Lead; Dave Norton, Entergy Policy Consultant - Critical Infrastructure Protection; Mike Assante, NBISE President and CEO; Mike Mertz, PNM Resources Sr Project Manager of FERC Compliance and Ben Miller, Constellation Supervisor of Information Security Operations. As you can probably imagine, this was a lively debate. There was no shortage of opinions, from the panelists or the audience. A recurring thread was that the existing NERC CIP standards may be decreasing security in some (but not all) implementations for the sake of compliance. Some great examples of satisfying both security AND compliance were also provided. It was a fantastic way to end the day and really primed the discussions during the reception.

Big thanks to nCircle for providing drinks and hors d'oeuvres. Liquor and food have a funny way of changing the dynamic. After a day immersed in the collaborative culture and high-value content, minds were reeling and ruminating. What was left of the inhibition to release opinions quickly vanished, which made for weighty and direct conversation. I eavesdropped some more, made more new friends and had my fair share of the hotel’s Bourbon.

Day two started with Tim Roxey, NERC Manager - Critical Infrastructure Protection. Tim’s presentation did a fantastic job of capturing the complexity of today’s power grid. The analogy of flocking birds as the intelligent future of power management was very intriguing. The vulnerabilities in such a complex system are not being fully considered - yet. Next came the highlight of the event in my opinion. James Arlen (Push The Stack Consulting) reinforced the information sharing purpose with his presentation “SCAD’oh - An Agnostic’s View: stop the ‘us v. them’ and come together to solve the problem.” James delivered the message with refreshing candor and corrosive humor. No one was safe. We all got a chance to laugh at each other and ourselves. We were all well aware of many opportunities for self-improvement by the time James left the stage. But everyone needed to hear every single word. If there was any ego remaining in the room, they simply weren’t listening. Up next was Steven Parker, EnergySec Director. Steve used puzzle pieces handed out at the beginning of the event to convey his message of sharing and collaboration. Each attendee put their name on their piece and then handed them back at the end of Day One. On Day Two, the puzzle was available for assembly at the registration table. The message: we are all parts of the cybersecurity puzzle. Only when we work together do we form the complete picture. Steve also included a briefing on the EnergySec state of the union including the announcement of the formation of the NESCO. The event was wrapped up by Sean McBride of Critical Intelligence with a presentation on developments in ICS defense. As usual, Critical Intelligence hit the mark based on their deep involvement in the subject. CI provides the light version of their open-source intelligence reports to EnergySec members.

Day Two ended with a final networking lunch. You probably guessed it but I eavesdropped again, chatted more and exchanged contact details on napkins because I was out of business cards.

Throughout the event I received strongly positive feedback from the attendees on the quality of the content in addition to the networking/collaboration benefit. It may sound like a big hugfest or kumbaya session, and it was to a certain extent but there was also plenty of dissenting opinion and experienced debate to keep things balanced. No doubt that everyone likes to know they're not alone in the difficult struggle to secure the North American power grid, and the level of professional exchange is what makes the EnergySec Kool-Aid taste so good.

Another interesting addition this year was the live tweeting from the event and commentary/discussion from those on the floor and those who were unable to attend. The real-time discussion went beyond the walls of the conference room. Very cool indeed. I finally met a bunch of people from Twitter face to face (@sintixerr @myrcurial @electricfork @CrucialCarl @tmdheard @jholcomb @pmhesse to name just a few).

Lastly, a HUGE thanks to Lisa James, EnergySec Chair and her supporting cast of volunteers, Jeri Freimuth and Ed Croft. I’ve been to very expensive conferences with paid armies of staff and the EnergySec logistics were better.

I’d like to close the post with a brief note about the National Electric Sector Cybersecurity Organization (NESCO) mentioned above. EnergySec was awarded a $5.8M grant over three years to stand up the NESCO. The NESCO will be supported by EPRI and their collaborative as the research element, NESCOR - the “R” is for resource. Some details are in the presentations from Carol Hawk, Bill Hunteman and Steven Parker. You can read more in the official press release. If you are interested in hearing more, contact Lisa James, EnergySec Chair at lisa@energysec.org.

The photo is the Sheraton Downtown Denver.

The Cabin Door Is Closed, Please Power Down All Electronic Devices...

I fly a lot. More than many. Sometimes over 50%. As such, I have some experience with the consumer side of the commercial aviation business. I am by no means an expert. Recently, I was asked to speak at an air traffic controller's conference on the subject of cyber security in the Next-Generation Air Transportation System. I was there to provide a perspective from the outside, more of a security technology discussion for what works in the overall CI/KR space. It was a panel, so the slide deck was short - which was good for me, because again, I'm not an expert in aviation. The panel had current and ex-FAA staffers, university professors, aviation consultants from the defense industry/sector and me. Do you remember the old Sesame Street song "one of these things is not like the other?" Throughout the event, I was constantly reaching for my smartphone to Google the acronyms that I'd never heard.

I had a similar newbie feeling when I started working in the electric power sector. Before then, I was just a consumer. I flipped the switch and expected the lights to come on. When the power went out at my house, I was the first to call my local utility and give them a piece of my mind as a paying customer. After all, us geeks can't live without our tech toys for longer than a few seconds. So the first time I actually spent more than five minutes on the Control Center floor, I was.. well.. floored. Even mild spring days in the 'shoulder months' can seem like a delicate balance of order and chaos. The electric system is interconnected, just like the aviation system. What happens in one area will quickly and directly affect other areas, some quite distant. The real-time seat-of-the-pants decisions by system operators is really what keeps the system running - not the technology. Sure, the technology is there, but it is only a tool.

I see a situation, whether it is Smart Grid in the power biz or the Next-Gen Air Transportation System in the aviation sector, where we are inserting a much wider technological distance between the human and the physical/kinetic endpoint. System operators are using ever-increasing layers of technology. Until fairly recently, they looked at some sort of analog or electro-mechanical instrumentation for operational decisions and then they would physically (manually) activate something. Today, we have operators using tools which are in turn, using other embedded tools, which may also be using further embedded tools - and so on. This can be a good thing for many reasons, but it can also be a bad thing. This trend, though perfectly natural - even expected, should be carefully monitored, carefully balanced. Especially when it involves critical infrastructure. We may even need to tip the scale toward sound security engineering instead of focusing solely the profit drivers. At least for a while.

We've ignored our critical infrastructures for so long that we are in desperate need of an overhaul. Nearly every one of the sectors in the National Infrastructure Protection Plan (NIPP) could be called brittle. Some money is starting to flow to these areas for much needed upgrades but the legacy technology and the bleeding edge enhancements need to work together in the same interconnected system. This creates a 'base of sand' problem. Legacy devices are underpinning tomorrow's technology gizmos with incredible distance between the two ends of the spectrum. We need a security engineer to put their stamp on the blueprints BEFORE they get the permit to build. When adding to or modifying an existing structure, the structural engineers factor those old trusses, supports and cracks in the foundation into the new design. I don't want to discount the great work being done here, but I think few would disagree that we have a cart-before-the-horse situation.

The most common recommendations I heard at the recent aviation conference were:
- Test bed for qualifying systems (approaching Certification and Accreditation)
- Minimizing potential of supply chain attacks
- Security Training for operators/controllers
- Situational Awareness (and integrity of decision support data)
- Information Sharing

Those of you following the power sector for the past few years should see some striking similarities. I'm willing to wager that nearly all CI/KR sectors are facing these same challenges. The only recommendation didn't see was slowing down to get security issues addressed in the design phase. I've been a security professional long enough to expect that, but I can't seem to bring myself to accept it - hence this post/rant.

Like nearly all of my posts, I am writing this as I fly home on a commercial airline. Now if only I can think of a solution to being crammed into a space smaller than my anatomical dimensions. I'm not important or rich enough for First Class seats. But every time I think I've got it bad, I remember my co-worker CJ and his 6'5" span. He's taller than most clearances at drive-thrus and parking garages. Unfortunately TSA frowns on bringing a crowbar to extract him from the seat.

The photo is from CodeProject