Special of the Day: Elephant

First, thanks to Joseph Menn of Reuters for the article. Joseph is a great reporter. He asks intelligent questions, and clearly, he "gets it" in my opinion. I just wanted to add a little more to my part of the story.

The quotes provided… "We've got this cancer that is growing inside our critical infrastructure. When are we going to go under the knife instead of letting this fester?" and "We need to restructure some regulations and incentives." …were part of a larger conversation where many things were discussed. Everyone should know by now that Industrial Control Systems, in particular the legacy devices, are prone to security vulnerabilities. For example, in the article two of the researchers (Eric Forner and Brian Meixell) took less than 16 hours to find something worthy of a BlackHat talk.

This is not an issue for any one country or company. We all buy the same gear from the same vendors.

Everyone asks if regulation would make the situation better. The simple answer is no. Regulation - by itself - won't solve this problem. A good example would be the NERC CIP Standards. They are often credited with getting the security ball rolling for the electric sector. They may or may not ultimately lead to a more secure grid, but I can say for certain that we have seen some very real "unintended consequences" from the existing regulatory model for the electric sector in North America. Other countries and continents are watching what is happening here and basing their future choices on our lessons learned.

For those that don't fully appreciate our current regulatory construct, I'll keep it simple. With the existing Powers That Be, there is no way to regulate cybersecurity for the power grid end-to-end. There's no department, agency, bureau, commission or corporation with jurisdiction to cover all of it. So, we're left with a patchwork of rules from authorities that work together through a model that more closely resembles a bar-fight than a choir singing from the same hymnal. Sometimes this friction is actually beneficial. Sometimes it isn't. Either way, the utilities are left with the bill. And like most situations, money matters.

Which brings me to incentives. Somehow, we found a way to incentivize things like Smart Grid. To the tune of billions of dollars. We haven't done the same thing for cybersecurity. We need incentives for R&D so new security technologies can make their way from idea to product. We need incentives to upgrade to these new security technologies and get rid of the excuse that it costs too much to take an outage and replace hardware. We need incentives and assistance to train the next wave of cybersecurity professionals for the industry. We need incentives to share information between utilities so they can gain situational awareness from their interconnected peers (and the government, if they so desire) - in a way that won't put them in a legal pinch for doing so.

Through all of this, electric utilities are not just sitting on their hands when it comes to cybersecurity. There are dedicated, responsible, and profoundly smart people keeping the juice flowing. They do it every day. They fix it fast when it breaks. And they work just as hard to keep it secure. This is serious business for them. They are using the equipment they have, and yes, some of it is vulnerable to the types of attack you will see at BlackHat next week. Yet, they get the job done safely and at the lowest cost possible. So, the next time you flip the switch on the wall, remember that your local utility is harnessing one of the most powerful forces in nature and squeezing it down a skinny wire to your house so you can see in the dark, dry your jeans and toast a bagel.

Well, what's the solution? Sure, we could make those greedy, irresponsible vendors fix their vulnerable gear. And yes, that's a good start, but the utility can only upgrade so fast  (don't forget about that whole "force of nature" thing -- you gotta work carefully and methodically around stuff like that). We could fix the regulatory model. Yeah, that's not a bad idea but remember we're not really set up for that. It would literally take an act of congress to make it happen - which isn't entirely out of the picture but I'm not holding my breath. We could throw money at the problem, but that will bring the carpetbaggers and crooks along for the ride. See my point? There isn't one silver bullet to solve this problem. Like brushing and flossing, like diet and exercise, it's going to take conversation and compromise. This is the hard road, but it's the right road and everyone knows it. They're just too lazy to do it right.

Join me. Grab a spoon, get your seat at the table and let's eat this elephant in the room together. It's not going to taste good. It's not going to be over quick. But we won't get past it until we decide to do it together. I'll bring the Bourbon and sarcasm to help wash it down.

Image source - http://www.flickr.com/photos/mobilestreetlife/4179063482/

"By Failing to Prepare, You Are Preparing to Fail"

I thought I might take a few moments to provide some backstory given the recent buzz about my post yesterday announcing my move to the independent consulting path, the respective management transition at EnergySec and the recent post by Dale Peterson of DigitalBond highlighting the struggles the organization went through in December 2012.

First, the astute quote from Benjamin Franklin in title of this post is about preparation. We knew at the onset that the organization would need a heavier than usual executive load to provide the industry clout, clear direction and depth of experience needed to take the organization from its all-volunteer state to a self-sustaining nonprofit. We also knew that the startup mindset and staffing model would need to change once we saw clear evidence that the organization would be able to support itself. This is just smart business.

In December of last year, DOE re-affirmed commitment to the NESCO and extended the agreement but chose to hold their contribution until EnergySec could catch up to its part of the cost-share. At that time, the executives, in discussion with our Board of Directors and the NESCO Advisory Board, made the decision to streamline and enhance the operational capabilities to capture more immediate revenue. This was an easy call because we were already seeing steadily increasing positive traction with the NESCO Tactical Analysis Center and the organizational memberships. We were also running financial models to identify when the organizational changes to senior management would need to take place. The December circumstances required that we make some prompt staffing adjustments, close the cost-share gap, and ultimately re-instate the DOE cost-share contribution. The strategy is working. This approach is aligning the organization with the expected financial targets, but it was invoked slightly sooner than we forecasted.

Sure, partnering with any federal agency for funding is a challenge (ever tried to get VC for a nonprofit?). I've always said that federal grant money is "free like a puppy." We knew the risks going in, and unfortunately a fair share of those risks were realized. The risk landscape also included things like economic recovery/recession, competitive landscape, and politics. But as a result of the experience, we have a well-managed, lean, agile and tightly focused EnergySec operating the NESCO program. This benefits everyone.

I'm genuinely pleased with the direction the organization is going. EnergySec's story is a testament to the challenges inherent in sharing information effectively. It is an example of the complications found in public-private partnerships. It is proof of the strength and determination demonstrated by people dedicated to the security of our industry. We've been doing this longer than anyone else and we're still learning - as we should be.

[image credit]

Next Step

You may have heard about the recent EnergySec change in management. Yes, I'm taking the next step.

But no worries. I founded the organization over 10 years ago, I'm still very much attached to EnergySec and always will be. This transition was actually planned so it's a good thing for everyone. When the EnergySec Board asked me to leave the private consulting world and take the wheel as CEO in 2010 it was really to advance the nonprofit mission to the next level with the inertia created by the funding of the DOE cooperative agreement and get the NESCO off the ground. My goal was to be the steep-curve leader of the outreach, promotion and community building necessary to bootstrap the nonprofit to a funded and sustainable point. After which, I'd replace myself with a more operational executive and shift to an oversight and governance role. This allows the organization to streamline and focus on delivering quality products and services. Steve Parker is a perfect choice for CEO. He's been a key player in the organization since inception and he's built most of the operational aspects with his own two hands. I'm still deeply involved in EnergySec as President Emeritus, executive committee member of the NESCO Advisory Board, company liaison to the North American Energy CISO Forum and most importantly through my seat on the Board of Directors. I still wear an EnergySec badge at many of the conferences I attend and I'm a guest instructor for some of their training/classes. I'm their biggest advocate.

In short, I'm not really going anywhere. Just handing the day-to-day EnergySec operations over to a trusted friend and executive while I go back out into the community to do what I do best - be a catalyst, build interesting connections, ask the hard questions and provide the honest answers when no one else will. I'm always drawn to the next wave and I'm seeing some fascinating things on the horizon.  At the moment, I'm doing a lot of independent consulting/advisory around the critical infrastructure security, regulatory and policy areas for utilities as well as software and hardware companies. If you have any interesting projects that might benefit from my participation, please let me know!

While I have your attention, I would like to issue a call to arms. EnergySec and NESCO are doing great things for/with the industry. They are doing the hard work that everyone said couldn't (or shouldn't) be done. I've always told my staff "If it were easy, someone else would have done it already." And they still live by that message today. We knew it would be difficult in ways we couldn't imagine. But it was worth every ounce of effort to make it happen because it was (and will always be) the right thing to do. We can show the rest of the critical infrastructures, the Federal onlookers (regulators, etc), our energy industry peers and even the hype-driven media that we "get it." We are advancing security from the bottom up, as a sector united, dedicated to the mission of safe, reliable power. The industry needs a place where the asset owners can have open, honest and candid discussions about the potentially sensitive subject of security. EnergySec is that place. We must all work together to keep it safe.

So, do your part (if you haven't already) and purchase the organizational membership. Get your executives to join the CISO Forum. Get your people involved. Sign up for the Rapid Notification System. Buy a subscription to the Tactical Analysis Center (TAC). I know there are many organizations out there asking for the roughly same commitment. But I challenge any of them to demonstrate the 10-year history of trust, value and loyalty to the security cause that EnergySec has proven. And as part of the nonprofit mission, we do it at a fraction of the cost. Higher value, higher trust, and lower cost? Sounds like an easy choice to me.


You can always reach me at my EnergySec email address or my personal account if needed.



[image credit]

WARNING: Zombie Policy Ahead

Maybe it's just my dark and twisted sense of humor, but I find it slightly funny when someone hacks a traffic control sign with the message "ZOMBIES AHEAD." It's probably not so funny to the person stuck at the back of the line in the resulting traffic jam though. Less humorous (because of potential panic and such), but along the same lines is the recent emergency broadcast system hack(s) warning of zombie attacks.

What I don't find funny is the seemingly endless night-of-the-living-dead cybersecurity legislation. Apparently, we are doomed to repeat our insane history by doing the same thing over and over again expecting different results. The Congressional graveyard of dead cybersecurity bills is coming back to life. Zombie bills are breaking through the soil and slowly ambling their way around the beltway seeking brains.

I'm sure someone somewhere has a comprehensive list of these dead cybersecurity bills, but from my own rough estimate it's at least a hundred of them in the past few years. What troubles me more is that we already have a rather significant foundation of cybersecurity legislation, regulation or otherwise legally oriented mumbling. I ask a simple question… Do we need new legislation/regulation or can we more effectively enforce (or maybe even refine) what we already have?

My hunch is that we won't make it out of 2013 without seeing at least one new cybersecurity law passing. Another prediction is that whatever we get will focus squarely on critical infrastructure.

If you've got skin in this game, now is the time to grab your virtual pitchfork and torch and storm the legislative castle (figuratively, not for real of course). By this, I mean use the power of your voice to create your future. If we're going to get zombie policy whether we like it or not, we should at least do our best to get something we can live with. That won't happen unless smart, informed people get involved and actually talk to Congressional staff. I know, I know… you're afraid that what you say can and will be used against you. Yep. It probably will. But we can't solve this problem with the same thinking that got us to where we are. The only way out is through. Together.

Here's the counterargument. If you don't engage you'll be labeled as "disinterested," "unresponsive," "apathetic," etc. If you do engage, and all you say is the same thing you've always said, you are no different than they are and you're essentially a zombie too. Something about a pot and a kettle should come to mind. Simply put, if you don't participate - in a meaningful and newly creative way - then you can't complain about what you get.

Image credit

Not Fast Enough

The problem with SCADA and ICS security isn't going to rapidly change for the better, no matter how bad the situation gets. Let's go ahead and accept the fact that Operational Technology (OT) environments are not able to withstand the degree of constant update enjoyed by traditional and future Information Technology (IT) environments. Complexity, cost, and culture are such formidable barriers at this time that our best short-term approach looks no better than a frantic game of "Whac-a-mole." Sure, long-term we can buy and implement better products - assuming the vendors provide us with better options - but for now, it would probably be best to assume your OT environments are not as secure as you think they are.

So what do you do?

To have any hope of "keeping the lights on," SCADA/ICS security should balance prevention, detection and response. You've heard it before. You've seen this portrayed on too many slide decks. So, has it truly sunk in? Simply put, you need to know that you can operate in a degraded capacity under duress with unknown system integrity. This usually means equipping experienced and engaged people with powerful and effective tools. It usually means changing architectures. It usually means changing culture. All of this means cost. Possibly BIG cost. Will the cost vs. benefit balance reach equilibrium or even tip in favor of benefit? In my opinion, yes. But not fast enough for many counting the coins.

[Photo by Maggie Smith]

Take Your Pick

In the recent TechNewsWorld article by Richard Adhikari "Study: Electric Grid Needs Full-Time Cyberguard," I was quoted as saying "We would welcome a single authority." At the end of that sentence, I also said "whomever that may be: DOE, DHS, etc." Note that I wasn't favoring any agency over another. The discussion was in reference to a recent report released from MIT, "The Future of the Electric Grid."

It is certainly true that the legislative, regulatory and overall policy sands are shifting. The existing bulk power system security regulations (NERC CIP) are changing. Multiple federal agencies are competing for control over the authority for grid security. The Distribution system, with its deep relationship to the consumer side of the grid modernization (smart grid) push, is hotly contested between the State Commissions and the feds.

All of this confusion has a numbing effect the utility executives. They are traditionally risk averse to begin with, and the policy forecast essentially indicates a 70% chance of storms ahead. The unintended consequence is that many organizations will only do the bare minimum required to be compliant with today's regulation. It is difficult to justify dedicating resources to future efforts with the significant possibility that things could change and that money may have been wasted.

With all of this churn, yes, I do think that some focus and harmonization in the policy landscape would be a good thing. Who can do this best? Well, the jury is still out on that one. I think all of the proposed agencies have their pros and cons. Ask me again in a year or so.

14 Seconds

I'm still a nobody (so I don't really qualify for a full 15 seconds) but I've been getting some media hits lately. Everyone told me, and I believed them to a certain extent, that everything you say can be twisted. I've dealt with many people who do just that and I have always felt that the truth will surface no matter what. Truth is like data. It wants to be free.

I sincerely make every effort to be balanced in my statements/positions and always say the good with the bad. My personal belief is that there's always good to be found. At times when I can't see it immediately, I try to drop the ego or emotion and look a little deeper. Invariably, it's there. With this in mind, I decided to dust off my personal blog and use it as the future platform to correct any misstatements, quotes taken out of context or just add the "whole story" where necessary.

So, let me start with the recent set of articles about various SCADA security topics (http://goo.gl/Kty17, http://goo.gl/KcvCh and http://goo.gl/txIDp). It is true, those statements are mine, and I did provide them in email interviews. They're actually pretty close to the mark, but I'd like to add a few of my other statements that were omitted, just for context...

"All of the above (and more) lead to a state where many are forced to operate with aging infrastructure extended beyond its lifespan. Note however, that many staff at municipal utilities are actually remarkably dedicated and resourceful people. They have to be, given the circumstances." 

"The threat is somewhat exaggerated, but it is still very real. The vulnerabilities are underestimated."

Please understand that I'm not casting aspersions on the Municipal Utilities of the world. Some are further along the security maturity path than others, but I have worked with many of them and I find them to be amazing people and amazing organizations.