The Consensus Audit Guidelines (CAG) appear, at this point, to be a reasonable set of guidelines for mediating some human threats. I’m looking forward to seeing what CAG offers and have no doubt there will be worthwhile and actionable controls in the document. That said, there are significant reasons approach CAG with skepticism and assess it critically.

The motivation for CAG is described in a set of slides at the Gilligan Group site. It starts with a focus on what CIO’s fear most: attacks, reduced operational capability, public criticism, data loss, etc. Then it rightly questions whether FISMA is adequately addressing those problems. It doesn’t and this is the genesis of the CAG.

Consensus photo by Eirik Newth.

Unfortunately CAG subsequently develops by pairing this first valid premise with a set of false premises.  These propositions are drawn from slides at gilligangroupinc.com, attributed to John Gilligan or Alan Paller:

1. All that matters are attacks. The central tenet of Bush’s Comprehensive National Cyber Initiative (CNCI) is adopted as the CAG theme: “Defense Must Be Informed by the Offense”. CAG envisions security as defense against penetration attacks. As any seasoned security practitioner knows, attacks are a limited subset of the threats to confidentiality, integrity and availability that information and information systems face.
2. Security through obscurity. CAG seems to have taken the unspoken CNCI theme to heart too, “The most effective security is not exposed to public criticism.” Since its very public December 11th announcement no drafts have been made publicly available for comment.
3. False dichotomy. CAG has been promoted as an alternative to the OMB/NIST approach to FISMA. It isn’t. An alternative would target a fuller range of threats to information and information system security. CAG should be considered a complement to NIST guidance, an addendum of security controls focused on defense against penetration by hackers. NIST has even acted on this approach by including some CAG controls into the 800-53 Rev. 3 catalog of controls.
4. There is too much NIST guidance! This is the implication of one CAG slide that lists 1200 pages of guidance, 15 FIPS docs and the assorted Special Publications not related to FISMA as detriments to security. It’s like complaining that Wikipedia has too many articles to contribute to improved learning. Speaking as someone who scrambled to secure Federal systems before FISMA and NIST’s extensive guidance, having that documentation greatly improves my ability to efficiently and effectively secure systems.
5. NIST guidance doesn’t tell me how to secure my systems! NIST’s FISMA guidance doesn’t step you through securing your SQL Server. The Chairman of the Joint Chiefs also doesn’t deliver your milk. Why not? It’s not their job. NIST’s FISMA guidance helps you to assess the risks to the system, decide how to secure it, secure it accordingly, check that a minimum of controls are in place and then accept responsibility for operating the system. NIST also provides documents, checklists, repositories, standards, working groups and validation of automated tools that help with the actual security implementation.
6. Automated security controls negate human errors. With the premise of all threats being attacks this is nearly a plausible premise. But not all security is technical. Not all threats come from the Internet. DHS, NIST, Mitre, and their partners have pursued automated security controls to enforce and audit security controls for years but automated security controls can only go so far. Human errors, glitches, unexpected conflicts and operational requirements will always factor into the implementation of security.
7. Audit compatibility as a hallmark of good security. There is a conflict of focus at the heart of the CAG, it seeks to both improve its subset of security and improve audit compatibility. For technical controls this is somewhat achievable using automation, something NIST has pursued for years with government and industry partners. For operational and management controls it results in audit checklists. But audits are fundamentally concerned with testing the particular and repeatable, security needs focus on evaluating the whole to ensure the necessary security results. An audit sees if antivirus software is installed, an evaluation sees if the antivirus software is effective.
8. Metrics, but only these metrics over here. When selecting the current crop of CAG controls decisions on what to include were reportedly based on metrics of the highest threats. Great idea, a quantitative approach often discovers counter-intuitive facts. Only the metrics were cherry picked. Instead of looking at all realized threats or real threat impacts only a count of common penetration attacks were considered.
9. With a sample of 1. As a basis for determining what security should focus on the whole breadth of the security profession was queried, so long as they were penetration testers. Yes, penetration testers are some very smart and talented people but penetration testing is to security what HUMINT is to intelligence services. Important players, expert practitioners but limited in scope and best used in conjunction with other intelligence assets.
10. Assessments rely on paper artifacts. The NIST guidance does not require paper artifacts. The first line in the NIST SP 800-53A preface is, “Security control assessments are not about checklists, simple pass-fail results, or generating paperwork to pass inspections or audits-rather, security controls assessments are the principal vehicle used to verify that the implementers and operators of information systems are meeting their stated security goals and objectives.” NIST SP 800-37 specifically and repeatedly states, “Security accreditation packages can be submitted in either paper or electronic format.”

CAG is a missed opportunity. Of the myriad problems with our current FISMA regime a lot of good could be achieved. The problems with guidance have many causes but can be addressed through cooperative development of best practices outside of NIST. The Assessment Cases for SP 800-53A is an example of how cooperative development can achieve great results and provide clear guidance. Other problems exist and can be addressed with better training and community developments.

My hope is that the Consensus Audit Guidelines will move towards a more open, collaborative development environment. The first release is sure to deliver useful security controls against penetration attacks. As with all good security practices it will likely need to go through a few iterations and lots of critical assessment to mature. An open environment would help foster a more complete consensus.

Consensus photo by mugley.

• Clouds of CAG Confusion
• A Funny Thing Happened Last Week on Capital Hill
• How to Not Let FISMA Become a Paperwork Exercise
• When Standards Aren’t Good Enough
• “Machines Don’t Cause Risk, People Do!”

Scenario: American Internet connections are attacked.  In the resulting chaos, the Government fails to respond at all, primarily because of infighting over jurisdiction issues between responders.  Mass hysteria ensues–40 years of darkness, cats sleeping with dogs kind of stuff.

Sounds similar to New Orleans after Hurricane Katrina?  Well, this now has a name: Cyber-Katrina.

At least, this is what Paul Kurtz talked about this week at Black Hat DC.  Now I understand what Kurtz is saying:  that we need to figure out the national-level response while we have time so that when it happens we won’t be frozen with bureaucratic paralysis.  Yes, it works for me, I’ve been saying it ever since I thought I was somebody important last year.  =)

But Paul…. don’t say you want to create a new Cyber-FEMA for the Internet.  That’s where the metaphor you’re using failed–if you carry it too far, what you’re saying is that you want to make a Government organization that will eventually fail when the nation needs it the most.  Saying you want a Cyber-FEMA is just an ugly thing to say after you think about it too long.

What Kurtz really meant to say is that we don’t have a national-level CERT that coordinates between the major players–DoD, DoJ, DHS, state and local governments, and the private sector for large-scale incident response.  What’s Kurtz is really saying if you read between the lines is that US-CERT needs to be a national-level CERT and needs funding, training, people, and connections to do this mission.  In order to fulfill what the administration wants, needs, and is almost promising to the public through their management agenda, US-CERT has to get real big, real fast.

But the trick is, how do you explain this concept to somebody who doesn’t have either the security understanding or the national policy experience to understand the issue?  You resort back to Cyber-Katrina and maybe bank on a little FUD in the process.  Then the press gets all crazy on it–like breaking SSL means Cyber-Katrina Real Soon Now.

Now for those of you who will never be a candidate for Obama’s Cybersecurity Czar job, let me break this down for you big-bird stylie.  Right now there are 3 major candidates vying to get the job.  Since there is no official recommendation (and there probably won’t be until April when the 60 days to develop a strategy is over), the 3 candidates are making their move to prove that they’re the right person to pick.  Think of it as their mini-platforms, just look out for when they start talking about themselves in the 3rd person.

FEMA Disaster Relief photo by Infrogmation. Could a Cyber-FEMA coordinate incident response for a Cyber-Katrina?

And in other news, I3P (with ties to Dartmouth) has issued their National Cyber Security Research and Development Challenges document which um… hashes over the same stuff we’ve seen from the National Strategy to Secure Cyberspace, the Systems and Technology Research and Design Plan, the CSIS Recommendations, and the Obama Agenda.  Only the I3P report has all this weird psychologically-oriented mumbo-jumbo that when I read it my eyes glazed over.

Guys, I’ve said this so many times I feel like a complete cynic: talk is cheap, security isn’t.  It seems like everybody has a plan but nobody’s willing to step up and fix the problem.  Not only that, but they’re taking each others recommendations, throwing them in a blender, and reissuing their own.  Wake me up when somebody actually does something.

It leads me to believe that, once again, those who talk don’t know, and those who know don’t talk.

Therefore, here’s the BSOFH’s guide to protecting the nation from Cyber-Katrina:

• Designate a Cybersecurity Czar
• Equip the Cybersecurity Czar with an $100B/year budget
• Nationalize Microsoft, Cisco, and one of the major all-in-one security companies (Symantec)
• Integrate all the IT assets you now own and force them to write good software
• Public execution of any developer who uses strcpy() because who knows what other stupid stuff they’ll do
• Require code review and vulnerability assessments for any IT product that is sold on the market
• Regulate all IT installations to follow Government-approved hardening guides
• Use US-CERT to monitor the military-industrial complex
• ?????
• Live in a secure Cyber-World

But hey, that’s not the American way–we’re not socialists, damnit! (well, except for mortgage companies and banks and automakers and um yeah….)  So far all the plans have called for cooperation with the public sector, and that’s worked out just smashingly because of an industry-wide conflict of interest–writing junk software means that you can sell for upgrades or new products later.

I think the problem is fixable, but I predict these are the conditions for it to happen:

• Massive failure of some infrastructure component due to IT security issues
• Massive ownage of Government IT systems that actually gets publicized
• Deaths caused by massive IT Security fail
• Osama Bin Laden starts writing exploit code
• Citizen outrage to the point where my grandmother writes a letter to the President

Until then, security issues will be always be a second-fiddle to wars, the economy, presidential impeachments, and a host of a bazillion other things.  Because of this, security conditions will get much, much worse before they get better.

And then the cynic in me can’t help but think that, deep down inside, what the nation needs is precisely an IT Security Fail along the lines of 9-11/Katrina/Pearl Harbor/Dien Bien Fu/Task Force Smith.

• In Response to “Cyber Security Coming to a Boil” Comments….
• Inside the Obama Administration’s Cyber Security Agenda
• Cyber Security coming to a boil
• Blow-By-Blow on S.773–The Cybersecurity Act of 2009–Part 4
• A Layered Model for Massively-Scaled Security Management

The accreditation decision is one of the most key activities in how the US Government secures its systems. It’s also one of the most misunderstood activities. This slideshow aims to explain the role of the Authorizing Official and to give you some understanding into why and how accreditation decisions are made.

I would like to give a big thanks to Joe Faraone and Graydon McKee who helped out.

The presentation is licensed under Creative Commons, so feel free to download it, email it, and use it in your own training.

• Your Friendly Neighborhood C&A Podcast Panel
• Certification and Accreditation Seminar, March 30th and 31st
• Building A Modern Security Policy For Social Media and Government
• DojoCon 2009 Presentation
• Massively Scaled Security Solutions for Massively Scaled IT

Interesting article in Security Focus on President Obama and cybersecurity.  Yes, I complained on twitter because the “document on homeland security” is not really any kind of a solution, more like a bullet list of goals that sound suspiciously like a warmed-over campaign platform.

Guess what?  Every President does this, they put out their agenda for everyone to see.  With the last administration, it was the 5-point President’s Management Agenda.

Let’s be honest here, as Bubba the Infantryman would say, “There are only a couple of ways to suck an egg, and this egg has been around for a long time.”  Any cybersecurity strategy will harken back to the National Strategy to Secure Cyberspace because the problems are the same.  If you remember back to when the NStSC was first released, a horde of critics appeared out of the woodwork to say that there wasn’t enough implementation details and that the strategy wouldn’t be implemented because of that.  Well, they were partly right.

And now there’s the President stating his agenda with the same ideas that people have been saying for 6 years in more detail than what and suddenly it’s new and innovative.  That’s politics for you, folks.  =)  Bubba, in a rare fit of wisdom would say “The way you can tell the true pioneers is that they have arrows sticking out of their backs” and it might seem apropos here, if maybe a little bit cynical.

Hidden Agenda Eats Agenda photo by emme-dk.

Let’s go through each of the points with a little bit of analysis from myself:

Great idea.   Between OMB, NIST, DHS, DoD, DOJ, and a cast of thousands, there is a huge turf war over who really owns security.  Each of these groups do a phenomenal job doing what it is they do, but coordination between them is sometimes more like a semi-anarchist commune than a grand unified effort.  I seem to remember saying at one point that this was needed.  Granted, I was specifically talking about the internal side of the InfoSec Equitites Issue, so the scope here is a little different.

The Cyber Czar is literally buried deep down inside DHS with no real authority, a presidential advisor like is in the agenda would report directly to the President. 

We have a very good R&D plan in place (.pdf caveat), it just needs to be adopted and better funded.  For those of you who need a project, this is like a wishlist on things that some very smart Government guys are willing to fund.

Ouch, I cringe when I read this one.  Not that it’s needed because when it really comes down to it, every CISO in the US is dependent on the software and hardware vendors and their service providers.

Something the world outside the Beltway doesn’t understand is that “standards” are roughly equal to “regulation”.  It’s much, much better if the Government goes to industry groups and says “hey, we want these things to be part of a standard, can you guys work to put it all together?” There might be some regulation that is needed but it should be kept as small as possible.  Where the Government can help is to sponsor some of the standards and work along with industry to help define standards.

Maybe the best model for this is the age-old “lead the horse to water, demonstrate to the horse how to drink, hold the horses mouth in the water, and you still can’t get them to drink.”  We’ve tried this model for a couple of years, what is needed now is some kind of incentive for the horse to drink and for vendors to secure their hardware, software, firmware, and service offerings.

Maybe this gets down to political beliefs, but I don’t think this is the Government’s responsibility to prevent corporate cyber-espionage, nor should you as a company allow the Government to dictate how you harden your desktops or  where you put your IDS.  If you are not smart enough to be in one of these high-tech industries, you should be smart enough to keep your trade secrets from going offshore, or else you’ll die like some weird brand of corporate darwinism.

Government can prosecute evildoers and coordinate with other countries for enforcement efforts, which is exactly what you would expect their level of involvement to be.

Yes, in some cases when it’s cyber-espionage directed at the Government by hacking contractors or suppliers (the military-industrial complex), then Government can do something about it with trickle-down standards in contracts, and they usually do.  Think ITAR export controls scoped to a multi-national corporation and you have a pretty good idea of what the future will hold.

This point is interesting to me.  We already have rules to flag large transactions or multiple transactions, that’s how Elliot Spitzer got caught.  Untraceable Internet payment schemes sounds like pulp-fiction stuff and income tax tracking to me, I would like to know if they really exist.

On the other hand, law enforcement does need training.  There really is a shortage of people with the law enforcement and technical security backgrounds who can do investigations.

National data breach law == good, because it standardizes all of the state laws that are such a hodge-podge you need a full-time staff dedicated to breaking down incidents by jursidiction.  We have something like this proposed, it’s S.459 which just needs to be resurrected and supported by the Executive Branch as part of their agenda.

A common standard could be good as long as it’s done right (industry standards v/s Government regulation), see my comments above.

Note some key points I want you to take away:

Nothing is new under the sun.  These problems have been around a long time, they won’t go away in the next 4 years.  We have to build on the work of people who have come before us because we know they’ve looked at the problem and came to the same conclusions we will eventually come to.

Partnership is emphasized.  This is because as much lip-service we give to the Government solving our problems, the American Way (TM) is for the Government not to be your Internet Nanny.  Government can set the environment to support private information security efforts but it really is up to the individual companies to protect themselves.

Industry needs to solve its own problems.  If you want the Government to solve the nation’s information security problems, it means that we take US-CERT and have them monitor everything whether you want them to or not.  Yes, that’s where things are heading, folks, and maybe I just spilled the beans on some uber-secret plan that I don’t know about yet.  Trust me, you don’t want the transparency that the Government watching your data would provide.

Be careful what you ask for.  You just might get it.  When it comes to IT security, be extra careful because you’ll end up with regulation which means more auditors.

Agenda Grafitti photo by anarchosyn.

• Beware the Cyber-Katrina!
• Cyber Security coming to a boil
• Blow-By-Blow on S.773–The Cybersecurity Act of 2009–Part 3
• Blow-By-Blow on S.773–The Cybersecurity Act of 2009–Part 4
• Blow-By-Blow on S.773–The Cybersecurity Act of 2009–Part 1

Back in 1995 the junior high school students around the world were taken in by a sensationalized and carefully marketed hoax film called Alien Autopsy. Alien Autopsy was in fact a cheap film purporting to be actual footage of an actual autopsy of the cadaver of an extraterrestrial. The film was marketed as footage shot during the famous 1947 Roswell incident.

Alien Autopsy photo by jurvetson.

Well, back in 1995 I was in a mood for a good laugh so I popped up some popcorn, chilled a six-pack of Mountain Dew and kicked up my feet for a little silly entertainment. A couple of friends came over just in time for the show. So, I popped more popcorn, chilled more drinks and we all had a great time giggling, guffawing, and generally acting like a bunch of nitwits having some good clean fun.

Then in 2005, my wife asked if I could sit down with her to watch something called Grey’s Anatomy. Thinking that I was about to relive a guilty pleasure from ten years before, I readily agreed. Let’s face it, a show called Grey’s Anatomy could only be a sequel to the 1995 Alien Autopsy.

Well, having been fooled, I shared my mistake and agony with the guys at work the next day. To say the least, they were amused at the story but entirely at my expense. Some mistakes in life should just never be mentioned again.

I hope that is not the case with today’s comments. Today, I’d like to encourage you all to down load and read my paper on the History of Digital Forensics (.pdf caveat applies). It is based on a paper I presented at NIST’s annual digital forensics conference. However, since the slides from briefings do not read well, I converted the presentation to prose. Dissect it as you think appropriate. That is to say, let me know what you think.

• Digital Forensics: Who should make the keys?
• Ed Bellis’s Little SCAP Project
• Database Activity Monitoring for the Government
• Security Automation Developers Conference Slides
• Barcode Hacking