This is a multi-post series.  You are at post 1.  Read post 2 here.  Read Post 3 here.

This post begins with me.  For the past hour or so I’ve been working on a control-by-control objective analysis of SANS 20 Critical Security Controls.  This is a blog post I’ve had sitting “in the hopper” for 9 months or so.  And to be honest, I see some good things in the 20 CSC literature.  I think that, from a holistic perspective, the 20 CSC is an attempt at creating a prioritization of this huge list of stuff that I have to do as an information security officer–something that’s really needed.  I go into 20 CSC with a very open mind.

Then I start reading the individual controls.  I’m a big believer in Bottom-Line-Up-Front, so let me get my opinion out there now: 20 Critical Security Controls is crap.  I’m sorry John G and Eric C.  Not only is 20 CSC bad from a perspective of controls, metrics, and auditing tests, but if it’s implemented across the Government, it will be the downfall of security programs.  I really believe this.

Now on to the rationale….

Opportunity Costs. I can’t get that phrase out of my head.  And I’m not talking money just yet–I’m talking time.  See, I’m an IT security guy working for a contractor supporting a Government agency–just like 75% of the people out there.  I have a whole bunch of things to do–both in the NIST guidelines and organizational policy.  If you add anything else to the stack without taking anything away,  all you’ve done is to dilute my efforts.  And that’s why I can’t support 20 CSC–they’re an unofficial standard that does not achieve its stated primary goal of reducing the amount of work that I have to do.  I know they wanted to create a parallel standard focusing on technical controls but you have to have one official standard because if it’s not official, I don’t have to do it and it’s not really a standard anymore, it’s it?

Scoping Problems. We really have 2 tiers inside of an agency that we need to look at: the Enterprise and the various components that depend on the Enterprise.  Let’s call them… general support systems and major applications.  Now the problem here is that when you make a catalog of controls, some controls are more applicable to one tier than the other.  With 20 CSC, you run the classic blunder of trying to reinvent the wheel for every small system that comes along.

Threat Capabilities != Controls. And this is maybe the secret why compliance doesn’t work like we think it will.  In a nice theoretical world, it’s a threat-vulnerability-countermeasure coupling and the catalog of controls accounts for all likely threats and vulnerabilities.  Well, it doesn’t work that way:  it’s not a 1-to-1 ratio.  Typical security management frameworks start from a regulatory perspective and work their way down to technical details while what we really want to do is to build controls based on the countermeasures that we need.  So yeah, 20 CSC has the right idea here, the problem is that it’s a set of controls created by people who don’t believe in controls–the authors have the threat and vulnerability piece down and some of the countermeasures but they don’t understand how to translate that into controls to give to implementers and their auditors.  The 20 CSC guys are smart, don’t get me wrong, but I can’t help but get the feeling that they don’t understand how the “rest of the world” is getting their job done out there in the Enterprise.

The Mapping is Weak. There is a traceability matrix in the 20 CSC to map each control back to NIST controls.  It’s really bad, mostly because the context of 800-53 controls doesn’t extend into 20 CSC.  I have serious heartburn with how this is presented inside the agencies because we’re not really doing audits using the 20 CSC, we’re using the mapping of NIST controls with a weird subtext and it’s a “voluntary assessment” not an audit.

Guidelines?!?!?! This is basic stuff.  If it’s something you audit against, it *HAS* to be a standard.  Guidelines are recommendations and can add in more technique and education.  Standards are like hard requirements, they only work if they’re narrowly-scoped, achievable, and testable.  This isn’t specific to 20 CSC, the NIST Risk Management Framework (intended to be a set of guidelines also) suffers from this problem, too.  However, if your intent is to design a technical security and auditing standard, you need to write it like a standard.  While I’m up on a soapbox, for the love of $Diety, quit calling security controls “requirements”.

Auditor Limitations. Let’s face it, how do I get an auditor to add an unmanaged device to the network and know if we’ve detected it or not.  This is a classic mistake in the controls world:  assuming that we have enough people with the correct skillsets who can conduct intrusive technical tests without the collusion of my IT staff.

And the real reason why I dislike the 20 Critical Security Controls:

Introduction of “Audit Requirements”. One of the chief criticisms of the NIST Risk Management Framework is that the controls are not specific enough.  20 CSC falls into this trap of nonspecificity (Controls 7, 8, 9, and 15, I’m talking to you) and is not official guidance–a combination that means that my auditor has just added requirements to my workload simply because of how they interpret the control.  This is very dangerous and why I believe 20 CSC will be the end of IT security as we know it.

In future posts (I had to break this into multiple segments):

• Control-by-control analysis
• What 20 CSC got right (Hey, some of it is good, just not for the reasons that it’s supposed to be good)

SA-2 “Guideline” photo by cliff1066™.

• 20 Critical Security Controls: What They Did Right and What They Did Wrong
• 20 Critical Security Controls: Control-by-Control
• Workin’ for the ‘Counters: an Analysis of my Love-Hate Relationship with the CPAs
• Your Security “Requirements” are Teh Suxxorz
• How to Not Let FISMA Become a Paperwork Exercise