You should have seen Special Publication 800-39 (PDF file, also check out more info on Fismapedia.org) out by now.  Dan Philpott and I just taught a class on understanding the document and how it affects security managers out them doing their job on a daily basis.  While the information is still fresh in my head, I thought I would jot down some notes that might help everybody else.

The Good:

NIST is doing some good stuff here trying to get IT Security and Information Assurance out of the “It’s the CISO’s problem, I have effectively outsourced any responsibility through the org chart” and into more of what DoD calls “mission assurance”.  IE, how do we go from point-in-time vulnerabilities (ie, things that can be scored with CVSS or tested through Security Test and Evaluation) to briefing executives on what the risk is to their organization (Department, Agency, or even business) coming from IT security problems.  It lays out an organization-wide risk management process and a framework (layer cakes within layer cakes) to share information up and down the organizational stack.  This is very good, and getting the mission/business/data/program owners to recognize their responsibilities is an awesome thing.

The Bad:

SP 800-39 is good in philosophy and a general theme of taking ownership of risk by the non-IT “business owners”, when it comes to specifics, it raises more questions than it answers.  For instance, it defines a function known as the Risk Executive.  As practiced today by people who “get stuff done”, the Risk Executive is like a board of the Business Unit owners (possibly as the Authorizing Officials), the CISO, and maybe a Chief Risk Officer or other senior executives.  But without the context and asking around to find out what people are doing to get executive buy-in, the Risk Executive seems fairly non-sequitor.  There are other things like that, but I think the best summary is “Wow, this is great, now how do I take this guidance and execute a plan based on it?”

The Ugly:

I have a pretty simple yardstick for evaluating any kind of standard or guideline: will this be something that my auditor will understand and will it help them help me?  With 800-39, I think that it is written abstractly and that most auditor-folk would have a hard time translating that into something that they could audit for.  This is both a blessing and a curse, and the huge recommendation that I have is that you brief your auditor beforehand on what 800-39 means to them and how you’re going to incorporate the guidance.

• Opportunity Costs and the 20 Critical Security Controls
• 20 Critical Security Controls: What They Did Right and What They Did Wrong
• NIST Cloud Conference Recap
• Ooh, “The Word” is out on S 3474
• Cloud Computing and the Government

Ah yes, I’ve explained this about a hundred times this week (at that thing that I can’t blog about, but @McKeay @MikD and @Sawaba were there so fill in the gaps), thought I should get this down somewhere.

the 3 factors that determine how much money you will make (or lose) in a consulting practice:

• Bill Rate: how much do you charge your customers.  This is pretty familiar to most folks.
• Utilization: what percentage of your employees’ time is spent being billable.  The trick here is if you can get them to work 50 hours/week because then they’re at 125% utilization and suspiciously close to “uncompensated overtime”, a concept I’ll maybe explain in the future.
• Leverage: the ratio of bosses to worker bees.  More experienced people are more expensive to have as employees.  Usually a company loses money on these folks because the bill rate is less than what they are paid.  Conversely, the biggest margin is on work done by junior folks.  A highly leveraged ratio is 1:25, a lowly leveraged ratio is 1:5 or even less.

Site Assessment photo by punkin3.14.

And then we have the security assessments business and security consulting in general.  Let’s face it, security assessments are a commodity market.  What this means is that since most competitors in the assessment space charge the same amount (or at least relatively close to each other), this means some things about the profitability of an assessment engagement:

• Assuming a Firm Fixed Price for the engagement, the Effective Bill Rate is inversely proportionate to the amount of hours you spend on the project.  IE, $30K/60 hours=$500/hour and 30K/240 hours = $125/hour.  I know this is a shocker, but the less amount of time you spend on an assessment, the bigger your margin but you would also expect the quality to suffer.
• Highly leveraged engagements let you keep margin but over time the quality suffers.  1:25 is incredibly lousy for quality but awesome for profit.  If you start looking at security assessment teams, they’re usually 1:4 or 1:5 which means that the assessment vendor is getting squeezed on margin.
• Keeping your people engaged as much as possible gives you that extra bit of margin.  Of course, if they’re spending 100% of their time on the road, they’ll get burned out really quickly.  This is not good for both staff longevity (and subsequent recruiting costs) and for work quality.

Now for the questions that this raises for me:

• Is there a 2-tier market where there are ninjas (expensive, high quality) and farmers (commodity prices, OK quality)?
• How do we keep audit/assessment quality up despite economic pressure?  IE, how do we create the conditions where the ninja business model is viable?
• Are we putting too much trust in our auditors/assessors for what we can reasonably expect them to perform successfully?
• How can any information security framework focused solely on audit/assessment survive past 5 years? (5-10 years is the SWAG time on how long it takes a technology to go from “nobody’s done this before” to “we have a tool to automate most of it”)
• What’s the alternative?

• Security Assessment Economics
• Split-Horizon Assessments and the Oversight Effect
• Security Assessments as Fraud, Waste, and Abuse
• The CyberArmy You Have…
• How to Not Let FISMA Become a Paperwork Exercise

Now I’ve been reasonably impressed with GovInfoSecurity.com and Eric Chabrow’s articles but this one supporting 20 CSC doesn’t make sense to me.  On one hand, you don’t have to treat your auditor’s word as gospel but on the other hand if we feed them what to say then suddenly it has merit?

Or is it just that all the security management frameworks suck and auditors remind us of that on a daily basis.  =)

However, it seems that there are 3 ways that people approach frameworks:

• From the Top–starting at the organization mission and working down the stack through policy, procedures, and then technology.  This is the approach taken by holistic frameworks like the NIST Risk Management Framework and ISO 27001/27002.  I think that if we start solely from this angle, then we end up with a massive case of analysis paralysis and policy created in a vacuum that is about as effective as it might sound.
• From the Bottom–starting with technology, then building procedures and policy where you need to.  This is the approach of the 20 Critical Security Controls.  When we start with this, we go all crazy buying bling and in 6 months it all implodes because it’s just not sustainable–you have no way to justify additional money or staff to operate the gear.
• And Then There’s Reality–what I really need is both approaches at the same time and I need it done a year ago. *sigh*

• SP 800-53A Now Finally Final
• Opportunity Costs and the 20 Critical Security Controls
• 20 Critical Security Controls: Control-by-Control
• A Niche to a Niche is Still Hard to Staff
• 20 Critical Security Controls: What They Did Right and What They Did Wrong

Going Off the Deep End

So I was thinking the other day (this is the part where people who know me in person usually go “oh cr*p”), partially spurred by a conversation I had with @csoandy and @secbarbie a couple of months ago.  I’ll get the idea out there: as an industry we need to embrace the concept of split-horizon assessments.

Two Purposes for Assessments

Because this is an insane approach that I’m just feeling out, let me go on a solo riff and explain what I’m talking about.  You see, I have two distinct purposes for getting a security assessment, both of which are in contention with each other:

• I want to fix my security by asking for money to fix the things that need attention.  When I get an assessment for this purpose, enumeration of my badness/suckness is good.  If I have a set of results that say that everything is great, then there’s no need for me to be given any more resources (time, money, people, gear).  Short-term, I’m fine, but what about my infrastructure-type long-term projects?  The net effect of a highly-scored annual assessment just might kill my program in 2 years as my funding and people are shifted elsewhere, especially in a .
• I want to keep my job and help my {company|agency|group} stay out of trouble by showing my zero-defects face and by demonstrating my due-diligence in protecting what has been given to me.  While the assessor has helped me short-term by identifying my problems and being a total hardass, if I’m not around in 6 months to adopt the recommendations into my security program, has the assessor actually helped me?

And this is the dilemma for just about every security manager out there.  One of the strategies is to alternate assessment types, but then your management wonder just what the heck it is you’re doing because you’re on top one year, then on the bottom the next.

Split Rock Lighthouse and Horizon photo by puliarf.

Assessor Window-Shopping

Now for the dirty little secret of the testing business:  there are really good testers who are the ninjas of the InfoSec world and there are really bad testers who don’t even validate their unlicensed Nessus scan.  I know, you’re shocked and it’s so blindingly obvious that Bruce Schneier will blog it 3 years from now.  =)

But there’s the part that you didn’t know:  security managers pick their assessor depending on the political mood inside their organization.  This is nowhere near a science, from what I’ve seen it involves a lot of navel-gazing on the part of the security team to see which is the lesser evil: having everybody think you’re incompetent or never getting anything new ever again?

Building a Better Rat Race

In order to accomplish both of the goals that I’ve listed, what I really need is a split-horizon assessment.  In other words, I need 2 reports from one assessment with different views for different audiences.  I know this sounds highly cynical, but it’s something we’ve been doing for some time now but just informally.  Might as well make it formal.

So are you sold on this concept yet?  In true form, I have an idea on how to get to a world of split-horizon assessments.  You can take any catalog of controls and divide it into “gotta have it” and “nice to have” (I almost divide these along the lines of “vulnerability mitigation” and “sustainable security program” or the “CISO” and “OMB and Congress”) buckets.  Then in your compliance assessment standard, require 2 reports for each assessment.  One is reported to the regulating authority and the other stays with the organization.

Indecision Strikes

I don’t know if I’ve solved the problemspace or not, but I’m looking for feedback “from the Peanut Gallery” so leave some comments.

• Security Assessment Economics
• Engagement Economics and Security Assessments
• Workin’ for the ‘Counters: an Analysis of my Love-Hate Relationship with the CPAs
• Security Assessments as Fraud, Waste, and Abuse
• A Funny Thing Happened Last Week on Capital Hill

Some days I feel like all this “continuous monitoring” talk around the beltway is just really a codeword for “buy our junk”, much like the old standby “defense in depth”, only instead of firewalls and IDS, it’s desktop and server configuration management.  Even better that it works for both products and services.  The BSOFH in me likes having a phrase like “Near Real-Time Continuous Compliance Monitoring” which can mean anything from “tying thermite grenades to the racks in case of being captured” to “I think I’ll make a ham sandwich for lunch and charge you for the privilege”.

Anyway, our IKANHAZFIZMA lolcats have finally found a control worth monitoring:  the world’s supply of overstuffed cheeseburgers.  This continuous monitoring thing is serious business, just like the Internets.

• Federal Computer Week and S.773
• Metricon is Next Week
• Oh Lookie, Somebody’s Doing What I Said To Do….
• CISOin’ Ain’t Easy, But It’s a Living
• Draft of SP 800-37 R1 is Out for Public Review

OK, since everybody seems to think that FISMA is some evil thing that needs reform, this is the version of events on “Planet Rybolov”:

Goals to surviving FISMA, based on all the criticisms I’ve read:

• Reduce paperwork requirements. Yes, some is needed.  Most is not.
• Reduce cost. There is much repetition in what we’re doing now, it borders on fraud, waste, and abuse.
• Increase technical effectiveness. IE, get from the procedural and managerial tasks and get down into the technical parts of security.

“Uphold our Values-Based Compliance Culture photo by kafka4prez.

So now, how do you keep from letting FISMA cripple you or turn into death-by-compliance:

• Prioritize. 25% of your controls need to not fail 100% of the time.  These are the ones that you test in-depth and more frequently.  Honestly, how often does your risk assessment policy get updated v/s your patch management?  Believe it or not, this is in SP 800-53R3 if you interpret it in the correct context.  More importantly, do not let your auditors dictate your priorities.
• Use common controls and shared infrastructure. Explicitly tell your system owners and ISSOs what you are providing as the agency CISO and/or the GSS that they are riding on.  As much as I hate meetings, if you own a General Support System (GSS), infrastructure (LAN/WAN, AD Forest, etc), or common controls (agency-wide policy, budget, Security Operations Center, etc), you have a fiduciary, legal, and moral obligation to get together with your constituency (the people who rely on the security you provide) and explain what it is you provide and allow them to tell you what additional support they need.
• Share Assessment Results. I’m talking about results from service providers with other agencies and systems.  We’re overtesting on the high-level stuff that doesn’t change and not on the detailed stuff that does change.  This is the nature of security assessments in that you start at the top and work your way down into the details, only most assessments don’t get down into the details because they’re busy reworking the top-level stuff over and over again.  Many years ago as a contractor managing infrastructure that multiple agencies used, it was unbelievably hard to get one agency to allow me to share security documents and assessment results with other agencies.  Shared assessment results mean that you can cut through the repetitious nature of what you’re doing and progressively get deeper into the technical, frequently-changing security aspects.
• Simplify the Paperwork. Yes, you still need to document what you’re doing, but the days of free-text prose and being graded on grammar and punctuation need to be over.  Do the controls section of System Security Plans as a Requirement Traceability Matrix.  More important than that, you need to go by-control by-component.  If you are hiring contractors and their job is to do copypasta directly from NIST documents and change the pronouns and tenses, you’re doing it wrong.  Don’t stand for that in your security policy or anything else that you do.
• Automate Wherever Possible. Note that the controls that change frequently and that need to not fail usually fit into this group.  It’s one of those “Things that make Rybolov go ‘Hmmmm'”.  Technology and automation provide both the problem and the solution.  Also see my first point up above.
• Fire 50% of Your Security Staff. Yes, I’m serious.  Those people you didn’t need anyway, primarily because they’re violating all the points I’ve made so far.  More importantly, 25 clueless people can mess things up faster than 5 clueful people can fix them, and that’s a problem for me.  Note that this does not apply to @csoandy, his headcount is A-OK.

The incredible thing to me is that this stuff is already there.  NIST writes “hooks” into their Special Publications to allow the smart people the room to do all these things.

And now the part where I hop up on my soapbox:  reforming FISMA by new legislation will not make any achievements above and beyond what we have today (with the exception of creating a CISO-esque position for the Exective Branch) because of the nature of audit and compliance.  In a public policy sense, the more items you have in legislation, the more the audit burden increases and the amount of repetition increases, and the amount of nonsense controls (ie, AntiVirus for Linux servers) increases.  Be careful what you ask for, you just might get it.

• “Machines Don’t Cause Risk, People Do!”
• Security Assessments as Fraud, Waste, and Abuse
• GAO’s 5 Steps to “Fix” FISMA
• When the Feds Come Calling
• Observations on SP 800-37R1