Check out this slideshow and this workshop paper from 2006 on some ideas that NIST and a fairly large advisory panel have put together about certification of C&A service providers.  I’ve heard about this for several years now, and it’s been fairly much on a hiatus since 2006, but it’s starting to get some eartime lately.

The interesting thing to me is the big question of certifying companies v/s individuals.  I think the endgame will involve doing both because you certify companies for methodology and you certify people for skills.

This is the problem with certification and accreditation services as I see it today:

• Security staffing shortage means lower priority:  If you are an agency CISO and have 2 skilled people, where are you going to put them?  Odds are, architecture, engineering, or some other high-payoff activity, meaning that C&A services are candidates for entry-level security staff.
• Centralized v/s project-specific funding:  Some agencies have a “stable” of C&A staff, if it’s done wrong, you end up with standardization and complete compliance but not real risk management.  The opposite of this is where all the C&A activities are done on a per-project basis and huge repetition of effort ensues.  Basic management technique is to blend the 2 approaches.
• Crossover of personnel from “risk-avoidance” cultures:  Taking people from compliance-centric roles such as legal and accounting and putting them into a risk-based culture is a sure recipe for failure, overspending, and frustration.
• Accreditation is somewhat broken:  Not a new concept–teaching business owners about IT security risk is always hard to do, even more so when they have to sign off on the risk.
• C&A services are a commodity market:  I covered this last week.  This is pivotal, remember it for later.
• Misinformation abounds:  Because the NIST Risk Management Framework evolves so rapidly, what’s valid today is not the same that will be valid in 2 years.

So what we’re looking at with this blog post is how would a program to certify the C&A service providers look like.  NIST has 3 viable options:

• Use Existing Certs: Require basic certification levels for role descriptions.  DoD 8570.1M follows this approach.  Individual-level certification would be CAP, CISSP, CG.*, CISA, etc.  The company-level certification would be something like ITIL or CMMI.
• Second-Party Credentialing:  The industry creates a new certification program to satisfy NIST’s need without any input from NIST.  Part of this has already happened with some of the certifications like CAP.
• NIST-Sponsored Certification:  NIST becomes the “owner” of the certification and commissions organizations to test each other.

Now just like DoD 8570.1M, I’m torn on this issue.  On one hand, it means that you’ll get a higher caliber of person performing services because they have to meet some kind of minimum standard.  On the other hand, introducing scarcity means that there will be even less people available to do the job.  But the big problem that I have is that if you introduce higher requirements on commodity services, you’re squeezing the market severely:  costs as a customer go up for basic services, vendors get even less of a margin on services, more charlatans show up because you’ve tipped over into higher-priced boutique services, and mayhem ensues.

Guys, I’m not really a rocket scientist on this, but really after all this effort, it seems to me that the #1 problem that the Government has is a lack of skilled people.  Yes, certifying people is a good thing because it helps weed out the dirtballs with a very rough sieve, but I get the feeling that maybe what we should be doing instead is trying to create more people with the skills we need.  Alas, that’s a future blog post….

However, the last thing that I want to see happen is a meta-game of what’s going on with certifications right now–who certifies those who certify?  I think it’s a vicious cycle of cross-certification that will end up with the entire Government security industry becoming one huge self-licking ice cream cone.  =)

• Now ISC2 Blogs have an Opinion on FISMA
• When the Feds Come Calling
• Splunk Goes After the FISMA Lucre, They’re not Alone
• Comments on SCAP 2008
• “Machines Don’t Cause Risk, People Do!”

OK, whoever named this product should be shot:  Ashampoo Magical Security.

However, as much as I love sprinkling on the Magic FISMA Fairy Dust, “Magical Security” is craziness.

I won’t go into too much detail on hackers, shampoo, washing, and South Pacific.  I have a feeling I’ll get plenty of comments to that effect.

• Stands Alone
• An Open Letter to the Next President of the United States
• Blog Statistics and Search Strings
• A Little Advice From Mike and Lee
• Got Training?

Don’t you just love those technical security people who always need more security tools even though it doesn’t fit in with what everybody else is doing?

• OMB Makes it to LOLCAT Fame
• LOLCATS and #CapSec
• Lolcats take on Laws, Sausage, Cyberwhatzits, and PCI
• Beware the Audit Hammer
• The InfoSec D-List and IKANHAZFIZMA

I’ve spent a couple of days traveling around to agencies to teach.  It was fun but tiring, and the best part of it is that since I’m not teaching pure doctrine, I can include the “here’s how it works in real life” parts and some of the BSOFH parts–what I refer to as the “security management heretic thoughts”.

Some basic statements, the rest of this post will explain:

• C&A is a commodity market
• Security controls assessment is a commodity market
• PCI assessment is a commodity market
• Most MSSP (or rather, Security Device Management Service Providers) services are commodity markets

Now my boss said the first one to me about 4 months ago and it really needed some time for me to grasp the implications.  What we mean by “commodity market” is that since there isn’t really much of a difference between vendors, the vendors have to compete on having the lower price.

Now what the smart people will try to do is to take the commodity service and try to make it more of a boutique service by increasing the value.  Problem is that it only works if the customers play along and figure out how your service is different–usually what happens is you lose in the market simply because now you’re “too expensive”.

Where Boutique Sits by miss_rogue.

Since the security assessment world is a services business, the only way to compete in a commodity market is to pay your people less and try to charge more. But oh yeah, we compete on price, so that only leaves the paychecks as the way to keep the margin up.

Some ways that vendors will try to keep the assessment costs down:

• Hire cheaper people (yes, paper CISSPs)
• Try to reduce the engegement to a formula/methodlogy (ack, a checklist)
• It’s all about billability:  what percentage of your people’s time is not billable to clients? 
• Put people on assessments who have tangential skills just to keep them billable
• Use Cost-Plus-Margin or Time-Plus-Materials so that you can work more hours
• Use Firm-Fixed-Price contracts with highly reduced services ($150 PCI assessments)

Now inside Government contracting, there’s a fact that’s not known outside of the beltway:  your margins are fixed by the Government.  In other words, they only allow you to have around a 13-15% margin.  The way to make money is that the pie is a much bigger pie, even though you only get a small piece of it.  And yes, they do look at your accounting records and yes, there are loopholes, but for the most part, you can only collect this little margin.  If you stop and think about it, the Government almost forces the majority of its contractors into a commodity market.

Then we wonder why C&A engagements go so haywire…

The problem with commodity markets and vulnerability/risk/pen-test assessments is that your results, and by extension your ability to secure your data, are only as good as the skills and creativity of the people that the vendor sends.  Sounds like a problem?  It is.

So knowing this, how can you as the client get the most out of your service providers? This is a quick list:

• Every year (or every other), get an assessment from somebody who has a good reputation for being thorough (ie, a boutique)
• Be willing to pay more for services than the bottom of the market but be sure that you get quality people to go along with it, otherwise you’ve just added to the vendor’s margin with no real improvements to yourself
• Get assessments from multiple vendors across the span of a year or two–more eyes means different checklists
• Provide the assessors with your own checklists so you can steer them (tip from Dave Mortman)
• Self-identify vulnerabilities when appropriate (especially with vulnerabilities from previous assessments)
• Typical contracting fixes such as scope management, reviewing resumes of key personnel, etc
• Get lucky when the vendor hires really good people who don’t know how much they’re really worth (that was me 5 years ago)
• More than I’m sure will end up in the comments to this post  =)

And the final technique is that it’s all about what you do with the assessment results.  If you feed them into a mitigation plan (goviespeak: POA&M) and improve your security, it’s a win.

• Engagement Economics and Security Assessments
• Categories of Security Controls in Outsourcing
• Splunk Goes After the FISMA Lucre, They’re not Alone
• When the Feds Come Calling
• Split-Horizon Assessments and the Oversight Effect

 Credit to Dan on this one…

• TIC: Made for LOLCATS
• Agencies CISOs Star in LOLCATS!
• LOLCATS Take on POA&Ms
• Techno-Geek LOLCATS
• Preliminary Findings on Cybersecurity Review Now Out

Dear NIST People,

I have this semi-random digital scribbling thingie called a blog.  You might have heard of them.  Hey, you might have even at one point heard of mine.  =)

On my blog I let it be known that I am what the rest of the world would call a “NIST Cheerleader”.  I watch your every move.  I comment on your new publications.  I teach your framework every quarter.  From time to time, I criticize, but only because I have a foot in the theory of information security that you live and a foot in the implementation with agencies who know where the theory and models break.

The best thing that you have given us is not the risk management framework, it was SP 800-30, “Risk Management Guide for Information Systems”.  It’s small, to-the-point, and scalable from a single server to an entire IT enterprise.  Sure, the quants hate it, but for the quals and Government, it’s good enough.  I know private-sector organizations that use it.  One of my friends and blog readers/commenters was the guy who taught a group of people how to do risk assessment, then these same people went on to help you write the book.

I heard that you were in the process of revising SP 800-30.  While this is much needed to catch up/modernize, I want to make sure that 800-30 does not follow the “live by the catalog, die by the catalog” path that we seem to be following lately.  In other words, please don’t change risk assessment process to the following:

1. Determine boundary
2. Determine criticality
3. Conduct a gap assessment against a catalog of controls (SP 800-53/800-53A)
4. Attach a priority to mitigation
5. Perform risk avoidance because compliance models are yes/no frameworks
6. Document
7. ???
8. Profit!

At Your Own Risk Photo by  Mykl Roventine.

The reason that I am writing this is to let you know that I have noticed a disturbing trend in how now that we have a catalog of controls, the risk management framework is focusing more and more heavily on the catalog as the vehicle for determine an adequate level of security.  Some of this is good, some of this is not.

Why am I so concerned about this?  Well, inside the Government we have 2 conflicting ideas on information security:  compliance v/s risk management.  While we are fairly decent Government-wide at compliance management, the problem that we have is in risk management because risk management is only as good as the people who perform the risk assessment.  Not that we don’t have competent people, but the unknowns are what will make or break your security program, and the only way that you can known the unknowns is to get multiple assessments aimed at risks outside of the control catalog.

However, if you change the risk assessment process to a “catalog of controls gap analysis” process, then we’ve completely lost risk management in favor of compliance management.  To me, this is a disturbing trend that needs to be stopped.

Thank you for your time

–Rybolov

• Observations on SP 800-37R1
• Workin’ for the ‘Counters: an Analysis of my Love-Hate Relationship with the CPAs
• Reinventing FedRAMP
• It’s a Problem of Scale!
• Now ISC2 Blogs have an Opinion on FISMA