There is another challenge that SCAP addresses without it being officially on the SCAP program’s agenda.  With the advent of SCAP we now have a common reporting criteria by which we can now judge SCAP certified products.  If you have ever used an automated vulnerability scanner as part of a penetration test or security audit, you know that not all vulnerability scanners are created equal.  Some have much lower false positive alert and reporting rates than others.  Likewise, it is known that false negative alert and reporting rates vary.  And, because of the various technical approaches taken by the scanners, some provide much more consistent results. The challenge has been that without a common criteria to test against, it is difficult for a small or even fairly large security organization to find the resources to effectively test these products in a fair apples to apples test.

This is where NIST has a real opportunity on its hands.  With the release of the SCAP protocol, we have the criteria by which performance comparisons can be made.  What we are lacking is a common test environment.

Benchmark photo by bzo.

Let me veer off-topic for a moment to provide some background.  In the last few years the Linux community has created various “live distributions” for various specialized requirements.  What live distributions are, are CD, DVD or Flash-media-based operating systems that are executed upon boot.  That is to say that they boot and run directly from CD or DVD.  So, by using a Linux live distribution, you can run Linux off of you home Windows-based laptop without ever installing Linux to your hard disk.  This has opened up a world of specialized possibilities for this community.  One of them is the standardized training environment.  For example, security testers have created DVL (damn vulnerable Linux http://www.damnvulnerablelinux.org/).  DVL is a live distribution that with well documented security vulnerabilities, this distribution is used as a training aid for teaching vulnerability assessment and mitigation. There are other similar efforts created with the same intent such as the excellent DE-ICE training targets (http://de-ice.net/hackerpedia/index.php/De-ICE.net_PenTest_Disks).

NIST could follow-up on the release of the SCAP protocol by also building and releasing a common testing environment based perhaps on live distribution technology. Such an environment with well documented vulnerabilities would allow for the creation of objective benchmarks to be created to rate the accuracy, reproducibility, completeness of the results of SCAP certified vulnerability testing and reporting products.  This would aid government agencies, businesses and even individuals in their purchasing decisions.  It would also allow provide vendors with an objective and common test environment in which they can test and improve their products.  I admit this would be a significant undertaking for NIST.  However, I would suggest that such a test environment could be designed in such a manner that it could be built and released as a series of inter-operable modules based on live distribution technology.  The initial release might only offer a relatively modest set of tests but with the release of each module building on the results of previous releases, a highly demanding and sophisticated test environment could soon be realized.  Because of the importance and utility of such a project, industry and outside security experts might want to participate in and contribute to such an endeavor.

• NIST and SCAP; Busting a cap on intruders Part 1
• Ed Bellis’s Little SCAP Project
• Comments on SCAP 2008
• Federated Vulnerability Management
• How to Not Let FISMA Become a Paperwork Exercise

I was attending a conference at NIST (the National Institute of Standards) concerning the SCAP program (Security Content Automation Protocol; pronounced ESS-cap).  SCAP is focused on providing the Federal government with automated, common,  interoperable security solutions.  Specifically the SCAP program has developed a common set of standards for reporting security vulnerabilities for use in automated security scanners, security appliances and reporting systems.

Well, why do we need SCAP?  If we use the Godfather of all vulnerability management tools, the NESSUS vulnerability scanner as an example, we have seen that industry has produced a number of similar products.  Each has its own strengths and rich feature set.  However, none of them use the same “language” for detecting or describing or reporting a potential vulnerability.  This not only means that these various products can only be used to operate with each other with some measure of difficulty but, trying to aggregate and manage the result of reports from these systems can be tedious.

“Tim Bray at XML 2005” photo by Roland.

As a result of these efforts and vision of the dedicated employees at NIST, industry is already scrambling to get their related products SCAP certified.  And, Federal agencies are also specifying in contracts that products must be SCAP certified in order to be qualified for purchase.  This is real progress and great news for the tax payer who will get real better value for their tax dollar.  But, it is not a revolution — yet.  Where I see the revolution emerging is in six-month to a year time frame when industry takes note of the SCAP program and we begin to see SCAP certified and SCAP interoperable products being ordered.  It will not be long after that that we may see the SCAP protocol used in even consumer-level products like personal firewalls.  This ability to give us all a common language will significantly reduce the cost of building and supporting vulnerability scanners and vulnerability reporting tools.  This cost reduction will allow resources to be freed up to address prevention and mitigation concerns in a more meaningful manner.

For example, industry has tools that enable network and security support professionals to detect a mis-configuration in a desktop machine in their network and correct it.  But, only the largest and most well funded security IT security departments have such tools.  With the advent of SCAP, these kind of services will be much more affordable and supportable and thus more common.  In fact, because much of this can be automated, I can even envision the McAfee, Symantec, and others who are well placed in the vulnerability scanning market to offer support services over the wire to smaller businesses and to consumers.  Moreover, as this technology improves and becomes commoditized, I can see ISP’s offering security scanning and mediation as a service to their customers.

• NIST and SCAP; SCAP @ Large Part 2
• Security Automation Developers Conference Slides
• Comments on SCAP 2008
• Save a Kitten, Write SCAP Content
• Federated Vulnerability Management

I just got back from the SCAP 2008 conference at NIST HQ, and this is a collection of my thoughts in a somewhat random order:

Presention slides are available at the NVD website

I blogged about SCAP a year ago, and started pushing it in conversations with security managers that I came across.  Really, if you’re managing security of anything and you don’t know what SCAP is, you need to get smart on it really fast, if for no other reason than that you will be pitched it by vendors sporting new certifications.

Introduction to SCAP:  SCAP is a collection of XML schemas/standards that allow technical security information to be exchanged between tools.  It consists of the following standards:

• Common Platform Enumeration (CPE): A standard to describe a specific hardware, OS, and software configuration.  Asset information, it’s fairly humdrum, but it makes the rest of SCAP possible–think target enumeration and you’re pretty close.
• Common Vulnerabilities and Exposures (CVE): A definition of publicly-known vulnerabilities and weaknesses.  Should be familiar to most security researches and patch monkies.
• Common Configuration Enumeration (CCE): Basically, like CVE but specific to misconfigurations.
• Common Vulnerability Scoring System (CVSS): A standard for determining the characteristics and impact of security vulnerabilities.  Hmmm, sounds suspiciously like standardization of what is a high, medium, and low criticality vulnerability.
• Open Vulnerability and Assessment Language (OVAL):  Actually, 3 schemas to describe the inventory of a computer, the configuration on that computer, and a report of what vulnerabilites were found on that computer.
• Extensible Configuration Checklist Description Format (XCCDF): A data set that describes checks for vulnerabilities, benchmarks, or misconfigurations.  Sounds like the updates to your favorite vulnerability scanning tool because it is.

Hall of Standards inside NIST HQ photo by ME!!!

What’s the big deal with SCAP: SCAP allows data exchanges between tools.  So, for example, you can take a technical policy compliance tool, load up the official Government hardening policy in XCCDF for, say, Windows 2003, run a compliance scan, export the data in OVAL, and load the results into a final application that can help your CISO keep track of all the vulnerabilities.  Basically, imagine that you’re DoD and have 1.5 million desktops–how do you manage all of the technical information on those without having tools that can import and export from each other?

And then there was the Federal Desktop Core Configuration (FDCC): OMB and Karen Evans handed SCAP its first trial-by-fire.  FDCC is a configuration standard that is to be rolled out to every Government desktop.  According to responses received by OMB from the departments in the executive branch (see, Karen, I WAS paying attention =)   ), there are roughly 3.5 Million desktops inside the Government.  The only way to manage these desktops is through automation, and SCAP is providing that.

He sings, he dances, that Tony Sager is a great guy: So he’s presented at Black Hat, now SCAP 2008 (.pdf caveat).  Basically, while the NSA has a great red-team (think pen-test) capability, they had a major change of heart and realized, like the rest of the security world (*cough*Ranum*cough*), that while attacking is fun, it isn’t very productive at defending your systems–there is much more work to be done for the defenders, and we need more clueful people doing that.

Vendors are jumping on the bandwagon with both feet: The amount of uptake from the vulnerability and policy compliance vendors is amazing.  I would give numbers of how many are certified, but I literally get a new announcement in my news reader ever week or so.  For vendors, being certified means that you can sell your product to the Government, not being certified means that you get to sit on the bench watching everybody else have all the fun.  The GSA SAIR Smart-Buy Blanket Purchase Agreement sweetens the deal immensely by having your product easily purchasable in massive quantities by the Government.

Where are the rest of the standards: Yes, FDCC is great, but where are the rest of the hardening standards in cute importable XML files, ready to be snarfed into my SCAP-compliant tool?  Truth be told, this is one problem with SCAP right now because everybody has been focusing on FDCC and hasn’t had time yet to look at the other platforms.  Key word is “yet” because it’s happening real soon now, and it’s fairly trivial to convert the already-existing DISA STIGs or CIS Benchmarks into XCCDF.  In fact, Sun was blindsided by somebody who had made some SCAP schemas for their products and they had no idea that anybody was working on it–new content gets added practically daily because of the open-source nature of SCAP.

Changing Government role: This is going to be controversial.  With NVD/CVE, the government became the authoritative source for vulnerabilities.  So far that’s worked pretty well.  With the rest of SCAP, the Government changes roles to be a provider of content and configurations.  If NIST is smart, they’ll stay out of this because they prefer to be in the R&D business and not the operations side of things.  Look for DHS to pick up the role of being a definitions provider.  Government has to be careful here because they could in some instances be competing with companies that sell SCAP-like feed services.  Not a happy spot for either side of the fence.

More information security trickle-down effect: A repeated theme at SCAP 2008 is that the public sector is interested in what Big SCAP can do for them.  The vendors are using SCAP certification as a differentiator for the time being, but expect to see SCAP for security management standards like PCI-DSS, HIPAA, and SOX–to be honest here, though, most of the vendors in this space cut their teeth on these standards, it’s just a matter of legwork to be able to export in SCAP schemas.  Woot, we all win thanks to the magic that is the Government flexing its IT budget dollars!

OS and Applications vendors: these guys are feeling the squeeze of standardization.  On one hand, the smart vendors (Oracle, Microsoft, Sun, Cisco) have people already working with DISA/NSA to help produce the configuration guides, they just have to sit back and let somebody turn the guides into SCAP content.  Some of the applications vendors still haven’t figured out that their software is about to be made obsolete in the Government market because they don’t have the knowledge base to self-certify with FDCC and later OS standards.  With a 3-year lead time required for some of the desktop applications before a feature request (make my junk work with FDCC) makes it into a product release, there had better be some cluebat work going on in the application vendor community.  Adobe, I’m talking to you and Lifecycle ES–if you need help, just call me.

But how about system integrators: Well, for the time being, system integrators have almost a free ride–they just have to deal with FDCC.  There are some of them that have some cool solutions built on the capabilities of SCAP, but for the most part I haven’t seen much movement except for people who do some R&D.  Unfortunately for system integrators, the Federal Acquisition Regulation now requires that anything you sell to the Government be configured IAW the NIST checklists program.  And just how do you think the NIST checklists program will be implemented?  I’ll take SCAP for $5Bazillion, Alex.  Smart sytem integrators will at least keep an eye on SCAP before it blindsides them 6 months from now.

Technical compliance tools are destined to be a commodity: For the longest time, the vulnerability assessment vendors made their reputation by having the best vulnerability signatures.  In order to get true compatibility across products, standardized SCAP feeds means that the pure-play security tools are going to have less things to differentiate themselves from all the other tools and they fall into a commodity market centered on the accuracy of their checks with reduced false positives and negatives.  While it may seem like a joyride for the time being (hey, we just got our ticket to sell to the Gubmint by being SCAP-certified), that will soon turn into frustration as the business model changes and the margins get smaller.  Smart vendors will figure out ways to differentiate themselves and will survive, the others will not.

Which leads me to this: Why is it that SCAP only applies to security tools?  I mean, seriously, guys like BigFix and NetIQ have crossover from technical policy compliance to network management systems–CPE in particular.  What we need is a similar effort applied to network and data center tools.  And don’t point me at SNMP, I’m talking rich data.  =)  On a positive note, expect some of the security pure-play tools to be bought up and incorporated into enterprise suites if they aren’t already.

Side notes:

I love how the many deer (well over 9000 deer on the NIST campus) all have ear tags.  It brings up all sorts of scientific studies ideas.  But apparently the deer are on birth control shots or something….

Former Potomac Forum students:  Whattayaknow, I met some of our former students who are probably reading this right now because I pimped out my blog probably too aggressively.  =)  Hi Shawn, Marc, and Bob!

Old friends:  Wow, I found some of them, too.  Hi Jess, Walid, Chris, and a cast of thousands.

Deer on NIST Gaithersburg Campus photo by Chucka_NC.

• Sir Bruce Mentions FDCC, World Goes Nuts
• Security Automation Developers Conference Slides
• Some Words From a FAR
• NIST and SCAP; Busting a cap on intruders Part 1
• Ed Bellis’s Little SCAP Project

FAR: it’s the Federal Acquisition Regulation, and it covers all the buying that the government does.  For contractors, the FAR is a big deal–violate it and you end up blackballed from Government contracts or having to pay back money to your customer, either of which is a very bad thing.

In early August, OMB issued Memo 08-22 (standard .pdf caveat blah blah blah) which gave some of the administratrivia about how they want to manage FDCC–how to report it in your FISMA report, what is and isn’t a desktop, and a rough outline on how to validate your level of compliance.

Now I have mixed feelings about FDCC, you all should know that by now, but I think the Government actually did a decent thing here–they added FDCC (and any other NIST secure configuration checklists) to the FAR.

Check this section of 800-22 out:

On February 28, 2008, revised Part 39 of the Federal Acquisition Regulation (FAR) was published which reads:
PART 39-ACQUISITION OF INFORMATION TECHNOLOGY
1. The authority citation for 48 CFR part 39 continues to read as follows: Authority: 40 U.S.C. 121(c); 10U.S.C. chapter 137; and 42 U.S.C. 2473(c).
2. Amend section 39.101 by revising paragraph (d) to read as follows:
39.101 Policy.
* * * * *

(d) In acquiring information technology, agencies shall include the appropriate IT security policies and requirements, including use of common security configurations available from the NIST’s website at http://checklists.nist.gov. Agency contracting officers should consult with the requiring official to ensure the appropriate standards are incorporated.

Translated into English, what this means is that the NIST configurations checklists are coded into law for Government IT purchases.

This carries a HUGE impact to both the Government and contractors.  For the Government, they just outsourced part of their security to Dell and HP, whether they know it or not.  For the desktop manufacturers, they just signed up to learn how FDCC works if they want some of the Government’s money. 

Remember back in the halcyon days of FDCC when I predicted that one of the critical keys to success for FDCC was to be able to buy OEM desktops with the FDCC images on them.  It’s slowly becoming a reality.

Oh what’s that, you don’t sell desktops?  Well, this applies to all NIST configuration checklists, so as NIST adds to the intellectual property in the checklists program, you get to play too.  Looking at the DISA STIGs as a model, you might end up with a checklist for literally everything.

So as somebody who has no relation to the US Federal Government, you must be asking by now how you can ride the FDCC wave?  Here’s Rybolov’s plan for secure desktop world domination:

• Wait for the government to attain 60-80% FDCC implementation
• Wait for desktops to have an FDCC option for installed OS
• Review your core applications on the FDCC compatibility list
• Adopt FDCC as your desktop hardening standard
• Buy your desktop hardware with the image pre-loaded
• The FDCC configuration rolls uphill to be the default OS that they sell
• ?????
• Profit!

And the Government security trickle-down effect keeps rolling on….

Cynically, you could say that the OMB memos as of late (FDCC, DNSSEC) are very well coached and that OMB doesn’t know anything about IT, much less IT security.  You probably would be right, but seriously, OMB doesn’t get paid to know IT, they get paid to manage and budget, and in this case I see some sound public policy by asking the people who do know what they’re talking about.

While we have on our cynical hats, we might as well give a nod to those FISMA naysayers who have been complaining for years that the law wasn’t technical/specific enough.   Now we have very static checklists and the power to decide what a secure configuration should be has been taken out of the hands of the techies who would know and given to research organizations and bureaucratic organizations who have no vested interest in making your gear work.

Lighthouse From AFAR photo by Kamoteus.

• Comments on SCAP 2008
• No, FISMA Doesn’t Require That, Silly Product Pushers
• Federated Vulnerability Management
• Current Government Security Initiatives
• When the News Breaks, We Fix it…

Our friends at NIST are hosting a SCAP conference and workshop from the 22 to the 25th of September.  Go check it out here.  Registration runs until the 16th.

• NIST Security Automation Conference
• Save a Kitten, Write SCAP Content
• NIST and SCAP; Busting a cap on intruders Part 1
• Wanted: Some SCAP Wranglers
• Special Publication 800-53 Revision 3 Workshop

Ever feel lost and lonely when staring at the business end of an ST&E?  Confounded and confused considering Configuration controls?  Perplexed and Puzzled at Planning procedures?  Anxious or amazed at Audit and Accountability assessments?  Annoyed at aimless alliteration?

NIST has heard your muttered curses and answered them!  (Except the annoying alliteration, which is my fault.)

Now available are the Assessment Cases for Special Publication 800-53A.  The Assessment Cases offer supplemental guidance on assessing security controls found in the recently released SP 800-53A Guide for Assessing the Security Controls in Federal Information Systems (PDF Warning).  These documents are in their Initial Public Draft so be sure to give them a look and provide some feedback.

The Assessment Cases contain consensus recommendations from the Assessment Cases Project on specific actions to perform when assessing security controls.  These specific actions are intended to complement the assessment procedures documented in NIST SP 800-53A.   Yes, you heard that right, Specific Actions.  Less time spent pondering how to “Examine: … other relevant documents or records”.

The Assessment Cases Project is an inter-agency workgroup headed by DoJ with members including NIST, DoE, DoT and ODNI-CIO.  Many thanks for the hard work of this workgroup’s membership.  You may not be able to hear it but I am applauding on this side of the keyboard.  And a big thanks to Patrick O’Reilly for pointing me to this wonderful resource.

• How to Not Let FISMA Become a Paperwork Exercise
• SP 800-53A Now Finally Final
• An Open Letter to NIST About SP 800-30
• Special Publication 800-53 Revision 3 Workshop
• The Guerilla CISO Rants: Don’t Write a System Security Plan