Well, several funny things happened, they happen every week.  But specifically I’m talking about the hearing in the House Committee on Homeland Security on FISMA reform–Federal Information Security: Current Challenges and Future Policy Considerations.  If you’re in information security and Government, you need to go read through the prepared statements and even watch the hearing.

Also referenced is HR.4900 which was introduced by Representative Watson as a modification to FISMA.  I also recommend that you have a look at it.

Now for my comments and rebuttals to the testimony:

• On the cost per sheet of FISMA compliance paper: If you buy into the State Department’s cost of $1700 per sheet, you’re absolutely daft.  The cost of a security program divided by the total number of sheets of paper is probably right.  In fact, if you do the security bits right, your cost per sheet will go up considerably because you’re doing much more security work while the volume of paperwork is reduced.
• Allocating budget for red teams: Do we really need penetration testing to prove that we have problems?  In Mike Smith’s world, we’re just not there yet, and proving that we’re not there is just an excuse to throw the InfoSec practitioners under the bus when they’re not the people who created the situation in the first place.
• Gus Guissanie: This guy is awesome and knows his stuff.  No, really, the guy is sharp.
• State Department Scanning: Hey, it almost seems like NIST has this in 800-53.  Oh wait, they do, only it’s given the same precedence as everything else.  More on this later.
• Technical Continuous Monitoring Tools: Does anybody else think that using products of FISMA (SCAP, CVE, CVSS) as evidence that FISMA is failing is a bit like dividing by zero?  We really have to be careful of this or we’ll destroy the universe.
• Number of Detected Attacks and Incidents as a Metric: Um, this always gets a “WTF?” from me.  Is the number increasing because we’re monitoring better or is it because we’re counting a whole bunch of small events as an attack (ie, IDS flagged on something), or is it because the amount of attacks are really increasing?  I asked this almost 2 years ago and nobody has answered it yet.
• The Limitations of GAO: GAO are just auditors.  Really, they depend on the agencies to not misrepresent facts and to give them an understanding of how their environment works.  Auditing and independent assessment is not the answer here because it’s not a fraud problem, it’s a resources and workforce development problem.
• OMB Metrics: I hardly ever talk bad about OMB, but their metrics suck.  Can you guys give me a call and I’ll give you some pointers?  Or rather, check out what I’ve already said about federated patch and vulnerability management then give me a call.

So now for Rybolov’s plan to fix FISMA:

1. You have to start with workforce management. This has been addressed numerous times and has a couple of different manifestations: DoDI 8570.10, contract clauses for levels of experience, role-based training, etc.  Until you have an adequate supply of clueful people to match the demand, you will continue to get subpar performance.
2. More testing will not help, it’s about execution. In the current culture, we believe that the more testing we do, the more likely the people being tested will be able to execute.  This is highly wrong and I’ve commented on it before.  I think that if it was really a fact of people being lazy or fraudulent then we would have fixed it by now.  My theory is that the problem is that we have too many wonks who know the law but not the tech and not enough techs that know the law.  In order to do the job, you need both.  This is also where I deviate from the SANS/20 Critical Security Controls approach and the IGs that love it.
3. Fix Plans of Actions and Milestones. These are supposed to be long-term/strategic problems, not the short-term/tactical application of patches–the tactical stuff should be automated.  The reasoning is that you use these plans for budget requests for the following years.
4. Fix the budget train. Right now the people with the budget (programs) are not the people running the IT and the security of it (CIO/CISO).  I don’t know if the answer here is a larger dedicated budget for CISO’s staff or a larger “CISO Tax” on all program budgets.  I could really policy-geek out on you here, just take my word for it that the people with the money are not the people protecting information and until you account for that, you will always have a problem.

Sights Around Capital Hill: Twice Sold Tales photo by brewbooks. Somehow seems fitting, I’ll let you figure out if there’s a connection. =)

• “Machines Don’t Cause Risk, People Do!”
• GAO’s 5 Steps to “Fix” FISMA
• How to Not Let FISMA Become a Paperwork Exercise
• In Other News, I’m Saying “Nyet” on S.3474
• The 10 CAG-egorically Wrong Ways to Introduce Standards

So by now NIST SP 800-37 R1 has made the rounds.  I want to take a couple of minutes to go over my theory on this update.

Summary of changes:

• Certification is gone.  Accreditation has now changed to “Authorization”.  This is interesting to me because it removes certification which I’ve always equated with compliance.
• There is more focus on continuous monitoring.
• NIST has made it more obvious that the process in 800-37 is the security aspects of a SDLC.
• There is much more more emphasis on enterprise-level controls.

So those of you out there who have been succeeding with the NIST Risk Management Framework  have been doing this all along, and it’s actually why you’ve succeeded.  For the rest of you, if you have to change your existing process, you’ve been doing it wrong.

Now for what’s missing and where you need to fill in the gaps:

• Prioritization of controls.  If everything is important, nothing is important.  You have to be able to determine which controls you need to succeed 100% of the time and which controls only need 75% reliability.  Hey, I even give credit to the SANS 20 Critical Security Controls, as flawed as they are, for this.
• Delineation of controls into shared/common, hybrid, and system-specific.  This is by design, it’s up to the departments and agencies to figure this out.  If you do this correctly, you save a ton of time and effort.  I remember the day my certifier told me that we didn’t recognize shared controls and that it was on me to provide evidence of controls that were provided at the enterprise–it still baffles me how you really expect one person on a project team to have the resources of the entire IT security staff.
• Continuous monitoring is up to you.  Along with prioritization, you have to determine which controls you need to monitor and a plan on how to do that.  Protip: these are usually technical controls that you can automate and should do so because it’s the only way to get the job done.
• Tailor, tailor, tailor.  It is not enough to use generic 800-53 controls.  It definitely is sub-par to use untailored 800-53A test procedures as your test plan.  These all depend on the implementation and need to be tailored to fit.

And finally, a shout-out to Dan Philpott at FISMAPedia.org.  Dan literally consumes new legislation, regulation, guidelines, and standards as they come out and annotates them with a wealth of analysis.

800-37 WordCloud by ME! Thanks to wordle.net for the tool to make it.

• SP 800-53A Now Finally Final
• NIST Cloud Conference Recap
• Special Publication 800-53 Revision 3 Workshop
• How to Not Let FISMA Become a Paperwork Exercise
• Some Comments on SP 800-39

OK, now for the control-by-control analysis of the 20 Critical Security Controls.  This is part 2.  Look here for the first installment.  Read part 3 here.

Critical Control 1: Inventory of Authorized and Unauthorized Devices. This is good: get an automated tool to do IT asset discovery.  Actually, you can combine this with Controls 2, 3, 4, 11, and 14 with some of the data center automation software–you know the usual suspects, just ask your ops folks how you get in on their tools.  This control suffers from scope problems because it doesn’t translate down to the smaller-system scale:  if I have a dozen servers in an application server farm inside of a datacenter, I’ll usually know if anybody adds something.  The metric here (detect all new devices in 24 hours) “blows goats” because you don’t know if you’re detecting everything.  A better test is for the auditor to do their own discovery scans and compare it to the list in the permanent discovery tool–that would be validation that the existing toolset does work–with a viable metric of “percentage of devices detected on the network”.  The 24 hour metric is more like a functional requirement for an asset discovery tool.  And as far as the isolation of unmanaged assets, I think it’s a great idea and the way things should be, except for the fact that you just gave us an audit requirement to implement NAC.

Critical Control 2: Inventory of Authorized and Unauthorized Software. Sounds like the precursor to whitelisting.  I think this is more apropos to the Enterprise unless your system is the end-user computing environment (laptops, desktops).  Yes, this control will help with stuff in a datacenter to detect when something’s been pwned but the real value is out at the endpoints.  So yes, not happy with the scope of this control.  The metric here is as bad as for Control 1 and I’m still not happy with it.  Besides, if you allow unauthorized software to be on an IT device for up to 24 hours, odds are you just got pwned.  The goal here should be to respond to detected unauthorized software within 24 hours.

Critical Control 3: Secure Configurations for Hardware and Software on Laptops, Workstations, and Servers. This is actually a good idea, provided that you give me a tool to apply the settings automagically because manual configuration sucks.  I think it’s about a dozen different controls all wrapped into one, it’s just trying to do too much in one little control.  The time-based metric for this control is really bad, it’s like watching a train wreck.  But hey, I’ll offer up my own: percentage of IT assets conforming to the designated configuration.  It’s hinted at in the implementation guide, make it officially the metric and this might be a control I can support.

Critical Control 4: Secure Configurations for Network Devices such as Firewalls, Routers, and Switches. This is basically Control 3 for network devices.  The comments there also apply here.

Critical Control 5: Boundary Defense. This control is too much stuff crammed into one space.  As a result, it’s not concise enough to be implemented–it’s all over the map.  In fact, I’ll go as far as to say that this isn’t really one control, it’s a control theme with a ton of controls inside of it.  The “audit requirements” here are going to utterly kill me as a security manager because there is so much of a disparity between the control and the actual controls therein.

Critical Control 6: Maintenance, Monitoring, and Analysis of Audit Logs. Some of this control should be part of Controls 3 and 4 because, let’s be honest here, you’re setting up logging on devices the way that the hardening guide says you should.  The part that’s needed in this control is aggregation of logs and review of logs–get them off all the endpoints and into a centralized log management solution.  This is mentioned as the last “advanced” implementation technique but if you’re operating a modern Enterprise, I don’t see how you can get the rest of the implementation done without some kind of SIEM piece.   I just don’t get the metric here, again with the 24 hours.  How about “percentage of devices reporting into the SIEM”?  Yeah, that’s the easy money here.  The testing of this control makes me do a facepalm:  “At a minimum the following devices must be tested: two routers, two firewalls, two switches, ten servers, and ten client systems.”  OK, we’ve got a LAN/WAN with 15000 endpoints and that’s all we’re going to test?

Critical Control 7: Application Software Security. You keep using those words, I do not think they mean what you think they mean.  Application security is a whole different world and 20 CSC doesn’t even begin to scratch the surface of it.  Oh, but guess what?  It’s a tie-in to the 25 Most Dangerous Programming Errors which is about all this control is:  a pointer to a different project.  The metric here is very weak because it’s not tied back to the actual control.

Critical Control 8: Controlled Use of Administrative Privileges. This should be part of Controls 3 and 4, along with something about getting an Identity and Access Management system so that you have one ID repository.  I know this is a shocker to you, but the metric here sucks.

Critical Control 9: Controlled Access Based on Need to Know. This is a great idea, but as a control it’s too broad to achieve, which is why the 20 CSC were created in the first place.  What do we really want here?  Network share ACLs are mentioned, which is a control in itself, but the rest of this is hazy and leaves much room for interpretation.  Cue “audit requirements” and the part where Rybolov says “If it’s this hazy, it’s not really a standard, it’s a guideline that I shouldn’t be audited against.

Critical Control 10: Continuous Vulnerability Assessment and Remediation. All-in-all, not too bad.  I would suggest “Average time to resolve scan findings” here as a metric or even something as “hoakey” as the FoundScan metric just to gauge overall trends.

Arm Control photo by Crotchsplay.

Critical Control 11: Account Monitoring and Control. Haven’t we seen this before?  Yep, this should be incorporated into Controls 8, 3, and 4.  However, periodic account reviews are awesome if you have the patience to do it.

Critical Control 12: Malware Defenses. OK, this isn’t too bad.  Once again, the metric sucks, but I do like some of the testing steps.  The way I would test this is to compare our system inventory with my total list of devices.  A simple diff later, we have a list of unmanaged devices.

Critical Control 13: Limitation and Control of Network Ports, Protocols, and Services. Host firewalls was not what I thought of… I’m thinking more like firewalls and network segmentation where you have to get change control approval to add a firewall rule.  As far as the host setup, this should be part of Control 3.

Critical Control 14: Wireless Device Control. Not bad, but this should be dumped into a technical standard that you use like a hardening guide.  Metric here still sucks, but I don’t really need to say this again… oh wait, I just did.

Critical Control 15: Data Loss Prevention. Puh-lease.  I’ll be the first to admit, I’m a big believer in DLP done right, and that it’s an awesome tool to solve some of the unique .  But I don’t think that the market is mature enough to add it into your catalog of controls.  Also this will fall flat on its face if your system is just a web application cluster:  DLP addresses the endpoints (desktops, laptops, mobiles) and the outbound gateways (email, web, etc).  The problem with this control is that if you don’t buy and implement a full DLP solution (cue Rich Mogull and his DLP guide), there isn’t anything else that has a similar capability.  This is one of those controls where the 800-53 mapping gets really creative–Good Ship Lollipop Creative because we’re tapdancing around the issue that DLP-type solutions aren’t specifically required in 800-53.

These controls don’t have automated ways to implement and test them:

Critical Control 16: Secure Network Engineering. This control is a steaming crater.  It’s very much a guideline instead of an auditable standard.

Critical Control 17: Penetration Tests and Red Team Exercises. Not bad.  Still too easy to shop around for the bargain-basement penetration test team.  But yeah, pretty good overall.

Critical Control 18: Incident Response Capability. Good control.  Hard to test/audit except to look at after-incident reports.

Critical Control 19: Data Recovery Capability. Not bad here.  Not real COOP/DR/ITCP but about on par with typical controls frameworks.

Critical Control 20: Security Skills Assessment and Appropriate Training to Fill Gaps. Good idea.  Hard to implement without something like 8570.10 to give you a matrix by job position.  You want to change the world here, give your own mapping in the control.

• Opportunity Costs and the 20 Critical Security Controls
• 20 Critical Security Controls: What They Did Right and What They Did Wrong
• Security Assessments as Fraud, Waste, and Abuse
• Federated Vulnerability Management
• The Guerilla CISO Rants: Don’t Write a System Security Plan

Every once in a while an opportunity presents itself to affect some real change in federal information security practice.  Now is such a time.  A slew of new NIST documents are being released between now and April.  These are the core NIST documents that describe how to satisfy FISMA.  They include NIST SPs 800-30 Revision 1, 800-39, 800-37 Revision 1 and 800-53A Revision 1. That’s where you come in.

The documents define what federal government practice will look like in the coming years.  If they are flawed then the practice will be flawed.  To prevent stupidity from leaking in when nobody is looking NIST releases the documents as drafts so everyone gets a chance to eyeball them.  First you eyeball, then you comment.  They look at the comments and they fix the flaws.  Fix the flaws now and you don’t live with them later.

The most important document in draft right now is the NIST Special Publication 800-37 Revision 1.  This document describes the central processes involved in the authorization of information systems that support the federal government.  Notice I didn’t say Certification and Accreditation?  That’s because C&A is deader than a sheep at a wolf convention. Want to know what replaces it?  Pick up a copy of NIST SP 800-37r1 FPD, give it a read and send in your comments.

Better yet, consider joining a formal document review process.  I’m leading a team of hale and hearty volunteers at OWASP in a NIST SP 800-37r1 FPD review and we’d love to have you come join the fun.   We’re on a tight schedule so now is the time to act.

Time is short, the comment period for NIST SP 800-37 Revision 1 FPD ends on December 31st, 2009.

• The Guerilla CISO Rants: Don’t Write a System Security Plan
• Observations on SP 800-37R1
• NIST Cloud Conference Recap
• William Jackson on FISMA: It Works, Maybe
• “Machines Don’t Cause Risk, People Do!”

A small presentation Dan Philpott and I put together for Potomac Forum about getting sane social media policy out of your security staff. I also recommend reading something I put out a couple of months ago about Social Media Threats and Web 2.0.

• DojoCon 2009 Presentation
• The Accreditation Decision and the Authorizing Official
• Massively Scaled Security Solutions for Massively Scaled IT
• Metricon 5 Wrapup
• Certification and Accreditation Seminar, March 30th and 31st

For those of you who didn’t know the real purpose of DojoCon, it was to raise money and awareness for Hackers for Charity. If you like anything that is in this post, go to HFC and make a donation of time, equipment, tech support, and maybe money. If you’ve never heard of HFC because you’re not one of the “InfoSec Cool Kids”, now is your chance–go read about them.

The video of my dojocon presentation. The microphone was off for the first couple of minutes but I look pretty animated.

And then the compliance panel that I tried not to dominate:

And finally, my slides are up on slideshare:

• Massively Scaled Security Solutions for Massively Scaled IT
• Building A Modern Security Policy For Social Media and Government
• Barcode Hacking
• Metricon 5 Wrapup
• DojoCon DDoS Video