Eh? What’s that mean?  Developer Days is a weeklong conference where they get down into the weeds about the various SCAP schemas and how they fit into the overall program of security automation. 

Highlights and new ideas:

Remedial Markup Language: Fledgeling schema to describe how to remediate a vulnerability.  A fully automated security system would scan and then use the RML content to automagically fix the finding… say, changing a configuration setting or installing a patch.  this would be much awesome if combined with the CVE/CWE so you have a vulnerability scanner that scans and fixes the problem.  Also needs to be kept in a bottle because the operations guys will have a heartattack if we are doing this without any human intervention.

Computer Network Defense: There is a pretty good scenario slide deck on using SCAP to automate hardening, auditing, monitoring, and defense.  The key from this deck is how the information flows using automation.

Common Control Identifier:  This schema is basically a catalog of controls (800-53, 8500.2, PCI, SoX, etc) in XML.  The awesomeness with this is that one control can contain a reference implementation for each technology and the checklist to validate it in XCCDF.  At this point, I get all misty…

Open Checklist Interactive Language: This schema is to capture questionaires.  Think managerial controls, operational controls, policy, and procedure captured in electronic format and fed into the regular mitigation and workflow tools that you use so that you can view “security of the enterprise at a glance” across technical and non-technical security.

Network Event Content Automation Protocol:  This is just a concept floating around right now on using XML to describe and automate responses to attacks.  If you’re familiar with ArcSight’s Common Event Format, this would be something similar but on steroids with workflow and a pony!

Attendance at developer days is limited, but thanks to all the “Powar of teh Intarwebs, you can go here and read the slides!

• Comments on SCAP 2008
• Save a Kitten, Write SCAP Content
• NIST and SCAP; Busting a cap on intruders Part 1
• Ed Bellis’s Little SCAP Project
• Federated Vulnerability Management

Letter from GAO on how Congress can fix FISMA.  And oh yeah, the press coverage on it.

Now supposedly this was in response to an inquiry from Congress about “Please comment on the need for improved cyber security relating to S.773, the proposed Cybersecurity Act of 2009.”  This is S.773.

GAO is mixing issues and has missed the mark on what Congress asked for.  S.773 is all about protecting critical infrastructure.  It only rarely mentions government internal IT issues.  S.773 has nothing at all to do with FISMA reform.  However, GAO doesn’t have much expertise in cybersecurity outside of the Federal Agencies (they have some, but I would never call it extensive), so they reported on what they know.

The GAO report used the often-cited metric of an increase in cybersecurity attacks against Government IT systems growing from “5,503 incidents reported in fiscal year 2006 to 16,843 incidents in fiscal year 2008” as proof that the agencies are not doing anything to fix the problem.  I’ve questioned these figures before, it’s associated with the measurement problem and increased reporting requirements more than an increase in attacks.  Truth be told, nobody knows if the attacks are increasing and, if so, at what rate.  I would guess they’re increasing, but we don’t know, so quit citing some “whacked” metric as proof.

Reform photo by shevy.

GAO’s recommendations for FISMA Reform:

Clarify requirements for testing and evaluating security controls.  In other words, the auditing shall continue until the scores improve.  Hate to tell you this, but really all you can test at the national level is if the FISMA framework is in place, the execution of the framework (and by extension, if an agency is secure or not) is largely untestable using any kind of a framework.

Require agency heads to provide an assurance statement on the overall adequacy and effectiveness of the agency’s information security program.  This is harkening back to the accounting roots of GAO.  Basically what we’re talking here is for the agency head to attest that his agency has made the best effort that it can to protect their IT.  I like part of this because part of what’s missing is “executive support” for IT security.  To be honest, though, most agency heads aren’t IT security dweebs, they would be signing an assurance statement based upon what their CIO/CISO put in the executive summary.

Enhance independent annual evaluations.  This has significant cost implications.  Besides, we’re getting more and more evaluations as time goes on with an increase in audit burden.  IE, in the Government IT security space, how much of your time is spent providing proof to auditors versus building security?  For some people, it’s their full-time job.

Strengthen annual reporting mechanisms.  More reporting.  I don’t think it needs to get strengthened, I think it needs to get “fixed”.  And by “fixed” I mean real metrics.  I’ve touched on this at least a hundred times, go check out some of it….

Strengthen OMB oversight of agency information security programs.  This one gives me brain-hurt.  OMB has exactly the amount of oversight that they need to do their job.  Just like more auditing, if you increase the oversight and the people doing the execution have the same amount of people and the same amount of funding and the same types of skills, do you really expect them to perform differently?

Rybolov’s synopsis:

When the only tool you have is a hammer, every problem looks like a nail, and I think that’s what GAO is doing here.  Since performance in IT security is obviously down, they suggest that more auditing and oversight will help.  But then again, at what point does the audit burden tip to the point where nobody is really doing any work at all except for answering to audit requests?

Going back to what Congress really asked for, We run up against a problem.  There isn’t a huge set of information about how the rest of the nation is doing with cybersecurity.  There’s the Verizon DBIR, the Data Loss DB, some surveys, and that’s about it.

So really, when you ask GAO to find out what the national cybersecurity situation is, all you’re going to get is a bunch of information about how government IT systems line up and maybe some anecdotes about critical infrastructure.

Coming to a blog near you (hopefully soon): Rybolov’s 5 steps to “fix” FISMA.

• A Funny Thing Happened Last Week on Capital Hill
• How to Not Let FISMA Become a Paperwork Exercise
• FISMA Report Cards Issued–Response is Rote by Now
• “Machines Don’t Cause Risk, People Do!”
• No, FISMA Doesn’t Require That, Silly Product Pushers

Face it, your security requirements suck. I’ll tell you why.  You write down controls verbatim from your catalog of controls (800-53, SoX, PCI, 27001, etc), put it into a contract, and wonder how come when it comes time for security testing, we just aren’t talking the same language.  Even worse, you put in the cr*ptastic “Contractor shall be compliant with FISMA and all applicable NIST standards”.  Yes, this happens more often than I could ever care to count, and I’ve seen it from both sides.

The problem with quoting back the “requirements” from a catalog of controls is that they’re not really requirements, they’re control objectives–abstract representations of what you need in order to protect your data, IT system, or business.  It’s a bit like brain surgery using a hammer and chisel–yes, it might work out for you, but I don’t really feel comfortable doing it or being on the receiving end.

And this is my beef with the way we manage security controls nowadays.  They’re not requirements, functionally they’re a high-level needs statement or even a security concept of operations.  Security controls need to be tailored into real requirements that are buildable, testable, measurable, and achievable.

Requirements photo by yummiec00kies.  There’s a social commentary in there about “Single, slim, and pleasant looking” but even I’m afraid to touch that one. =)

Did you say “Wrecks and Female Pigs’? In the contracting world, we have 2 vehicles that we use primarily for security controls: Statements of Work (SOW) and Engineering Requirements.

• Statements of Work follow along the lines of activities performed by people.  For instance, “contractor shall perform monthly 100% vulnerability scanning of the $FooProject.”
• Engineering Requirements are exactly what you want to have build.  For instance, “Prior to displaying the login screen, the application shall display the approved Generic Government Agency warning banner as shown below…”

Let’s have a quick exercise, shall we?

What 800-53 says: The information system produces audit records that contain sufficient information to, at a minimum, establish what type of event occurred, when (date and time) the event occurred, where the event occurred, the source of the event, the outcome (success or failure) of the event, and the identity of any user/subject associated with the event.

How It gets translated into a contract: Since it’s more along the lines of a security functional requirement (ie, it’s a specific functionality not a task we want people to do), we brake it out into multiple requirements:

The $BarApplication shall produce audit records with the following content:

• Event description such as the following:
  • Access the $Baz subsystem
  • Mounting external hard drive
  • Connecting to database
  • User entered administrator mode
• Date/time stamp in ‘YYYY-MM-DD HH:MM:SS’ format;
• Hostname where the event occured;
• Process name or program that generated the event;
• Outcome of the event as one of the following: success, warn, or fail; and
• Username and UserID that generated the event.

For a COTS product (ie, Windows 2003 server, Cisco IOS), when it comes to logging, I get what I get, and this means I don’t have a requirement for logging unless I’m designing the engineering requirements for Windows.

What 800-53 says: The The organization configures the information system to provide only essential capabilities and specifically prohibits and/or restricts the use of the following functions, ports, protocols, and/or services: [Assignment: organization-defined list of prohibited and/or restricted functions, ports, protocols, and/or services].

How It gets translated into a contract: Since it’s more along the lines of a security functional requirement, we brake it out into multiple requirements:

The $Barsystem shall have the software firewall turned on and only the following traffic shall be allowed:

• TCP port 443 to the command server
• UDP port 123 to the time server at this address
• etc…..

If we drop the system into a pre-existing infrastructure, we don’t need firewall rules per-se as part of the requirements, what we do need is a SOW along the following lines:

The system shall use our approved process for firewall change control, see a copy here…

So what’s missing, and how do we fix the sorry state of requirements?

This is the interesting part, and right now I’m not sure if we can, given the state of the industry and the infosec labor shortage:  we need security engineers who understand engineering requirements and project management in addition to vulnerability management.

Don’t abandon hope yet, let’s look at some things that can help….

Security requirements are a “best effort” proposition.  By this, I mean that we have our requirements and they don’t fit in all cases, so what we do is we throw them out there and if you can’t meet the requirement, we waiver it (live with it, hope for the best) or apply a compensating control (shield it from bad things happening).  This is unnerving because what we end up doing is arguing all the time over whether the requirements that were written need to be done or not.  This drives the engineers nuts.

It’s a significant amount of work to translate control objectives into requirements.  The easiest, fastest way to fix the “controls view” of a project is to scope out things that are provided by infrastructure or by policies and procedures at the enterprise level.  Hmmm, sounds like explicitly stating what our shared/common controls are.

You can manage controls by exclusion or inclusion:

• Inclusion:  We have a “default null” for controls and we will explicitly say in the requirements what controls you do need.  This works for small projects like standing up a pair of webservers in an existing infrastructure.
• Exclusion:  We give you the entire catalog of controls and then tell you which ones don’t apply to you.  This works best with large projects such as the outsourcing of an entire IT department.

We need a reference implementation per technology.  Let’s face it, how many times have I taken the 800-53 controls and broken them down into controls relevant for a desktop OS?  At least 5 in the last 3 years.  The way you really need to do this is that you have a hardening guide and that is the authoritative set of requirements for that technology.  It makes life simple.  Not that I’m saying deviate from doctrine and don’t do 800-53 controls and 800-53A test procedures, but that’s the point of having a hardening guide–it’s really just a set of tailored controls specific to a certain technology type.  The work has been done for you, quit trying to re-engineer the wheel.

Use a Joint Responsibilities Matrix.  Basically this breaks down the catalog of controls into the following columns:

• Control Designator
• Control Title
• Provided by the Government/Infrastructure/Common Control
• Provided by the Contractor/Project Team/Engineer

• When Standards Aren’t Good Enough
• Opportunity Costs and the 20 Critical Security Controls
• 20 Critical Security Controls: Control-by-Control
• 20 Critical Security Controls: What They Did Right and What They Did Wrong
• NIST Cloud Conference Recap

And by “We”, I mean the security industry as a whole.  And yes, this is your public-policy lesson for today, let me drag my soapbox over here and sit for a spell while I talk at you.

By “Survive”, I mean that we need some kind of self-regulatory framework that fulfills the niche that PCI-DSS occupies currently. Keep reading, I’ll explain.

And the “Why” is a magical phrase, everybody say it after me: self-regulatory organization.  In other words, the IT industry (and the Payment Card Industry) needs to regulate itself before it crosses the line into being considered for statutory regulation (ie, making a law) by the Federal Government.

Remember the PCI-DSS hearings with the House Committe on Homeland Security (AKA the Thompson Committee)?  All the Security Twits were abuzz about it, and it did my heart great justice to hear all the cool kids become security and public policy wonks at least for an afternoon.  Well, there is a little secret here and that is that when Congress gets involved, they’re gathering information to determine if they need to regulate an industry.  That’s about all Congress can do: make laws that you (and the Executive Branch) have to follow, maybe divvy up some tax money, and bring people in to testify.  Other than that, it’s just positioning to gain favor with other politicians and maybe some votes in the next election.

Regulation means audits and more compliance.  They go together like TCP and IP.  Most regulatory laws have at least some designation for a party who will perform oversight.  They have to do this because, well, if you’re not audited/assessed/evaluated/whatever, then it’s really an optional law, which doesn’t make sense at all.

Yay Audits photo by joebeone.

Another magical phrase that the public policy sector can share with the information security world: audit burden.  Audit burden is how much a company or individual pays both in direct costs (paying the auditors) and in indirect costs (babysitting the auditors, producing evidence for the auditors, taking people away from making money to talk to auditors, “audit requirements”, etc).  I think we can all agree that low audit burden is good, high audit burden is bad.  In fact, I think that’s one of the problems with FISMA as implemented is that it has a high audit burden with moderately tangible results. But I digress, this post is about PCI-DSS.

There’s even a concept that is mulling around in the back of my head to make a metric that compares the audit burden to the amount of security that it provides to the amount of assurance that it provides against statutory regulation.  It almost sounds like the start of a balanced scorecard for security management frameworks, now if I could get @alexhutton to jump on it, his quant brain would churn out great things in short order.

But this is the lesson for today: self-regulation is preferrable to legislation.

• Self-regulation is defined by people in the industry.  Think about the State Bar Association setting the standards for who is allowed to practice law.
• Standards ideally become codified versions of “best practices”.  OK, this is if they’re done correctly, more to follow.
• Standards are more flexible than laws.  As hard/cumbersome as it is to change a standard, the time involved in changing a law is prohibitive most of the time unless you’re running for reelection.
• Standards sometimes can be “tainted” to force out competition, laws are even more so.

The sad fact here is that if we don’t figure out as an industry how to make PCI-DSS or any other forms of self-regulation work, Congress will regulate for us.  Don’t like PCI-DSS because of the audit burden, wait until you have a law that requires you to do the same controls framework.  It will be the same thing, only with bigger penalties for failure, larger audit burdens to avoid the larger penalties, larger industries created to satisfy the market demand for audit.  Come meet the new regulatory body, same as the old only bigger and meaner. =)

However, self-regulation works if you do it right, and by right I mean this:

• The process is transparent and not the product of a secret back-room cabbal.
• Representation from all the shareholders.  For PCI-DSS, that would be Visa/MasterCard, banks, processors, large merchants, small merchants, and some of the actual customers.
• The standards committee knows how to compromise and come to a consensus.  IE, we can’t have both full hard drive encryption, a WAF, code review, and sacrificing of chickens in the server room, so we’ll make one of the 4 mandatory.
• The regulatory organization has a grievance process for its constituency to present valid (AKA “Not just more whining”) discrepencies in the standards and processes for clarification or consideration for change.
• The standard is “owned” by every member of the constituency.  Right now, people governed by PCI-DSS are not feeling that the standard is their standard and that they have a say in what comprises the standard and that they are the ones being helped by the standard.  Some of that is true, some of that is an image problem.  The way you combat this is by doing the things that I mentioned in the previous bullets.

Hmm, sounds like making an ISO standard, which brings its own set of politics.

While we need some form of self-regulation, right now PCI-DSS and ISO 27001 are the closest that we have in the private sector.  Yeah, it sucks, but it sucks the least, just like our form of government.

• Observations on PCI-DSS and Circular Arguments
• Metricon 5 Wrapup
• Blow-By-Blow on S.773–The Cybersecurity Act of 2009–Part 3
• Preliminary Findings on Cybersecurity Review Now Out
• “Machines Don’t Cause Risk, People Do!”

Ack, Plans of Action and Milestones.  I love them and I hate them.

For those of you who “don’t habla Federali”, a POA&M is basically an IOU from the system owner to the accreditor that yes, we will fix something but for some reason we can’t do it right now.  Usually these are findings from Security Test and Evaluation (ST&E) or Certification and Accreditation (C&A).  In fact, some places I’ve worked, they won’t make new POA&Ms unless they’re traceable back to ST&E results.

Functions that a POA&M fulfills:

• Issue tracking to resolution
• Serves as a “risk register”
• Used as the justification for budget
• Generate mitigation metrics
• Can be used for data-mining to find common vulnerabilities across systems

But today, we’re going to talk about POA&M abuse.  I’ve seen my fair share of this.

Conflicting Goals: The basic problem is that we want POA&Ms to satisfy too many conflicting functions.  IE, if we use the number of open POA&Ms as a metric to determine if our system owners are doing their job and closing out issues but we also turn around and report these at an enterprise level to OMB or at the department level, then it’s a conflict of interest to get these closed as fast as possible, even if it means losing your ability to track things at the system level or to spend the time doing things that solve long-term security problems–our vulnerability/weakness/risk management process forces us into creating small, easily-to-satisfy POA&Ms instead of long-term projects.

Near-Term v/s Long-Term:  If we set up POA&Ms with due dates of 30-60-90 (for high, moderate, and low risks) days, we don’t really have time at all to turn these POA&Ms into budget support.  Well, if we manage the budget up to 3 years in advance and we have 90 days for high-risk findings, then that means we’ll have exactly 0 input into the budget from any POA&M unless we can delay the bugger for 2 years or so, much too long for it to actually be fixable.

Bad POA&Ms:  Let’s face it, sometimes the one-for-one nature of ST&E, C&A, and risk assessment findings to POA&Ms means that you get POA&Ms that are “bad” and by that I mean that they can’t be satisfied or they’re not really something that you need to fix.

Some of the bad POA&Ms I’ve seen, these are paraphrased from the original:

• The solution uses {Microsoft|Sun|Oracle} products which has a history of vulnerabilities.
• The project team needs to tell the vendor to put IPV6 into their product roadmap
• The project team needs to implement X which is a common control provided at the enterprise level
• The System Owner and DAA have accepted this risk but we’re still turning it into a POA&M
• This is a common control that we really should handle at the enterprise level but we’re putting it on your POA&M list for a simple web application

Plan of Action for Refresh Philly photo by jonny goldstein.

Keys to POA&M Nirvana:  So over the years, I’ve observed some techniques for success in working with POA&Ms:

• Agree on the evidence/proof of POA&M closure when the POA&M is created
• Fix it before it becomes a POA&M
• Have a waiver or exception process that requires a cost-benefit-risk analysis
• Start with”high-level” POA&Ms and work down to more detailed POA&Ms as your security program matures
• POA&Ms are between the System Owner and the DAA, but the System Owner can turn around and negotiate a POA&M as a cedural with an outsourced IT provider

And then the keys to Building Good POA&Ms:

• Actionable–ie, they have something that you need to do
• Achievable–they can be accomplished
• Demonstrable–you can demonstrate that the POA&M has been satisfied
• Properly-Scoped–absorbed at the agency level, the common control level, or the system level
• They are SMART: Specific, Manageable, Attainable, Relevant, and within a specified Timeframe
• They are DUMB: Doable, Understandable, Manageable, and Beneficial

Yes, I stole the last 2 bullets from the picture above, but they make really good sense in a way that “know thyself” is awesome advice from the Oracle at Delphi.

• Security Assessments as Fraud, Waste, and Abuse
• Observations on SP 800-37R1
• Reinventing FedRAMP
• How to Not Let FISMA Become a Paperwork Exercise
• NIST Cloud Conference Recap

So how does the Guerilla-CISO staff communicate with the locals on jaunts to foreign lands such as Deleware, New Jersey, and Afghanistan?  The answer is simple, we use interpreters, known in infantrese as “terps”.  Yes, you might not trust them deep down inside because they harbor all kinds of loyalties so complex that you can spend the rest of your life figuring out, but you can’t do the job without them.

But in remembering how we used our interpreters, I’m reminded of some basic concepts that might be transferable to the IT security and risk management world.  Or maybe not, at least kick back and enjoy the storytelling while it’s free. =)

Know When to Treat Them Like Mushrooms: And by that, we mean “keep them in the dark and feed them bullsh*t”.  What really mean is to tell potentially adversarial people that you’re working with the least amount of information that they need to do their job in order to limit the frequency and impact of them doing something nasty.  When you’re planning a patrol, the worst way to ruin your week is to tell the terps when you’re leaving and where you’re going.  That way, they can call their Taliban friends when you’re not looking and they’ll have a surprise waiting for you.  No, it won’t be a birthday cake.  The way I would get a terp is that one would be assigned to me by our battalion staff and the night before the patrol I would tell the specific terp that we were leaving in the morning, give them a time that I would come by to check up on them, and that they would need to bring enough gear for 5 days.  Before they got into my vehicles and we rolled away, I would look through their gear to make sure they didn’t have any kind of communications device (radio or telephone) to let their buddies know where we were at.

Fudge the Schedule to Minimize Project Risk: Terps–even the good ones–are notorious for being on “local time”, which for a patrol means one hour later than you told them you were leaving.  The good part about this is that it’s way better than true local time, which has a margin of error of a week and a half.  In order to keep from being late, always tell the terps when you’ll need them an hour and a half before you really do, then check up on them every half hour or so.  Out on patrol, I would cut that margin down to half an hour because they didn’t have all the typical distractions to make them late.

Talk Slowly, Avoid Complex Sentences: The first skill to learn when using terps is to say things that their understanding of English can handle.  When they’re doing their job for you, simple sentences works best.  I know I’m walking down the road of heresy, but this is where quantitative risk assessment done poorly doesn’t work for me because now I something that’s entirely too complex to interpret to the non-IT crowd.  In fact, it probably is worse than no risk assessment at all because it comes accross as “consultantspeak” with no tangible link back to reality.

Put Your Resources Where the Greatest Risk Is: To a vehicle patrol out in the desert, most of the action happens at the front of the patrol.  That’s where you need a terp.  That way, the small stuff, such as asking a local farmer to move his goats and sheep out of the road so you can drive through, stays small–without a terp up front, a 2-minute conversation becomes 15 minutes of hassle as you first have to get the terp up to the front of the patrol then tell them what’s going on.

Pigs, Chicken, and Roadside Bombs: We all know the story about how in the eggs and bacon breakfast, the chicken is a participant but the pig is committed.  Well, when I go on a patrol with a terp, I want them to be committed.  That means riding in the front vehicle with me.  It’s my “poison pill” defense in knowing that if my terp tipped off the Taliban and they blow up the lead vehicle with me in it, at least they would also get the terp.  A little bit of risk-sharing in a venture goes a long way at getting honesty out of people.

Share Risk in a Culturally-Acceptable Way: Our terps would balk at the idea of riding in the front vehicle most of the time.  I don’t blame them, it’s the vehicle most likely to be turned into 2 tons of slag metal thanks to pressure plates hooked up to IEDs.  The typical American response is something along the lines of “It’s your country, you’re riding up front with me so if I get blown up, you do to”.  Yes, I share that ideal, but the Afghanis don’t understand country loyalties, the only thing they understand is their tribe, their village, and their family.  The Guerilla-CISO method here is to get down inside their heads by saying “Come ride with me, if we die, we die together like brothers”.  You’re saying the same thing basically but you’re framing it in a cultural context that they can’t say no to.

Reward People Willing to Embrace Your Risks: One of the ways that I was effective in dealing with the terps was that I would check in occassionally to see if they were doing alright during down-time from missions.  They would show me some Bollywood movies dubbed into Pashto, I would give them fatty American foods (Little Debbie FTW!).  They would play their music.  I would make fun of their music and amaze them because they never figured out how I knew that the song had drums, a stringed instrument, and somebody singing (hey, all their favorite songs have that).  They would share their “foot bread” (the bread is stamped flat by people walking on it before it’s cooked, I was too scared to ask if they washed their feet first) with me.  I would teach them how to say “Barbara (their assignment scheduler back on an airbase) was a <censored> for putting them out in the middle of nowhere on this assignment” and other savory phrases.  These forays weren’t for my own enjoyment, but to build rapport with the terps so that they would understand when I would give them some risk management love, Guerilla-CISO style.

Police, Afghan Army and an Interpreter photo by ME!.  The guy in the baseball cap and glasses is one of the best terps I ever worked with.

• Security Assessments as Fraud, Waste, and Abuse
• How to Not Let FISMA Become a Paperwork Exercise
• “Machines Don’t Cause Risk, People Do!”
• An Open Letter to NIST About SP 800-30
• Opportunity Costs and the 20 Critical Security Controls