Letter from GAO on how Congress can fix FISMA.  And oh yeah, the press coverage on it.

Now supposedly this was in response to an inquiry from Congress about “Please comment on the need for improved cyber security relating to S.773, the proposed Cybersecurity Act of 2009.”  This is S.773.

GAO is mixing issues and has missed the mark on what Congress asked for.  S.773 is all about protecting critical infrastructure.  It only rarely mentions government internal IT issues.  S.773 has nothing at all to do with FISMA reform.  However, GAO doesn’t have much expertise in cybersecurity outside of the Federal Agencies (they have some, but I would never call it extensive), so they reported on what they know.

The GAO report used the often-cited metric of an increase in cybersecurity attacks against Government IT systems growing from “5,503 incidents reported in fiscal year 2006 to 16,843 incidents in fiscal year 2008” as proof that the agencies are not doing anything to fix the problem.  I’ve questioned these figures before, it’s associated with the measurement problem and increased reporting requirements more than an increase in attacks.  Truth be told, nobody knows if the attacks are increasing and, if so, at what rate.  I would guess they’re increasing, but we don’t know, so quit citing some “whacked” metric as proof.

Reform photo by shevy.

GAO’s recommendations for FISMA Reform:

Clarify requirements for testing and evaluating security controls.  In other words, the auditing shall continue until the scores improve.  Hate to tell you this, but really all you can test at the national level is if the FISMA framework is in place, the execution of the framework (and by extension, if an agency is secure or not) is largely untestable using any kind of a framework.

Require agency heads to provide an assurance statement on the overall adequacy and effectiveness of the agency’s information security program.  This is harkening back to the accounting roots of GAO.  Basically what we’re talking here is for the agency head to attest that his agency has made the best effort that it can to protect their IT.  I like part of this because part of what’s missing is “executive support” for IT security.  To be honest, though, most agency heads aren’t IT security dweebs, they would be signing an assurance statement based upon what their CIO/CISO put in the executive summary.

Enhance independent annual evaluations.  This has significant cost implications.  Besides, we’re getting more and more evaluations as time goes on with an increase in audit burden.  IE, in the Government IT security space, how much of your time is spent providing proof to auditors versus building security?  For some people, it’s their full-time job.

Strengthen annual reporting mechanisms.  More reporting.  I don’t think it needs to get strengthened, I think it needs to get “fixed”.  And by “fixed” I mean real metrics.  I’ve touched on this at least a hundred times, go check out some of it….

Strengthen OMB oversight of agency information security programs.  This one gives me brain-hurt.  OMB has exactly the amount of oversight that they need to do their job.  Just like more auditing, if you increase the oversight and the people doing the execution have the same amount of people and the same amount of funding and the same types of skills, do you really expect them to perform differently?

Rybolov’s synopsis:

When the only tool you have is a hammer, every problem looks like a nail, and I think that’s what GAO is doing here.  Since performance in IT security is obviously down, they suggest that more auditing and oversight will help.  But then again, at what point does the audit burden tip to the point where nobody is really doing any work at all except for answering to audit requests?

Going back to what Congress really asked for, We run up against a problem.  There isn’t a huge set of information about how the rest of the nation is doing with cybersecurity.  There’s the Verizon DBIR, the Data Loss DB, some surveys, and that’s about it.

So really, when you ask GAO to find out what the national cybersecurity situation is, all you’re going to get is a bunch of information about how government IT systems line up and maybe some anecdotes about critical infrastructure.

Coming to a blog near you (hopefully soon): Rybolov’s 5 steps to “fix” FISMA.

• A Funny Thing Happened Last Week on Capital Hill
• How to Not Let FISMA Become a Paperwork Exercise
• FISMA Report Cards Issued–Response is Rote by Now
• “Machines Don’t Cause Risk, People Do!”
• No, FISMA Doesn’t Require That, Silly Product Pushers

Rybolov really struck a note with me (as he usually does) with his blog entry with his decision that S.3474 was a bad thing. It reminds me of a conversation I had with a friend recently. Basically she ask me why bad thing happen even after smart people put their heads together and try to deal with the problem before facing a crisis. Intrigued with her question, I asked her what specifically she was asking about. She shared that she had been thinking about the tragedy of the Titanic sinking.

Of course she was referring to the sinking of the passenger ship RMS Titanic on the evening of 14 April 1912. She made two points, first that the experts declared that the ship was “unsinkable” – how could they be so wrong. Second, she wondered how the ship could be so poorly equipped with boats and safety equipment such that there was such great loss of life.

The Titanic’s Disaster photo by bobster1985.

Little did she know that I have had an odd fascination with the Titanic disaster since childhood and have basically read much of the common public material about the event. So, I replied that that no expert had ever declared her unsinkable, that it was basically something that was made up by the press and the dark spineless things that hang around the press. However, I added the designers and owners of the ship had made much of her advanced safety features when she was launched. A critical feature was including water-tight bulkheads in her design. This was something of an advanced and novel feature at the time. What it meant was that you could poke a pretty big hole in the ship, and as long as the whole was not spread over several of these water-tight compartments she would stay afloat. The problem was that the iceberg that she hit (the Titanic, not my friend), ignored all of this a tore a big gash along about a third of the length of the ship.

So, my friend pressed again about the lack of safety equipment, especially lifeboats. I told her that the problem here was that the Titanic indeed did meet all of the safety requirements of the time. And that a big part of the problem was that the safety requirements were drafted in 1894 at a time when there were rapid changes and in the size and design of ships of this kind. Those regulations indicated that all passenger ships over 10,000 tons required 16 life boats, and that’s how many the Titanic had. At the time the regulations were written there were hardly any ships over 10,000 tons in size. However, when Titanic was launched she was designed to be over 50,000 tons when fully loaded. The fact was that if each of these lifeboats was fully loaded they could barely hold half of the of the passengers and crew of the ship if fully loaded. What is worse, when the ship did sink, not all of the boats were usable because of speed and angle in which the ship began sinking.

So, the bottom-line was that when the Titanic was reviewed by the safety accountants, they took out their check-list and went over the ship with a fine tooth comb. When the day was done the ship fully met all the safety criteria and was certified as safe.

This is where I see the parallels between root causes of the Titanic disaster and the odd situation we find ourselves in today in terms of IT security. Security by checklist –especially out of date checklists—simply doesn’t work. Moreover, the entire mental framework that mixes up accounting practices and thoughts with security discipline and research is an utter failure. Audits only uncover the most egregious security failures. And, they uncover them at a point in time. The result is that audits can be gamed, and even ignored. On the other hand, formal reviews by experienced security professionals are rarely ignored. Sometimes not all of the resources are available to militate against some of the vulnerabilities pointed out by the professionals. And sometimes there is debate about the validity of specific observations made by security professionals. But, they are rarely ignored.

Interesting enough, because of the mixed IT security record of many government agencies, Congress is proposing – more audits! It seems to me what they should be considering is strengthening the management of IT security and moving from security audits often performed by unqualified individuals and teams toward security assessments conducted by security professionals. And since professionals are conducting these proposed assessments, they should be required to comment on the seriousness of deficiencies and possible mitigation actions. An additional assessment that the professionals should be required to report on is the adequacy of funding, staffing and higher management support. I don’t really see any point in giving a security program a failing grade if the existing program is well managed but subverted and underfunded by the department’s leadership.

• FedRAMP: It’s Here but Not Yet Here
• In Other News, I’m Saying “Nyet” on S.3474
• GAO’s 5 Steps to “Fix” FISMA
• Blow-By-Blow on S.773–The Cybersecurity Act of 2009–Part 2
• “Machines Don’t Cause Risk, People Do!”

No big surprise by now, I work for an accounting firm.  Oh, what’s that?  Oh yes, that’s right, it’s a consulting firm with a high percentage of accountants, including a plethora of CPAs.  “Accounting firm” is so 1950s-ish. =)

It’s my secret theory (well, not so much of a secret now, just between the Internet and me) that the primary problem we have in information security is that as a field we have borrowed heavily from public accounting.  The only problem is that public accounting is different from what we do.

Goals for public accounting run something like this:

• Eliminate fraud through oversight
• Protect the company’s money from rogue agents
• Protect the shareholders of public companies
• Ensure accountability of actions

Accounting for Non-Accountants photo by happyeclair.

As a result of their goals, accountants have an interesting set of values:

• Signatures are sacred
• Separation of duties is sacrosanct
• Auditing is designed to act as a deterrent to fraud
• “Professional Skepticism” is a much-valued trait
• Zero-Defects is a good condition

In other words, accountants live in a panopticon of tranparency, the concept being that through oversight and transparency, people will not become evildoers and those that do will be caught.  Pretty simple idea, makes me think about IDS in an entirely new light.

Words that accountants use that mean something entirely different from the way you or I use them:

• Fraud, Waste, and Abuse: They’re talking about spending money, I’m usually talking about people doing something ethically wrong.
• Investigation: They’re looking at the numbers to see how a particular number was created.  Me, I bring the nice people with guns when I do an investigation.
• Incident: Their version is what I would call an event.  When I call something an incident, we’re headed towards an investigation.
• Security test and evaluation: To them, it’s a compliance audit.  To me, it’s determining the frequency that the system will fail and if we have a way to fix it once it does.  Remember this, it’s a critical difference.
• Control: I think their version has something to do with having oversight and separation of duties.  Me, when I see this word, I think “countermeasure to a specific threat and vulnerability”.
• Audit: An activity designed to prove that fraud has not happened.  Usually we don’t use the word unless we absolutely have to.
• Technical: They’re talking about the highly-detailed accounting rules.  I’m talking about if you know how to build your own server and OS using lumps of raw silicon and a soldering iron.
• Checklist: They’re talking about a sacred list that condenses all the rules into an easily-auditable format.  Me, I’m thinking that a checklist is something that will fail because my threats and their capabilities don’t fit into nice little lists.
• Forensics: Their version is what I would call “research to find out where the money went to” and involves looking at a bunch of numbers.  My version has something to do with logs, memory dumps, and hard drive images.
• Risk Management: This has something to do with higher interest rates for high-risk loans.  For me, it’s looking for countermeasures and knowing what things to skimp on even though the catalog of controls says you have to have it.

In short, pretty much anything they could say about our line of work has a different meaning.  This is why I believe it’s a problem if we adopt too much of their methodology and management models because they are doing similar activities to what security people do, only for different purposes.

In order to understand the mentality that we’re working with, let’s give you a couple of scenarios:

After-Work Optional Training Session: The accountants not only make you put your name on the attendance roster but you have to sign it as well.  Are they worried that you’re committing fraud by showing up at training that you were not supposed to, so they need some sort of signature nonrepudiation to prove that you were there?  No!  They just make you sign it because they believe in the power of the signature and that’s just how they do things, no matter how trivial.

The Role of Security: To an accountant, the role of security in an organization is to reduce fraud by “hack-proof” configurations and monitoring.  This is a problem in that since security is economics, we’re somehow subordinate to the finance people.

Let’s look at the world of the typical security practitioner:

• The guidance that security professionals have is very contradictory, missing, or non-relevant.
• Really what we do comes down to risk management, which means that sometimes it makes more sense to break the rules (even though there is a rule that says break the rules, which should freak your brain out by now if you’re an accountant).
• We have a constantly changing environment that rules cannot keep up with.

Now this whole blog post, although rambling on about accountants, is aimed at getting a message across.  In the US Federal Government, we use a process called certification and accreditation (C&A).  The certification part is pretty easy to understand–it’s like compliance, do you have it and does it work.  CPAs will readily understand that as a controls assessment.  That’s very much a transferable concept.

But in accreditation, you give the risks to a senior manager/executive and they accept the risks associated with operating the system.  The CPA’s zero-defects world comes through and they lie on the ground doing the cockroach.  Their skills aren’t transferable when dealing with risk management, only compliance with a set of rules.

Once again, the problem with security in Government is that it’s cultural.

And don’t get me wrong, I like accountants and they do what I do not have neither the skills nor the desire to do.  I just think that there aren’t as many transferable skills between our jobs as there might seem on the surface.

• White Professional Male Turned CISSP Seeks Mega-Sized Security Management Model
• An Open Letter to NIST About SP 800-30
• How to Not Let FISMA Become a Paperwork Exercise
• Security Assessment Economics
• Split-Horizon Assessments and the Oversight Effect

So what exactly is a “system”?  After all this time, it’s still probably one of the most misunderstood ways that we manage security in the Government.

The short answer is this:  a system is what you say it is.  Long answer is it depends on the following factors:

• Maturity of your agency
• Budget processes and Exhibit 300s
• The extent of your common controls
• Political boundaries between inter-agency organizations
• Agency missions
• Amount of highly-regulated data such as PII or financial

Yes, this all gets complicated.  But really, whatever you say is a system is a system, the designation is just for you so you can manage the enterprise in pieces.  There are 3 main techniques that I use to determine what is a system:

• As a budget line-item: If it has an Exhibit 300, then it’s a system.  This works better for Plan of Actions and Milestones (POA&Ms) but in reality there might not be a 1:1 correllation between systems and Exhibit 300s.
• As a data type: If it has a particular type of data, then it’s a system.  This works well for special-purpose systems or where a type of data is regulated, such as PII or financial data.
• As a project or program: if it’s the same people that built it and maintain it, then it’s a system.  This dovetails in nicely with any kind of SDLC or with any kind of outsourcing.

Inventory photo by nutmeg.

Inventory management techniques that work:

• Less systems are better.  Each system incurs overhead in effort and cost.
• More systems works when you have no idea what is out there, but will cripple you in the long term because of the overhead.
• Start with many systems, assess each as its own piece, then consolidate them into a general support system or common controls package.
• Set a threshold for project size in either pieces of hardware or dollar value.  If the project exceeds that threshold, then it’s a system.
• Determine if something will be a system when the budget request is made.  Good CISOs realize this and have a place on the investment control board or capital planning investment board.

Guerilla CISO war story time:

Way back when all this was new, one of the agency CISOs would have a roundtable every quarter or so.  Won’t name who, but some of my blog readers do.  Almost every meeting devolved at some point into the time-honored sticking point of “what is a system?”  Everybody wanted to know if they had “2 servers, 3 PCs, a database, a dog, and a dickfore”, was that a system.  After one too many iterations, the gray-hair in the group would put up “Exhibit 300=System” on the whiteboard before every meeting.  Then when the inevitable conversation of “what is a system?” would come up, he would just point to the board.

And another story:

Several years ago I was working an IT outsourcing contract with an inventory that was determined using the budget line-item technique.  Turned out we had all sorts of systems, some of which didn’t make sense, like the desktop client to manage the local admin account.  One of my first priorities was to consolidate as many systems as I could.  Not that I was altruistic about saving money or anything, it was that the less systems I had, the less paperwork needed to be generated. =)   Most of the systems I rolled up into a general support system aimed at basic user connectivity.

• When the Feds Come Calling
• The Guerilla CISO Rants: Don’t Write a System Security Plan
• New SP 800-60 is Out, Categorize Yerselves Mo Better
• How Much Security Should the Outsourcer Do?
• Needed: Agency CSOs

I’ve touched on this about a bazillion times, let me start today with a very simple statement:  due to the scale of the US Government, we cannot find enough skilled security people.

Part of the problem is that good security people need to know the following skills:

• IT technology: since the data more often than not is in a computer, you need to understand them
• People technology: policies and procedures for managing people
• Business sense:  understanding that you’re supporting business goals
• And for Government:  politics

Back when I was PFC Rybolov, my battalion commander told me something along the lines of “The intelligence world is a hard job, you have to be able to out-infantry the infantry, out-mechanic the mechanics, out-radio the radio guys, and you need to know a language.”  Security is pretty much the same thing–you have to out-techie the techies, out-business the MBAs, and out-jerkify the auditors.  =)

Sound complicated?  Yes, it is, and it’s hard to find people who can do all this.  IT is an employment niche, IT security is a niche to a niche.  And there isn’t enough people who have the experience to do it.

So how do we mitigate the staffing shortage?  Here is what we are doing today in the Government:

• CyberCorps scholarship program for undergrads and graduate students with a minimum government service obligation.
• Using other career fields in “crossover roles”–yes, accountants can be used for some light security tasks.  Some things that we think of as security are really Quality Assurance and Change Control jobs that we have a vested interest in making work.
• Using contractors in some roles such as ISSO, ISSM, etc.
• Automation as much as possible.  Technical is easier, the policy and procedures side takes longer.  What you’ll find out eventually is that good IT management is good security management.
• Hanging on methodologies to “automate” the process side of security.

Now this is cool and all, but it’s hard to sustain and really hard to justify as a long-term solution.  In order to support the Government, we need to create more people.  Cybercorps is a start, but the need is so much larger than the supply that we have to consider better ways to create Government security dweebs.

Do we need Security Awareness and Training?  Yes we do, but much more than what is being provided (think system administrator training and procurement specialist training, not end-user training), and as an internal recruiting pipeline.  Still, I don’t think that we can recruit enough people to “the dark side” and that we need to look outside the Beltway for people.  Problem is that DC is such an insular community and we don’t speak the same language as the rest of the world.

• NIST’S FISMA Pase II–Who Certifies Those who Certify the Certifiers?
• A Short History of Cyberwar Lookalikes
• Security Assessments as Fraud, Waste, and Abuse
• Cloud Computing and the Government
• In Response to “Cyber Security Coming to a Boil” Comments….

Check out this slideshow and this workshop paper from 2006 on some ideas that NIST and a fairly large advisory panel have put together about certification of C&A service providers.  I’ve heard about this for several years now, and it’s been fairly much on a hiatus since 2006, but it’s starting to get some eartime lately.

The interesting thing to me is the big question of certifying companies v/s individuals.  I think the endgame will involve doing both because you certify companies for methodology and you certify people for skills.

This is the problem with certification and accreditation services as I see it today:

• Security staffing shortage means lower priority:  If you are an agency CISO and have 2 skilled people, where are you going to put them?  Odds are, architecture, engineering, or some other high-payoff activity, meaning that C&A services are candidates for entry-level security staff.
• Centralized v/s project-specific funding:  Some agencies have a “stable” of C&A staff, if it’s done wrong, you end up with standardization and complete compliance but not real risk management.  The opposite of this is where all the C&A activities are done on a per-project basis and huge repetition of effort ensues.  Basic management technique is to blend the 2 approaches.
• Crossover of personnel from “risk-avoidance” cultures:  Taking people from compliance-centric roles such as legal and accounting and putting them into a risk-based culture is a sure recipe for failure, overspending, and frustration.
• Accreditation is somewhat broken:  Not a new concept–teaching business owners about IT security risk is always hard to do, even more so when they have to sign off on the risk.
• C&A services are a commodity market:  I covered this last week.  This is pivotal, remember it for later.
• Misinformation abounds:  Because the NIST Risk Management Framework evolves so rapidly, what’s valid today is not the same that will be valid in 2 years.

So what we’re looking at with this blog post is how would a program to certify the C&A service providers look like.  NIST has 3 viable options:

• Use Existing Certs: Require basic certification levels for role descriptions.  DoD 8570.1M follows this approach.  Individual-level certification would be CAP, CISSP, CG.*, CISA, etc.  The company-level certification would be something like ITIL or CMMI.
• Second-Party Credentialing:  The industry creates a new certification program to satisfy NIST’s need without any input from NIST.  Part of this has already happened with some of the certifications like CAP.
• NIST-Sponsored Certification:  NIST becomes the “owner” of the certification and commissions organizations to test each other.

Now just like DoD 8570.1M, I’m torn on this issue.  On one hand, it means that you’ll get a higher caliber of person performing services because they have to meet some kind of minimum standard.  On the other hand, introducing scarcity means that there will be even less people available to do the job.  But the big problem that I have is that if you introduce higher requirements on commodity services, you’re squeezing the market severely:  costs as a customer go up for basic services, vendors get even less of a margin on services, more charlatans show up because you’ve tipped over into higher-priced boutique services, and mayhem ensues.

Guys, I’m not really a rocket scientist on this, but really after all this effort, it seems to me that the #1 problem that the Government has is a lack of skilled people.  Yes, certifying people is a good thing because it helps weed out the dirtballs with a very rough sieve, but I get the feeling that maybe what we should be doing instead is trying to create more people with the skills we need.  Alas, that’s a future blog post….

However, the last thing that I want to see happen is a meta-game of what’s going on with certifications right now–who certifies those who certify?  I think it’s a vicious cycle of cross-certification that will end up with the entire Government security industry becoming one huge self-licking ice cream cone.  =)

• Now ISC2 Blogs have an Opinion on FISMA
• When the Feds Come Calling
• Splunk Goes After the FISMA Lucre, They’re not Alone
• Comments on SCAP 2008
• “Machines Don’t Cause Risk, People Do!”