Disclaimer up front: I’ve worked with DHS as a contractor. I have friends in DHS. I have DHS as a client agency. I’ve felt some of their growing pains. I’m also a taxpayer and a wannabe civil libertarian when it suits me. =)

Background: Last week, Scott Charbo, DHS CIO, was given a pretty good grilling by the House Committee on Homeland Security. Responses have ranged anywhere from apathy to outrage, with the mainstream media wondering why DHS is doing so poorly at security of their systems.

The testimony is online at the following url: http://homeland.house.gov/hearings/index.asp?ID=65

First of all, at the bottom there is a link that takes you to the movie. You have to watch this before you read the transcripts. Caution: this is a good 1.5 hours of viewing. http://boss.streamos.com/wmedia/homeland/chs/cyberjune.wvx

My first comment is something along the lines of what the public is saying. If you’re responsibly for cybersecurity (not my preferred title, btw) for the nation, how can you honestly stand in front of us as the Government’s cybersecurity leader when you’re failing at securing your own house?

Scott Charbo gave an excellent answer, one that the public needs to seriously think about. There are 2 information security groups inside DHS. One is the Assistant Secretary for Cybersecurity and Telecommunications who works with the rest of the government and industry to help secure the infrastructure. The other is the DHS Chief Information Security Officer who works under the CIO to security DHS-internal systems. What this means is that the 2 topics are divorced both on an organizational chart and in funding sources. It’s still a PR problem, but there is a specific reason why this problem exists.

The Q&A Session Led by Chairman Langevin:

• Titan Rain–Everybody doing IT in the government should know about Titan Rain. As a CIO to say that you haven’t heard about it, it’s a red flag. This doesn’t bode well for the agency that has been charged with cybersecurity for the rest of the agencies.
• Ingress and Egress filtering on workstations–This is usually too noisy, so what you do is filtering on the aggregate data flow from multiple machines. Otherwise, you end up with a NMCI problem where every workstation had a HID on it. It’s expensive and probably rates lower on the scale of priorities than other security spending. Maybe in the future when endpoint security is an all-in-one, it will make much more sense, and the technology is starting to get to that point.
• Nationwide Risk Assessment–yes, it’s a fantastic idea. The question is, how do you eat this elephant? Really it takes an ongoing campaign of assessing individual parts (bite-sized pieces, pun intended) and then addressing and prioritizing them as a whole. Some of that is taken care of ala GAO reporting. Some of that (SCADA systems, commercial telecoms, anything we have a dependency on) needs to be discovered and assessed. You have to be careful when trying to boil the ocean.
• Classified Spillage–it’s one of the “dirty little secrets” (pun intended) of the classified world. Short of context-filtering on the non-classified side (cheap pitch for Verdasys here), there is nothing that you can do technically to prevent a user from manually typing classified data into a non-classified system. But then again, you can’t prevent a user from talking about classified data on a metro train.
• Contractor Laptops–note to self: if you are testifying in front of congress, never answer a question with only a “No.” Are contractors plugging into government IT systems? Yes, and DHS isn’t the only one. It depends on the facility. If you go into a classified facility, then plugging into a classified network is bad. If you go into a development environment to upload code that you built on your laptop, then most people would say that’s OK. Somewhere in there is a spectrum of activity that needs to be decided on whether it’s allowed or not.
• Budgeting–The role of CIO pretty much is in an advisory role when it comes to budget. Inside DHS (and all the other agencies), Congress manages the budget down to the sub-agency level. Mr Charbo can request funds (and if he got the message, he should request more security funding next year), but holding him responsible for the budget that is given to him by Congress hardly seems fair or really what I would call responsible governing. However, a good point was made by Mr Etheridge about the fact that DHS is a very young organizations and that they most likely need to be spending more than the average on security. But then again, they are spending quite a bit of money on building IT systems, so a smaller percentage is to be expected (ie, the size of the pot got bigger, so you have a smaller percent of that pot).
• Auditing Telcos–You cannot audit the carrier clouds. You use compensating controls to limit your risk. I’ve talked about this before. However, why is the telco managing the agency’s firewall? It sounds like somebody was doing routing on the firewall or doing some kind of logical segregation on their switches (ie, untrusted and trusted on the same switch using VLANs), which shouldn’t be happening for your main edge. Here, GAO is pointing at one system where they were allowed to audit a MPLS cloud and saying that they should be able to audit a DHS MPLS cloud. It just doesn’t work that way. You might be able to do a partial edit or you pay the vendor more to implement specific controls that you need, but that’s the extent of it.
• Einstein–Link for those of you who are interested, and a blog post from Richard Bejtlich about it. It’s a monitoring system used by quite a few agencies.
• Interconnectivity Between Classified and Non-Classified Systems: GAO points to the fact that DHS did not have a valid system inventory or established interconnects. While I agree with some of the concept, I guess I just don’t like the presentation layer of that statement, like we’re confusing security and compliance again.

I still contend that if another agency the size of DHS is not reporting as many incidents as DHS is, then they’re either not monitoring or they don’t have the same criteria as to what an incident is. I think DHS got banged on first because they provided transparency and fairly valid metrics on what is going on with their networks. Playing the role of “Armchair CIO”, I would turn it back on the other agencies and ask why they didn’t have the same level of incidents to report.

I give DHS quite a bit of credit for avoiding the urge to present a “zero defects” picture to GAO, OMB, Congress, and the public.

Best quote of the day is from Keith A. Rhodes who is the Chief Technologist and Director of the Center for Technology and Engineering at the Government Accountability Office.

“The risk assessment that you’re talking about, risk is a function of threat, vulnerability, and impact, so all three pieces have to be done. Yes, there has to be a threat assessment. There also has to be a realization of vulnerability, and there has to be an understanding of impact. No one, certainly not I, certainly not my colleague, Mr Wilshusen, is going to say ‘Secure everything, lock everything down.’ That’s impossible. It’s also impossible to have perfect security, but we have to drive toward zero tolerance on key systems.”

By this time, you’re all thinking “What will it take to get DHS to winning in IT security?”

There are some people who believe that DHS will never make it. “It’s too large, the Department is too new.”

Realistically, I think the earliest realistic timeframe for DHS is 5-10 years and 3 CIOs down the road. Scott Charbo will build as much as he can until he meets serious resistance, then it’s time to bring in a new face to push the ball forward just because the newness can get things done.

Once again leading me to my point that security is all about personnel management.

While DHS has overcome quite a few hurdles, I think it’s amazing that they managed to score any more than an “F”.

What I didn’t hear in this hearing is something along the lines of “Mr Chairman, we only have a limited amount of personnel, time, and budget. As an agency, we are forced to make decisions on what is more important to us: to migrate all the organizational elements to OneNet and build a NOC, SOC, and redundant data centers, or to maintain legacy major applications and put HIDs on all of our workstations. While you might disagree somewhat with our priorities, I doubt that anybody would chose a path that is radically different from where we have gone and are going.” That’s the message that the country needs to hear in order to understand the conflicts between operations, budget, and security that today’s CIO has to manage, and why the indicators at times might provide the impression that the government is not concerned about security.

But then again, I’m a little bit more confrontational because I can afford to be, not being in charge of the IT assets for a huge agency. =)

• Looking for the Charbo Testimony
• Ack! With the Mandates!
• GAO’s 5 Steps to “Fix” FISMA
• Effective Inventory Management
• HR 5983–DHS Now Responsible for Contractor Security