The video of my dojocon presentation. The microphone was off for the first couple of minutes but I look pretty animated.

And then the compliance panel that I tried not to dominate:

And finally, my slides are up on slideshare:

Bookmark to:

Hide Sites

However, AppSecDC has another set of themes that are mostly unique to OWASP and AppSecDC in particular:

• The OWASP Approach to Security: it’s not process/product, it’s education and outreach.  Thanks to Doug Wilson for this idea.  Basically with host and network security, the option is to buy stuff and throw it at the problem.  With application security, it’s “go out and touch a developer today” and “use ESAPI as a tool to help the developers write better and secure code more quickly”.  This is a new concept to the system integrator that I am, but I like it much better than my usual approach.
• Government and Application Security: we’re about 5 years behind industry, how do we catch up?  To this effort, we have some notable Government speakers such as a keynote by Joe Jarzombek, Director for Software Assurance in the National Cyber Security Division of the U.S. Department of Homeland Security.
• OWASP Top 10 2009/2010: This will be announced at AppSecDC with much w00tness and excitement.
• OWASP National Summit: this will be held the day before the official conference.

Convinced you want to go?  Check out the conference site.

Bookmark to:

Hide Sites

Presentation Abstract:

The US Federal Government is the world’s largest consumer of IT products and, by extension, one of the largest consumers of IT security products and services. This talk covers some of the problems with security on such a massive scale; how and why some technical, operational, and managerial solutions are working or not working; and how these lessons can be applied to smaller-scale security environments.

Bookmark to:

Hide Sites

The part where I pitch for Hoff:  Chris Hoff has some awesome ideas on security and cloud computing, check out his blog and presentations.

Bookmark to:

Hide Sites

Now this is my major beef with the way we write SSPs today:  this is all information that is contained in other artifacts that I have to pay people to do cut-and-paste to get it into a SSP template.  As practiced, we seriously have a problem with polyinstantiation of data in various lifecycle artifacts that is cut-and-pasted into an SSP.  Every time you change the upstream document, you create a difference between that document and the SSP.

This is a practice I would like to change, but I can’t do it all by myself.

This is the skeleton outline of an SSP from Special Publication 800-18, the guide to writing an SSP:

1. Information System Name/Title–On the investment/FISMA inventory, the Exhibit 300/53, etc
2. Information System Categorization–usually on a FIPS-199 memorandum
3. Information System Owner–In an assignment memo
4. Authorizing Official–In an assignment memo
5. Other Designated Contacts–In an assignment memo
6. Assignment of Security Responsibility–In assignment memos
7. Information System Operational Status–On the investment/FISMA inventory, the Exhibit 300/53, etc
8. Information System Type–On the investment/FISMA inventory, the Exhibit 300/53, etc
9. General System Description/Purpose–In the design document, Exhibit 300/53
10. System Environment–Common controls not inside the scope of our system
11. System Interconnections/Information Sharing–from Interconnection Security Agreements
12. Related Laws/Regulations/Policies–Should be part of the system categorization but hardly ever is on templates
13. Minimum Security Controls–800-53 controls descriptions which can easily be done in a Requirements Traceability Matrix
14. Information System Security Plan Completion Date–specific to each document
15. Information System Security Plan Approval Date–specific to each document

Now some of this has changed in practice a little bit–# 10 can functionally be replaced with a designation of common controls and hybrid controls.

So my line of thinking is that if we provide a 2-6-page system description with the names of the “guilty parties” and some inventory information, controls-specific Requirements Traceability Matrix, and a System Design Document, then we have the functional equivalent of an SSP.

Why have I declared an InfoSec fatwah against SSPs as currently practiced?

Well, my philosophy for operation is based on some concepts I’ve picked up through the years:

• Why run when you can walk, why walk when you can sit, why sit when you can lay down.  There is a time to spend effort on determining what the security controls are for a project.  You need to have them documented but it’s not cost-effective to be worried about format, which we do probably too much of today.
• Make it easy to do the right thing.  If we polyinstantiate security information, we have made something harder to maintain.  Easier to maintain means that it will get maintained instead of being shelfware.  I would rather have updated and accurate security information than overly verbose and well-polished documents that are inaccurate.
• Security is not a “security guy thing”–most problems are actually a management and project team problem.  My idea uses their SDLC artifacts instead of security-specific versions of artifacts.  My idea puts the project problems back in the project space where it belongs.
• If I have a security engineer who has a finite amount of hours in a day, I have to choose what they spend their time on.  If it’s a matter of vulnerability mitigation, patching, etc, or correcting SSP grammar, I know what I want him to do.  Then again, I’m still an infantryman deep down inside and I realize I have biases against flowery writing.

Criticisms to not writing a dedicated SSP document:

“My auditors are used to seeing the information in the same format at someplace they worked previously”. Believe it or not, I hear this quite a bit.  My response is along the lines of the fact that if you make your standard be what I’m suggesting for a security plan, then you’ve met all of the FISMA and 800-53 requirements and my personal requirement to “don’t do stupid stuff if you can help it”.

“My auditors will grill me to death if they have to page back and forth between several documents”.  This one also I’ve heard.  There are a couple of ways to deal with this.  One way to deal with this is that in your 800-53 Requirements Traceability Matrix you reference the source document.  Most auditors at this point bring up that you need to reference the official name, date of publication, and specific page/section of the reference and I think they need to get a life because they’ve taken us back to the maintainability problem.

“This is all too new-school and I can’t get over it”. Then you are a dinosaur and your kind deserves extinction.  =)

.

This blog post is for grecs at novainfosecportal.com who perked up instantly when I mentioned the concept months ago.  Finally got around to putting the text somewhere.

How to Plan the Perfect Dinner Party photo by kevindooley.

Bookmark to:

Hide Sites

Threat: Evil Twin Agency Attack. A person registers on a social media site using the name of a Government entity.  They then represent that entity to the public and say whatever it is that they want that agency to say.

What’s the Big Deal: Since for the most part there is no way to prove the authenticity of Government entities on social media sites short of a “catch us on <social media site>” tag on their .gov homepage.  This isn’t an attack unique to Government but because of the authority that people give to Government Internet presences means that the attacker gains perceived legitimacy.

Countermeasures: Monitoring by the agencies looking for their official and unofficial presences on Social Media and Web 2.0 sites.  Any new registrations on social media are vetted for authenticity through the agency’s public affairs office.  Agencies should have an official presence on social media to reserve their namespace and put these account names on their official website.

References:

• Evil Twin Attack on GNUCITIZEN

.

Threat: Web Hoax. A non-government person sets up their own social media or website and claims to be the Government.

What’s the Big Deal: This is similar to the evil twin attack only maybe of a different scale.  For example, an entire social media site can be set up pretending to be a Government agency doing social networking and collecting data on citizens or asking citizens to do things on behalf of the Government.  There is also a thin line between parody and

Countermeasures: Monitoring of URLs that claim to be Government-owned.  This is easily done with some Google advanced operators and some RSS fun.

References:

• Network Neighborhood Watch

.

Threat: Privacy Violations on Forums. A Government-operated social media site collects Personally Identifiable Information about visitors when they register to participate in forums, blog comments, etc.

What’s the Big Deal: If you’re a Government agency and going to be collecting PII, you need to do a Privacy Impact Assessment which is overkill if you’re collecting names and email which could be false anyway.  However, the PIA is a lengthy process and utterly destroys the quickness of web development as we know it.

Countermeasures: It has been proposed in some circles that Government social media sites use third-party ID providers such as OpenID to authenticate simple commenters and forum posts.  This isn’t an original idea, Noel Dickover has been asking around about it for at least 9 months that I know of.

References:

• Should Government be an ID Provider
• OpenID

.

Threat: Monitoring v/s Law Enforcement v/s Intelligence Collection. The Government has to be careful about monitoring social media sites.  Depending on which agency is doing it, at some point you collect enough information from enough sources that you’re now monitoring US persons.

What’s the Big Deal: If you’re collecting information and doing traffic analysis on people, you’re most likely running up against wiretap laws and/or FISA.

Countermeasures: Government needs Rules of Engagement for creating 2-way dialog with citizens complete with standards for the following practices:

• RSS feed aggregation for primary and secondary purposes
• RSS feed republishing
• Social networking monitoring for evil twin and hoax site attacks
• Typical “Web 2.0 Marketing” tactics such as group analysis

References:

• How to Create a Social Media Monitoring Strategy.  Note that most of what this blog post advises can get a Government agency in trouble even though it’s sound advice for the private sector.

.

Threat: Hacked?  Not Us! The Government does weird stuff with web sites.  My web browser always carps at the government-issued SSL certificates because they use their own certificate authority.

What’s the Big Deal: Even though I know a Government site is legitimate, I still have problems getting alert popups.  Being hacked with a XSS or other attack has much more weight than for other sites because people expect to get weird errors from Government sites and just click through.  Also the sheer volume of traffic on Government websites means that they are a lucrative target if the attacker’s end goal is to infect desktops.

Countermeasures: The standard web server anti-XSS and other web application security stuff works here.  Another happy thing would be to get the Federal CA Certificate embedded in web browsers by default like Thawt and Verisign.

References:

• MyBarackObama profile hack punts malware
• Barack Obama’s Site Leading to Trojan

.

Threat: Oh Hai I Reset Your Password For You AKA “The Sarah Palin Attack”.  The password reset functions in social media sites work if you’re not a public figure.  Once the details of your life become scrutinized, your pet’s name, mother’s maiden name, etc, all become public knowledge.

What’s the Big Deal: It depends on what kind of data you have in the social media site.  This can range anywhere from the attacker getting access to one social media site that they get lucky with to complete pwnage of your VIP’s online accounts.

Countermeasures: Engagement with the social media site to get special considerations for Government VIPS.  Use of organizational accounts v/s personal accounts on social media sites.  Information poisoning on password reset questions for VIPs–don’t put the real data up there.  =)

References:

• Sarah Palin “hack”

Tranparency in Action photo by Jeff Belmonte.

Bookmark to:

Hide Sites

On another note, I know that one day I’ll walk into somebody’s office or cubicle and they will have IKANHAZFIZMA lolcats pinned to the wall and it will be the highlight of the week for me. If you’re a lolcat printer, drop me an email.

Bookmark to:

Hide Sites

• October 5-7: SecTor, Toronto, ON, Canada.  I’ll be talking about “Massively Scaled Security Solutions for Massively Scaled IT” which an allusion to the size of the US Federal Government IT budget and techniques that they use to manage it.  The Rybolov Layered Information Security Management Model seen here earlier weighs heavily into the presentation, as does a ton of other ideas trying to get people to understand that hazy information security management area above the enterprise.
• November 6-7: DojoCon, Laurel, MD.  I’ll be talking about the “Current State of Compliance” which somewhere along the lines has a punchline of “It’s going to happen anyway, might as well drive the bus instead of being under the bus”.  There is also a compliance panel following my talk and I’ll be on it with Cyberhiker and Dan Philpott.
• November 10-14: AppSec DC, Washington, DC.  I’ll be running amok making part of the conference work.  I’m not speaking at this one which is a good thing because, well, everytime I start talking web apps and security it takes me back to all the bad code I wrote in the late 90’s.  But hey, didn’t we all?

So in between preparing slides, running amok as a volunteer, and the usual work-life imbalance, I haven’t had much free time lately to add to the blog.  Plenty of ideas and blog fodder are floating around inside my head.  After the conventions I’ll put up my materials for the rest of the world to pick on.

Bookmark to:

Hide Sites

Hat tip to Dan Philpott.

Bookmark to:

Hide Sites

Abstract: The use of social media for federal services and interactions is growing tremendously, supported by initiatives from the administration, directives from government leaders, and demands from the public. This situation presents both opportunity and risk. Guidelines and recommendations for using social media technologies in a manner that minimizes the risk are analyzed and presented in this document.

This document is intended as guidance for any federal agency that uses social media services to collaborate and communicate among employees, partners, other federal agencies, and the public.

Bookmark to:

Hide Sites

Crazy Homeless Guy: (walks up to BSOFH) Can I ask you a question?

BSOFH: (Somewhat startled, nobody really talks to him unless they’re trying to sell him something) Uhhhh, sure.

Crazy Homeless Guy: You know that there are people who claim to be able to say… take that truck over there and just by moving their finger make it fly into the Washington Monument.  Don’t you think that this is a threat to national security?

BSOFH: (Realizes that Crazy Homeless Guy is crazy and homeless) Not necessarily, you see.  I would definitely classify it as a threat.  However, when you’re looking at threats from people, you have to look at motives, opportunity, and motives.  Until you have all three, it’s more of an unrealized threat.

Crazy Homeless Guy: But what if these same guys could kill the President the same way, isn’t that a national threat?

BSOFH: Um, could be.  But then again, let’s look at a similar analogy:  firearm ownership.  Millions of people safely own weapons and yet there isn’t this huge upswell to shoot the President now is there?  Really, we have laws against shooting people and when somebody does that, we find them and put them in jail or *something*.  We don’t criminalize the threat, we criminalize the action.  Flicking a finger doesn’t kill people, psycho people kill people.

Crazy Homeless Guy: Or even if these same people could use the same amount of effort to kill everybody on the planet.  You know the <censored, I don’t like being sued by cults> people claim to have this ability.

BSOFH: (Jokingly, realizing that somebody has been taking 4chan too seriously) Well, I wouldn’t care too much because I would be… well, dead.  But yes, possibly.  But then again, since the dawn of the nuclear age and all through the Cold War we’ve had similar threats and people with capabilities created by technology instead of word study and the power of the human mind.  You have to look at these things from a risk standpoint.  While yes, these people have the capability to do something of high impact such as kill every human on the face of the earth, the track record of something like this happening is relatively small.  I mean, is there any historical record of a <censored, I don’t like being sued by cults> actually killing anybody through sheer force of their mind?  In other words, this is a very high impact, low probability event–something some people call a black swan event.  While yes, this is a matter of national security that these people potentially have this capability, we only have so many resources to protect things and we have our hands full dealing with risks that actually have occured in recent history.  In other words, risk management would say that this event you’re speaking of is an acceptable risk because of more pressing risks.

Crazy Homeless Guy: (Obviously beaten into oblivion by somebody crazier than himself) Well, I’ve never thought about it that way.  I’m really scared by these people.  Hold me, BSOFH.

BSOFH: Um, how about no?  You’re a Crazy Homeless Guy after all.  I have to get back to work now.  Come hang out sometime if you want to talk some quantitative risk analysis and we’ll start attaching dollar figures to the risks of <censored, I don’t like being sued by cults> killing all of humanity.  Doesn’t that sound like fun?  If we can get you cleared to get into the building, we can have a couple of whiteboarding sessions to determine the process flow and maybe an 800-30-stylie risk assessment just to present our case to the DHS Psychic Warfare Division.

Crazy Homeless Guy: Uh, I gotta find a better corner to stand on.  Maybe over by 16th and Pennsylvania I can find somebody more sympathetic to my cause.

BSOFH: You’re crazy, man!

Crazy Homeless Guy: You’re crazy, too, man!

And the moral of the story is that no matter how crazy you think you are, somebody else will always show up to prove you wrong.  And yeah, black swan events where we all die are dumb to prepare for because we’ll all be dead–near total fatalities only matter if you’re one of the survivors.

This story is dedicated to Alex H, David M, and some guy named Bayes.

OMG It’s a Psychic Black Swan photo by gnuckx cc0.

Bookmark to:

Hide Sites

The workshop will also be about standards convergence: how ODNI, DoD, and NIST are moving towards one standard and what this means for the intelligence community and military.

Ron Ross from NIST will talk about how the NIST Risk Management Framework is changing from a static, controls-based approach to a more dynamic “real-time continuous monitoring”.

Bookmark to:

Hide Sites

Root cause is I’m a dork, but more about that later.

Anyway, I needed a way to troubleshoot and fix it.  The biggest problem I had was that the problem was very sporadic–sometime it would be 2 weeks between crashes, other times it would be 3 times in one day.  This is so begging for a stress-test really badly.  Looking on the Internet, I found a couple of articles about running a load-tester on apache and information on the tuning settings but not really much about a methodology (yeah yeah I work for a Big 4 firm, the word still makes me shudder even though it’s the right one to use here) to actually solve the problem of apache tuning.

So the “materials” I needed:

• One server running apache.  Mine runs Apache2 under Debian Stable.  This is a little bit different from the average distro out there in that the process is apache2 and the command is apache2ctl where normally you would have httpd and httpdctl.  If you try this at home, you’ll need to use the latter commands.
• An apache tuning guide or 3.  Here’s the simplest/most straightforward one I’ve seen.
• A stress-tester.  Siege is awesome for this.
• Some simple shell commands: htop (top works here too), ps, grep, and wc.

Now for the method to my madness…

I ssh into my server using three different sessions.  On one I run htop.  Htop is a version of top that gives you a colored output and supports multiple processors.  The output without stress-testing looks something like this:

(Click for a life-size image)

I keep one session free to edit files and do an emergency “killall apache2″ if things get out of hand (and they will really quickly, I had to pull the plug about 20 times throughout this process).   I run a simple command on another ssh session to get a count of how many apache threads I have running:

rybolov@server:~$ ps aux | grep apache2 | grep start | wc -l
11

OK, so far so good.  I’ve got 11 threads running with no load and RAM usage of 190MB.  I needed the extra “grep start” because it removes the text editor I have open on apache2.conf and anything else I might be doing in the background.

I also killed apache, waited 10 seconds, and looked at the typical RAM use.  With no apache running, I use about 80MB just for the OS and everything else I’m running.  This means that I’m using 110MB of RAM for 11 apache threads, which means I’m using ~10MB of RAM for each apache thread.  Now that’s something important I can use.

I took my tuning settings in apache2.conf (httpd.conf for most distros) (Apache2 uses the prefork module which uses threads, read the tuning guide for more info) and set them at the defaults listed in the tuning guide.  They became something like the following:

<IfModule prefork.c>
  StartServers            8
  MinSpareServers         5
  MaxSpareServers        20
  MaxClients            150
  MaxRequestsPerChild  1000
</IfModule>

Notice how the MaxClients is set at 150?  This will prove to be my downfall later.  Turns out that my server is RAM-poor for as much processor as it has or Wordpress is a RAM hog (or both, which is the case =)  ).  I’ll eventually upgrade my server, but since it’s a cloud server from Mosso, I pay by the RAM and drive space.

After each edit of apache2.conf, you need to give apache a configuration test and reload:

server:~# apache2ctl configtest
Syntax OK                        <- If something else comes back, fix it!!
server:~# apache2ctl restart

I’m now ready to stress-test using the default setup.  This is the awesome part.  First, I need to simulate a load.  I make an url seedfile so that siege will bounce around between a handful of pages.  I make a file siege.urls.txt and put in a collection of urls so that it looks like the following:

http://www.guerilla-ciso.com/
http://www.guerilla-ciso.com/about
http://www.guerilla-ciso.com/contact
http://www.guerilla-ciso.com/papers-and-presentations
....<about 20 lines deleted here, you get the point>
http://www.guerilla-ciso.com/page/2
http://www.guerilla-ciso.com/page/3
http://www.guerilla-ciso.com/page/4

I’m sure there is an efficient and fun way to make this, like say, a text-only sitemap or sproxy which is made by the same guy who does siege, but since I only needed about 30 urls, I just cut-n-pasted them off the blog homepage.

I fire up siege and give my webserver a thorough drubbing, running 50 connections for 10 minutes and using my url seedfile.  BTW, I’m running siege on the webserver itself, so there isn’t anything in the way of network latency.  <enter sinister laugh of evil as I sadistically torture my apache and the underlying OS>

server:~# siege -c 50 -t 600s -f siege.urls.txt
** SIEGE 2.66
** Preparing 50 concurrent users for battle.    <-The guy writing siege has a wicked sense of humor.
The server is now under siege...                <-Man the ramparts, Apache, they're coming for you!
HTTP/1.1 200   1.08 secs:   16416 bytes ==> /
HTTP/1.1 200   1.07 secs:   16416 bytes ==> /
....<about 2 bazillion lines deleted here, you get the idea>
HTTP/1.1 200   4.66 secs:    8748 bytes ==> /about
HTTP/1.1 200   3.92 secs:    8748 bytes ==> /about
Lifting the server siege...      done.

Transactions:                  61 hits   <-No, this isn't actual, I abbreviated the siege output
Availability:              100.00 %      <-with a ctrl-c just to get some results so I didn't
Elapsed time:                6.70 secs   <-have to scroll through all that output from the real test.
Data transferred:            0.87 MB
Response time:                3.27 secs
Transaction rate:            9.10 trans/sec
Throughput:                0.13 MB/sec
Concurrency:               29.75
Successful transactions:          61
Failed transactions:               0
Longest transaction:            5.61
Shortest transaction:            1.07

Now I watch the output of htop.  Under stress, the output looks something like this:

(Click for a life-size image)

Hmm, looks like I have a ton of apache threads soaking up all my RAM.  What happens is that in about 30 seconds, the OS starts swapping and the swap use just keeps growing until the OS is unresponsive.  This is a very interesting cascade failure because writing to swap incurs a load which makes the OS write to swap more.  Maybe I need to limit either the amount of RAM used per apache or limit the maximum amount of threads that apache spawns.  The tuning guide tells us how…

There is one setting that is the most important in tuning apache, it’s MaxClients.  This is the maximum number of servers (with the worker module) or threads (prefork module).  Looking at my apache tuning guide, I get a wonderful formula: ($SizeOfTotalRAM – $SizeOfRAMForOS) / $RAMUsePerThread = MaxClients.  So in my case, (512 – 80) / 11 = 39.something.  Oops, this is a far cry from the 150 that comes as default.  I also know that the RAM/thread number I used was without any load on apache, so with a load on and generating dynamic content (aka Wordpress) , I’ll probably use ~15MB per thread.

One other trick that I can use:  Since I think that what’s killing me is the number of apache threads, I can run with a reduced amount of simultaneous connections and watch htop.  When htop shows that I’ve just started to write to swap, I can run my ps command to find out how many apache threads I have running.

rybolov@server:~$ ps aux | grep apache2 | grep start | wc -l
28

Now this is about what I expected:  With 28 threads going, I tipped over into using swap.  Reversing my tuning formula, I get (28 threads x 15 MB/thread) +80 MB for OS = 500 MB used.  Hmm, this makes much sense to me, since the OS starts swapping when you use ~480MB of RAM.

So I go back to my prefork module tuning.

<IfModule mpm_prefork_module>
 StartServers          8
 MinSpareServers       5
 MaxSpareServers      10
 MaxClients           25
 MaxRequestsPerChild   2000
</IfModule>

I set MaxClients at 25 because 28 seems to be the tipping point, so that gives me a little bit of “wiggle room” in case something else happens at the same time I’m serving under a huge load.  I also tweaked some of the other settings slightly.

Then it’s time for another siege torture session.  I run the same command as above and watch the htop output.  With the tuning settings I have now, the server dips into swap about 120MB and survives the full 10 minutes.  I’m sure the performance is degraded somewhat by going into swap, but I’m happy with it for now because the server stays alive.  It wasn’t all that smooth, I had to do a little bit of trial and error first, starting with MaxClients 25 and working my way up to 35 under a reduce siege load (-c 25 -t 60s) to see what would happen, then increasing the load from siege (-c50 -t 600s) and ratcheting MaxClients back down to 25.

And as far as me being a dork… well, aside from the huge MaxClients setting (That’s the default, don’t blame me), I set MaxRequestsPerChild to 100 instead of 1000, meaning that every 100 http requests I was rolling over and making a new thread.  That would lead to cascade failure under a load. (duh!)

Bookmark to:

Hide Sites

Piles of Paper photo by °Florian.

So what will this tool be like? It is hard to tell at this point. Some information will be entered directly but the system appears designed to accept uploads of some documents, such as those supporting M-07-16. Similar to the spreadsheets used for FY 2008 there will be separate questions for the Chief Information Officer, Inspector General and Senior Agency Official for Privacy. Microagencies will still have abbreviated questions to fill out. Additional information on the automated tool, including full instructions and a beta version will be available in August, 2009.

Given the required information has changed very little the automated system is unlikely to significantly ease the reporting burden. This system appears primarily designed to ease the data processing requirements for OMB. With Excel spreadsheets no longer holding data many concerns relating to file versions, data aggregation and analysis are greatly eased.

It is worth noting that a common outcome of systems re-engineered to become more efficient is that managers look to find ways to utilize the new efficiency. What does this mean? Now that OMB has the ability to easily analyze data which took a great amount of effort to process before they may want to improve what is reported. A great deal has been said over the years about the inefficiencies in the current reporting regime. This may be OMB’s opportunity to start collecting an increased amount of information that may better reflect agencies actual security posture. This is pure speculation and other factors may moderate OMB’s next steps, such as the reporting burden on agencies, but it is worth consideration.

One pleasant outcome to the implementation of this new automated tool is the reporting deadline has been pushed back to November 18, 2009.

Agencies are still responsible for submitting document files to satisfy M-07-16. The automated tool does not appear to allow direct input of this information. However the document requirements are slightly different. Breach notification policy document need only be submitted if it has changed. It is no longer sufficient to simply report progress on eliminating SSNs and reducing PII, an implementation plan and a progress update must be submitted. The requirement for a policy document covering rules of behavior and consequences has been removed.

In addition to the automated tool there are other, more subtle changes to OMB’s FY 2009 reporting. Let’s step through them, point by point:

10. It is reiterated that NIST guidance is required. This point has been expanded to state that legacy systems, agencies have one year to come into compliance with NIST documents new material. For new systems agencies are expected to be in compliance upon system deployment.

13 & 15. Wording indicating that disagreements on reports should be resolved prior to submission and that the agency head’s view will be authoritative have been removed. This may have been done to reduce redundancy as M-09-29’s preface indicates agency reports must reflect the agency head’s view.

52. The requirement for an central web page with working links to agency PIAs and Federal Register published SORNs has been removed.

A complete side-by-side comparison of changes between the two documents is available at FISMApedia.org.

All in all the changes to OMB’s guidance this year will not change agencies reporting burden significantly. And that may not be a bad thing.

Bookmark to:

Hide Sites

Well, earlier this year, the Government started a website called Data.gov.  This is much awesomeness, Viva Las Transpareny!  However, it’s missing something very relevant to my interests: information security management data.

So, I want people to go to data.gov’s “request a dataset” page and request the following:

Complete responses from the Departments and Agencies to the FISMA reporting requirements for FY2004-2009 based on OMB Memoranda 04-25, 05-15, 06-20, 07-19, 08-21, and 09-29.

Raw incident data for years 2005-2007 as reported to OMB and summarized in their report to Congress on FY2007 FISMA performance and published at http://www.whitehouse.gov/omb/inforeg/reports/2007_fisma_report.pdf

Raw incident data for years 2007 and later in any type and format similar to the Verizon Data Breach Incident Report available at http://www.verizonbusiness.com/resources/security/reports/2009_databreach_rp.pdf

This information is necessary for researchers to study the effectiveness of information security management techniques and regulatory schemes and for industry to propose changes to national-level information security management frameworks and legislation such as FISMA.  This information for the most part has been released in a summary format to Congress and the release of the complete dataset on data.gov would greatly aid the information security community.

It might be a fool’s errand at this point, but it doesn’t hurt to ask, and it only takes a couple of minutes to do.  =)

Bookmark to:

Hide Sites

• Layer 7: Global Layer. This layer is regulated by treaties with other nation-states or international standards.  I fit cybercrime treaties in here along with the RFCs that make the Internet work.  Problem is that security hasn’t really reached much to this level unless you want to consider multinational vendors and top-level cert coordination centers like CERT-CC.
• Layer 6: National-Level Layer. This layer is an aggregation of Federations and industries and primarily consists of Federal law and everything lumped into a “critical infrastructure” bucket.  Most US Federal laws fit into this layer.
• Layer 5: Federation/Community Layer. What I’m talking here with this layer is an industry federated or formed in some sort of community.  Think major verticals such as energy supply.  It’s not a coincidence that this layer lines up with DHS’s critical infrastructure and key resources breakdown but it can also refer to self-regulated industries such as the function of PCI-DSS or NERC.
• Layer 4: Enterprise Layer. Most security thought, products, and tools are focused on this layer and the layers below.  This is the realm of the CSO and CISO and roughly equates to a large corporation.
• Layer 3: Project Layer. Collecting disparate technologies and data into a similar piece such as the LAN/WAN, a web application project, etc.  In the Government world, this is the location for the Information System Security Officer (ISSO) or the System Security Engineer (SSE).
• Layer 2: Integration Layer. Hardware, software, and firmware combine to become products and solutions and is focused primarily on engineering.
• Layer 1: Code Layer. Down into the code that makes everything work.  This is where the application security people live.

There are tons of way to use the model.I’m thinking each layer has a set of characteristics like the following:

• Scope
• Level of centralization
• Responsiveness
• Domain expertise
• Authority
• Timeliness
• Stakeholders
• Regulatory bodies
• Many more that I haven’t thought about yet

Chocolate Layer Cake photo by foooooey.

My whole point for this model is that I’m going to try to use it to describe the levels at which a particular problem resides at and to stimulate discussion on what is the appropriate level at which to solve it.  For instance, take a technology and you can trace it up and down the stack. Say Security Event and Incident Monitoring:

• Layer 7: Global Layer. Coordination between national-level CERTs in stopping malware and hacking attacks.
• Layer 6: National-Level Layer. Attack data from Layer 5 is aggregated and correlated to respond to large incidents on the scale of Cyberwar.
• Layer 5: Federation/Community Layer. Events are filtered from Layer 4 and only the confirmed events or interest are correlated to determine trends.
• Layer 4: Enterprise Layer. Events are aggregated by a SIEM with events of interest flagged for response.
• Layer 3: Project Layer. Logs are analyzed in some manner.  This is most likely the highest in the model that we
• Layer 2: Integration Layer. Event logs have to be written to disk and stored for a period of time.
• Layer 1: Code Layer. Code has to be programmed to create event logs.

I do have an ulterior motive.  I created this model because most of our security thought, doctrine, tools, products, and solutions work at Layer 4 and below.  What we need is discussion on Layers 5 and above because when we try to create massively-scaled security solutions, we start to run into a drought of information at what to do above the Enterprise.  There are other bits of doctrine that I want to bring up, like trying to solve any problem at the lowest level for which it makes sense.  So in other words, we can use the model to propose changes to the way we manage security… say we have a problem like the lack of data on data breaches.  What we’re saying when we say that we need a Federal data breach law is that because of the scope and the amount of responsibility and competing interests at Layer 5, that we need a solution at Layer 6, but in any case we should start at the bottom and work our way up the model until we find an adequate scope and scale.

So, this is my question to you, Internet: have I just reinvented enterprise public policy, IT architecture (Federal Enterprise Architecture) and business blueprinting, or did I create some kind of derivative view of technology, security, and public policy that I can now use?

Bookmark to:

Hide Sites

Right now there is a “pretty OK” framework for SCAP.  IE, we have published standards, and there are some SCAP-certified tools out there to do patch and vulnerability management.

What’s missing right now is SCAP content.  I don’t think this is going to get solved en-masse, it’s more like there needs to be an awareness campaign directed at end-users, vulnerability researchers, and people who write small-ish tools.

So I sat around at home trying to figure out how to get people to use/write more SCAP content and finally settled on “Everytime you use SCAP content, a kitten runs free”.

Anyway, this is a presentation I gave at my local OWASP chapter.

Bookmark to:

Hide Sites

Read one version of the story here

So the whole solution is that, well, we can’t extend FISMA to eHealthcare when the data is owned by the Government because that security management stuff gets in the way.  And this post is about why they’re wrong and right, but not in the places that they think they are.

Government agencies need to protect the data that they have by providing “adequate security”.  I’ve covered this a bazillion places already. Somewhere somehow along the lines we let the definition of adequate security mean “You have to play by our rulebook” which is complete and utter bunk.  The framework is an expedient and a level-setting experience across the government.  It’s not made to be one-size-fits-all, but is instead meant to be tailored to each individual case.

The Government Information Security Trickle-Down Effect is a name I use for FISMA/NIST Framework requirements being transferred from the Government to service providers, whether they’re in healthcare or IT or making screws that sometimes can be used on the B2 bombers.  It will hit you if you take Government data but only because you have no regulatory framework of your own with which you can demonstrate that you have “adequate security”.  In other words, if you provide a demonstrable level of data protection equal to or superior to what the Government provides, then you should reasonably be able to take the Government data, it’s finding the right “esperanto” to demonstrate your security foo.

If only there was a regulatory scheme already in place that we could use to hold the healthcare industry to.  Oh wait, there is: HIPAA.  Granted, HIPAA doesn’t really have a lot of teeth and its effects are maybe demonstrable, but it does fill most of the legal requirement to provide “adequate security”, and that’s what’s the important thing, and more importantly, what is required by FISMA.

And this is my problem with this whole string of articles:  The power vacuum has claimed eHealthcare.  Seriously, there should be somebody who is in charge of the data who can make a decision on what kinds of protections that they want for it.  In this case, there are plenty of people with different ideas on what that level of protection is so they are asking OMB for an official ruling.  If you go to OMB asking for their guidance on applying FISMA to eHealthcare records, you get what you deserve, which is a “Yes, it’s in scope, how do you think you should do this?”

So what the eHealthcare people really are looking for is a set of firm requirements from their handlers (aka OMB) on how to hold service providers accountable for the data that they are giving them.  This isn’t a binary question on whether FISMA applies to eHealthcare data (yes, it does), it’s a question of “how much is enough?” or even “what level of compensating controls do we need?”

But then again, we’re beaten down by compliance at this point.  I know I feel it from time to time.  After you’ve been beaten badly for years, all you want to do is for the batterer to tell you what you need to do so the hurting will stop.

So for the eHealthcare agencies, here is a solution for you.  In your agreements/contracts to provide data to the healthcare providers, require the following:

• Provider shall produde annual statements for HIPAA compliance
• Provider shall be certified under a security management program such as an  ISO 27001, SAS-70 Type II, or even PCI-DSS
• Provider shall report any incident resulting in a potential data breach of 500 or more records within 24 hours
• Financial penalties for data breaches based on number of records
• Provider shall allow the Government to perform risk assessments of their data protection controls

That should be enough compensating controls to provide “adequate security” for your eHealthcare data.  You can even put a line through some of these that are too draconian or high-cost.  Take that to OMB and tell them how you’re doing it and how they would like to spend the taxpayers’ money to do anything other than this.

Case Files and Medical Records photo by benuski.

Bookmark to:

Hide Sites

• You have to give up your high-paying private-sector job to be a Government employee
• You have tons of responsibility
• You have no real authority
• You have no dedicated budget
• You have no staffers
• The job has had half a dozen people filling it in the last 7 years
• The job has been open longer than it’s been staffed over the past 7 years

And yet this is what we’re asking candidates to do in order to even be a candidate for the Cybersecurity Coordinator.  Yes, this is the exact same problem that all CISOs have with having a huge helping of responsibility and none of the authority to get things done, only we scaled it up and out to a national-level CISO position.

Somebody’s even gone as far to say that the lack of candidates for the job is the security field’s way of sending the message that you didn’t scope the job right.  I think this opinion has much merit.  CISOs being what they are, they’re usually pretty astute at walking into an ambush, and this job has all the makings of a good one.

I’ll even turn it around the other way and say that the security industry has yet to produce a CISO’s CISO–somebody who can do politics, budget, security, IT, and consensus-building all in one person.  We have lots of people who can manage the enterprise and below, but it’s that additional little bit of political intrigue that is what we’re missing.  Security people usually avoid politics like the bubonic plague because we’re an industry full of people who say it like it really is.  This is a detriment in sales and politics.

So in true Guerilla-CISO fashion of not pointing out problems without offering something as a fix (no matter how much of a strawman arguement it really is), this is what we need to do to get people interested in being the Cybersecurity Czar^wCoordinator:

• A really well-defined scope.  One person cannot do everything that we are asking for at this price (or any price for that matter).
• A budget for an operating staff where the number is more than than 8 digits.
• Statutory authority over the various departments and agencies responsible for cybersecurity: NCSD, S&T, DoJ, FBI, Commerce.  Indirect influence doesn’t work here, never has.
• The direct ear of the President.  Councils are OK, but puhlease, you want to get the job done, this is what it will take.

Then I read back through my list and realized that we really do need a law to create the Cybersecurity Czar position with everything that I just mentioned.  But here’s the rub: legislation is slow, the bills to make the Cybersecurity Czar aren’t even going to be looked at until the next congressional session because we’re still trying to figure out the budget for last year.

I also think that what we’re calling the Cybersecurity Czar is really 2 jobs.  You need somebody working for the Government CIO Vivek Kundra as the executive-branch CISO and you need a more senior person who worries about the military-industrial base, the critical infrastructure, the support to American commerce, and the protection of little old grandmas who represent the end-users.

Tsar’s Cannon photo by Siyad Ma.  Now that’s some teeth for the position.

Bookmark to:

Hide Sites

So, let’s talk about services firms, contractors fit into this nicely since, well, they perform services.

There are 4 types of work that services firms do (and contractors are services firms):

• Brains: nobody else has done this before, but we hire a whole bunch of PhD people who can research how to get this done.  We charge really high prices but it’s because in the downtime, our people are doing presentations, going to symposiums, and working on things that you don’t even know exist.  Think old-school L0pht.  Think half of Mitre.  Think sharks with friggin laser beams, lasing and eating everything in sight.
• Gray Hair: We’ve done this before and know most of the problems that we can experience, along with the battle scars to prove it.  We charge quite a bit because we’re good and it takes less of us to get it done than our competitors.  Think most good IT engineers.  Think DLP and DAM right now.  Think infantry platoon sergeants.
• Procedural: There is a fairly sizeable market starting to grow around this service so we have to standardize quite a bit to reduce our costs to provide the service.  We use methodologies and tools so that we can take an army of trained college graduates, put them in a project, and they can execute according to plan.  Think audit staff.  Think help desk staff.  Think of an efficient DMV.
• Commodity: There isn’t a differentiator between competitors, so companies compete on price.  The way you make money is by making your cost of production lower or selling in volume.  Think Anti-Virus software (sorry friends, it’s true).  Think security guards.  Think peanut butter.

This is also the maturity model for technology, so you can take any kind of tech, drop it in at the top, and it percolates down to the bottom.  Think Internet use: First it was the academics, then the contractors, then the technology early adopters on CompuServe, then free Internet access to all.  For most technology, it’s a 5-10 year cycle to get from the top to the bottom.  You already know this: the skills you have now will be obsolete in 5 years.

Procedural Permit Required photo by Dawn Endico.

Now looking at government contracting….

As a government contractor, you are audited financially by DCAA and they add up all your costs and let you keep a fixed margin of around 13-20%.  You can pull some Stupid Contractor Tricks ™ like paying salaries and working your people 60 hours/week (this is called uncompensated overtime), but there still is a limit to what you can do.

This fixed margin forces you into high-volume work to turn a profit.  This in turn forces you into procedural or even commodity work.

If your project is strictly time and material, you make more money off the cheaper folks but for quality of work reasons, you have to provide them with a playbook of some sort.  This pushes you directly into the procedural tier.

There are some contractors providing services at the Brains and Gray Hair stages, only they are few and far between.

Traditional types of contractor security services:

• Security Program Management and Governance
• Audit and Penetration Testing
• Compliance and Certification and Accreditation Support
• Security Operations (think Managed Security Services)

Then back around to cyberwar…

Cyberwar right now is definitely at the top of the skill hierarchy.  We don’t have an official national strategy.  We have a Cybersecurity Coordinator that hasn’t been filled yet.  We need Brains people and their skills to figure this out.  In fact, we have a leadership drought.

And yet the existing contractor skillset is based on procedural offerings.  To be honest, I see lots of people with cybersecurity offerings, but what they really have is rebranded service offerings because the skills sets of the workforce haven’t changed.

Some of the procedural offerings work, but only if you keep them in limited scope.  The security operations folks have quite a few tranferable skills, so do the pen-testers.  However, these are all at the tactical level.  The managerial skills don’t transfer really at all unless you have people that are just well-rounded, usually with some kind of IT ops background.

But, and this is the important thing, we’re not ready to hire contractors until we do get some leadership in place. And that’s why the $25M question right now is “Who will that person be?”  Until that time, anything from the vendors and contractors is just posturing.

Once we get a national leadership and direction, then it’s a matter of lining up the services being offered with the needs at the time.  What I think we’ll find out at that time is that we’re grossly underrepresented in some areas and sadly underrepresented in some areas and that these areas are directly inverse to the skills that our current workforce has.  This part scares me.

We need workforce development.  There are some problems with this, mostly because it takes so long to “grow” somebody with the skills to get the job done–maybe 5-10 years with education and experience.  Sadly, about the time we build this workforce, the problem will have slid down the scale so that procedural offerings will probably work.  This frustrates me greatly.

The summary part…

Well, just like I don’t want to belong to any club that would stoop so low to have me as a member, it could be possible that almost all the contractors offering services aren’t the people that you want to hire for the job.

But then again, we need to figure out the leadership part first.  Sadly, that’s where we need the most love.  It’s been how many months with a significant leadership vacuum?  9? 12? 7 years?

The most critical step in building a cyberwar/cyberdefense/cyberfoo capability is in building a workforce.  We’re still stuck with the “option” of building the airplane while it’s taxiing down the runway.

Bookmark to:

Hide Sites