For that reason, I propose that to find the answer we take an interdisciplinary approach and look outside the field of information security. Let us begin with a foundational term, “hazard.” A hazard is an outcome that constitutes a source of danger. A risk is a situation in which more than one outcome is possible (and hence not certain), and at least one outcome involves a hazard.

Jack states that, if asked to provide a list of key risks within their scope of responsibilities, many infosec professionals would answer with a list of issues. I think this is probably correct. The problem, he says, is that such lists make it difficult to measure, compare, and/or prioritize issues. Again, I agree. The only point I would add is that it is sometimes difficult to measure, compare, and/or prioritize ‘real’ risks (i.e., items that are a function of both probability and impact) . Different risks may have qualitatively different types of impacts (e.g., monetary loss, inconvenience, loss of life, etc.).  And, as Nicholas Rescher pointed out long ago, it’s far from obvious that there is a common unit of measurement we can use to compare such risks.  We may have to measure the risks of different hazard types using different units of measurement, just as we use different units of measurement for lengths, temperatures, and weights (Rescher, Risk, pp. 20-21).

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: hazard, jack jones, nicholas rescher, risk

Feed enhanced by Better Feed from Ozh

• The Big One (Pacific Earthquake) – Almost certain
• Synthetic Life – Almost certain
• Self-Aware Machines – Likely
• Extra Dimensions – 50-50
• Alien Intelligence – Unlikely
• Human Cloning – Likely
• Nuclear War – Unlikely
• Fusion Energy – Very unlikely
• Everyday (Room-Temperature) Superconductors – 50-50
• Asteroid Collision – Unlikely
• Deadly Pandemic – 50-50
• Polar Meltdown – Likely

The three events that would be catastrophic and also have a 50-50 or greater chance of occurring, are: the Pacific earthquake, a deadly pandemic and Polar meltdown. There is no disastrous oil spill on the list, possibly because the lead time of the magazine was too long to capture the breaking news. Also, it has already happened so has no place in a forward-looking piece, although prediction of the potential impact of the oil spill might be a worthwhile topic.

What struck me most about the list is that the threat of a devastating cyber attack or cyber war is blaringly absent. It is frustrating to see that cybersecurity is either relegated to minor consideration when discussing threats to our lives and livelihoods or ignored altogether, as it was in the above Scientific American article.

If I were to write a piece on the “thirteenth event,” it would be about the degree to which the World’s critical cyber infrastructure is at risk and that there is “almost certainty” that there will be a major cyber event, brought about intentionally or possibly accidentally, within the next decade. If the response to such a cyber event follows the pattern of the Gulf of Mexico oil spill, government will defer the elimination the source of the problem to the private sector and will take on some responsibility for helping people put their lives back together. Like the oil spill, a cyber event will likely be long and debilitating with the failure of numerous attempts to stem the source by Internet service providers, backbone telecommunications companies, and a host of vendors and industry experts from various sectors.

It is still too early to know what the outcome of the oil spill will be in terms of laws, regulations, liabilities and responsibilities. It is likely that government will play a much more active role in reducing risks and in preparing for responses to events, if not banning deep-water drilling altogether.

If one were to derive the cyber event analogy, government will introduce laws, regulations, restrictions, guidelines, penalties, etc. but not until after a major cyber event. There are those, such as Vice Admiral Michael McConnell (USN, Ret.), who believe nothing will happen until after a major cyber catastrophe, as I describe in my March 29, 2010 Bloginfosec column “Cybergeddon – Ho Hum.” Why not learn the lessons of the oil spill disaster and put in place parallel measures for cyber, as with oil drilling, before a cyber catastrophe occurs, rather than after the fact?

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: cyber attack, disaster planning, Gulf oil disaster, Scientific American, spotlight

Feed enhanced by Better Feed from Ozh

I have known Mark for quite a few years through our joint work at FSSCC, BITS, FSTC, FS-ISAC and other financial services industry groups – all these organizations are mentioned in the book. I met Laksh at the 2010 RSA Conference. Both are with PayPal, which is known for its sophisticated, leading-edge approach to security.

The book is comprehensive. It covers areas with which most infosec professionals and software developers are not likely to be familiar. For example, the authors recount the history of application security testing as far back as to the Orange Book and Common Criteria (CC). Incidentally, Mark co-authored an excellent book on the CC, namely “Computer Security Assurance Using The Common Criteria” (Thomson, 2005). In the current book, issues with the CC approach are raised … and by someone who should know!

Among the many useful chapters, I personally derived the most from Chapters 8 and 9, which are about testing custom applications and commercial-off-the-shelf software respectively. But that is because I managed a project for BITS/FSTC on software assurance for financial firms, which is actually mentioned on page 210. I also was interested in reading Chapter 11 on metrics and maturity models. I found the coverage to be extensive, although I have my own opinions regarding the lack of meaningful metrics for security in general and application security in particular.

I suspect, however, that many readers will be more interested in the design and coding phases of the SDLC (software development life cycle), rather than the testing stage. And these readers will not be disappointed. It was encouraging to see that resiliency is given top billing, as it is often neglected by developers, although software engineers might well see the importance of building resilient systems. To illustrate my own recognition of the importance of resiliency, I recently published an article “Investing in Software Resiliency” in the September/October issue of Crosstalk magazine, which is available at http://www.stsc.hill.af.mil/crosstalk/2009/09/0909axelrod.html

Having given the reader a taste of what he or she needs to know in order to produce or acquire secure and resilient software, the authors point the reader to sources of further education, including the various certifications that can be earned.

The book is rounded out with a very helpful glossary of terms, and a couple of appendices. The first covers the top 25 most dangerous programming errors (according to CWE/SANS), and the second describes OWASP’s Enterprise Security API project.

All in all this is a book packed with valuable information for those designing, developing or supporting secure and resilient software. It is full of useful and actionable suggestions. And it fills a gap that really needed filling. It gives the reader a sound grounding and good understanding of the issues relating to the development of secure and resilient software and points the reader in the right direction for building further upon the base established by the book.

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: Common Criteria, Laksh Raghavan, Mark Merkow, Orange Book, resiliency, SDLC, Security Metrics, software assurance, software security, software testing, spotlight

Feed enhanced by Better Feed from Ozh

Let’s ignore for the moment whether or not the U.S. Government, in the form of the much-maligned (apparently deservedly so) Mineral Management Service (MMS), the Department of the Interior’s regulatory body, actually misled BP. We’ll first take a look what it is claimed caused the errors in the predicted outcome of a catastrophic spill.

The basic assumption described in the article, which proved to be wrong, was that “most of the oil would rapidly evaporate or get broken up by waves or weather.” However, the head of the MMS environmental division in 2001 warned that “the oil spill trajectory models [known as OSRA or “oil spill risk analysis”] currently used by the oil industry for the preparation of oil spill response plans may not be adequate [emphasis added] for deep water.” According to the model “[t]he bulk of the Gulf Coast … would not see oil reach shore even with a catastrophic offshore spill.” Well, at least one thing is clear(er), the OSRA model was exercised for catastrophic spills, which suggests that the models, rather than the input data, were at fault.

How does this measure up with respect to cybersecurity? Well, for one thing, you wouldn’t be able to blame the models … we don’t have any. As for data, we don’t have much of them either, so bad data couldn’t be faulted. It reminds me of a lawyer’s position on publishing one’s company’s privacy policy on the Web, namely, that if you don’t comply with the published policy, you’ll be in more trouble than if you didn’t have a formal privacy policy. Noncompliance is viewed as more serious than omission, I was told. However, to be fair, the lawyer advised that one should publish the policy, but needs to make very sure that it is being followed.

This is exactly my position on cybersecurity. We should have good data and accurate models for cyber attacks and their consequences, so that were a catastrophic cyber event to occur, we would know how to deal with it because we will have run the model, noted the effects, and worked to mitigate the potential impact. I am proud to be working on both the development of simulation models and on the collection of accurate and meaningful cybersecurity data. Both efforts are “in progress” and moving more slowly than I would have hoped, but at least the train has left the station.

And now back to the question as to whether BP implemented riskier technologies because the U.S. government’s models understated the impact of a major spill. This is the perennial “moral hazard” issue … if you believe that the risk to you or your company is lower, do you take greater risks, knowing that you can blame someone else and/or offset the impact on another party? That would perhaps have been valid, had not the CEOs from other oil companies testified before Congress that their companies would have taken a more conservative approach than did BP. And presumably they had access to the same risk models as BP (much as they had the same contingency plans … walruses and all).

Unfortunately we see the same in the cyber world. Since public and private organizations all are on the same page when it comes to protecting cyberspace (the “it’s not my responsibility” page), it is unlikely that meaningful protective measures will be taken. And were something really bad to happen, they can all claim that it is because the government didn’t provide them with a CSRA, or “cyber spill risk analysis.”

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: BP, cyber spill risk analysis, oil spill, risk models, simulation models, spotlight, The Wall Street Journal

Feed enhanced by Better Feed from Ozh

Leonhard draws a parallel between the oil spill and the financial crisis in terms of faulty risk assessment and unanticipated catastrophic events. He also describes how limiting the liability of oil companies to $75 million creates a moral hazard situation where oil company decision makers are prepared to increase their risk knowing that their downside is limited, much as financial executives took on greater risks when they were effectively indemnified against losses.

What about cybersecurity? Are cyber decision makers avoiding meaningful risk-reduction efforts because, if a cybergeddon were to occur, they would not be held personally responsible for stopping the flow or cleaning up the mess? Is it because no one considers him or herself personally liable or responsible? Would we see another “tragedy of the commons” play out as we are with the oil cleanup in the Gulf of Mexico.

It is expected that government emergency services and national security services would be rushed to help with the consequences of a cyber attack or the accidental taking down of the Internet. These consequences might be the loss of communications, transportation, financial services, health services, and other critical sectors.

For a mind-blowing description of the horrifying consequences of a black-swan electromagnetic pulse (EMP) attack, read William R, Forstchen’s book One Second After, which I referenced in my March 22, 2010 Bloginfosec column “EMP-athy for Toyota.” For an authoritative analysis of the risks and remedies for EMP and other black swans, see the newly-published (June 2010) 120-page report High-Impact, Low-Frequency Event Risk to the North American Bulk Power System, published by NERC (North American Electric Reliability Corporation, available at www.nerc.com/files/HILF.pdf.  The report lumps together in one section both EMP and GMD (geomagnetic disturbances). GMD are generally natural events (cf. the Icelandic volcano), whereas EMP can be natural or manmade, either accidentally (cf. the Gulf oil spill) or intentionally (such as the detonation of a nuclear test or a terrorist attack). As an example of GMD, a geomagnetic storm took out the Canadian Hydro Quebec system in March 1989 and 6 million people lost power for 9 hours or more. However, the report is low key when it comes to the impact of EMP on control centers’ networks and computer systems, stating on page 108 that:

“… a program of measurement of shielding effectiveness should be done … the geometry of cables with power and communications entering the facilities should be analyzed. After this information is obtained, assessments of the vulnerability and need for hardening will be completed.”

The above are “shoulds,” not “musts,” indicating that the authors believe that the probability of a major EMP is very small.

While many, including NERC, think that a particular large-scale EMP attack is not likely, some of the resulting consequences, such as those described in Forstchen’s book, could be the result of any of a number of causes, including a large-scale Internet collapse. What is clear is that our catastrophic contingency plans are woefully inadequate, in large part because cyber risks are not properly quantified and assigned, as well as being downplayed by authorities. The government agencies, which might be called upon to resolve a catastrophic cyber disaster do not appear to have either the expertise or the tools to control a massive attack that overwhelms our in-place cyber blowout protectors, which have not been tested under extreme conditions, not previously experienced (sound familiar?). Those in the private sector, such as the big telecommunications companies, which might have the resources, do not see securing the Internet as their responsibility. And the private sector will not become accountable until government insists that they do so, and Congress puts teeth into such insistence.

© BlogInfoSec.com, 2010. | Permalink | One comment | Add to del.icio.us
Post tags: black swans, David Leonhardt, disaster planning, disaster recovery, electromagnetic pulse attack, EMP, Gulf oil disaster, NERC, risk, risk assessment, spotlight, William R. Forstchen

Feed enhanced by Better Feed from Ozh

I also very much agree with his recommendations, listed in his response to my column, of more investment in incident prevention and decent contingency planning with adequate resources. I also accept that both public and private sectors are fallible.

It seems to me that Gary might have somewhat misconstrued my statements as being unquestioning faith in massive government control. Far from it. I am a strong proponent of laissez-faire and capitalism. After all, I was schooled in economics at Glasgow University, the home of Adam Smith, the father of the “invisible hand.”

Nevertheless there are times when government must intercede, like it or not, as when the private sector fails to protect the common good. And when neither government nor business accept responsibility for the “commons,” government has to fill the gap. I personally think that government intervention, laws and regulations are a result of failure of the private sector to address specific problems adequately (see my article “Son of Y2K: Time to Go Back to the Bunker …” Information Security, Vol. 3, No. 11, November 2000). I believe that such extraordinary measures should be carefully considered and taken only in dire emergencies, such as the recent meltdown of the global financial systems, where government intervention was effective in stemming the downward spiral, despite a whole host of undesirable consequences. But, having been well taught the deficiencies of planned economies by the late Professor Alec Nove, a brilliant expert on the Soviet planned economy, I recognize the failings of centrally managed economies, and would never advocate such.

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: Adam Smith, Contingency Planning, Gary Hinson, private sector, spotlight

Feed enhanced by Better Feed from Ozh

On ABC News’s “Good Morning America” on Monday, May 31, 2010, Chairman of the Joint Chiefs Adm. Mike Mullen was asked by anchor and chief political correspondent George Stephanopoulos what the military can do to help cap the oil gusher and clean up the mess. You can see the interview at  http://blogs.abcnews.com/george/2010/05/mullen-military-continues-to-take-active-role-in-oil-spill-clean-up.html  in which you can hear Adm. Mullen discuss “whether military intervention is a viable solution.”

In the interview, Adm. Mullen was reportedly responding to remarks made the prior day by Colin Powell to the effect that “it’s time for a comprehensive, total attack on this problem.”. Mullen pointed out that the military does not have the necessary equipment, which the oil industry has, and therefore must rely on British Petroleum (BP) to deal with the gusher. That is to say, the government is at the mercy of private industry. It is also clear from various reports that BP management has its own agenda, of which one item is to minimize the company’s costs.

If we extend this view to other aspects of the critical infrastructure, particularly cyberspace, we see an equally bleak picture. For example, on the anniversary of President Obama’s May 29, 2009 speech on cybersecurity, Jack Goldsmith and Melissa Hathaway published an article, “The cybersecurity changes we need,” in the May 28, 2010 issue of The Washington Post, available at  http://www.washingtonpost.com/wp-dyn/content/article/2010/05/28/AR2010052803698.html  In the article, they point out that the Obama administration “has made little progress toward this goal [of making the digital infrastructure secure, trustworthy and resilient] … largely because cybersecurity is seen as a tax on short-term economic growth.”

Are Goldsmith and Hathaway insinuating that White House economic adviser Larry Summers and his staff had put cybersecurity on the back burner in favor of the economic agenda? I addressed this in my June 15, 2009 column (yes, it’s been a year), “Here We Go Again … Demoted Security,” in which I suggested that cybersecurity and economic growth are not an either-or proposition. In fact, rather than being a tax on economic growth, cybersecurity can foster growth and protect and preserve its long-term continuation. Security really can be an enabler if done right.

I have also stated that there is great danger in the fact that no one is in charge of the critical infrastructure “commons” … see my May 18, 2010 column “What Richard Told Rachel.” The Gulf of Mexico oil fiasco is a good illustration of this.

Much as there are problems with government grabbing control of huge chunks of the private sector, even during crises, such takeovers and bailouts have recently been shown to work in the financial and auto industries, despite there also being some negative consequences. Perhaps there would be major benefits from government intervention into what is currently the private sector’s cyberspace. I think so. Depending on one’s point of view, we might not yet be facing an immediate cyber catastrophe (though others believe we’re already in one), but wouldn’t it be better to apply an ounce of prevention now rather than megatons of cure after a catastrophic cyber event?

© BlogInfoSec.com, 2010. | Permalink | One comment | Add to del.icio.us
Post tags: Admiral Mike Mullen, BP, British Petroleum, Gulf oil disaster, Jack Goldsmith, Larry Summers, Melissa Hathaway, security as enabler, spotlight, The Washington Post

Feed enhanced by Better Feed from Ozh

We all like to think that superheroes … Superman, Batman, and Iron Man … will arrive in the nick of time to save us from catastrophic consequences. Well, how about Cyber Man or Woman? If a nation is subjected to a major cyber attack, which takes down systems and networks and grinds the economy to a standstill, is it reasonable to assume that a cyber superhero (played by Bruce Willis, no doubt) will pull us out of the jaws of disaster?

The concern about cyber attacks is reinforced by the article “The Enemy Within” by Mark Bowden in the June 2010 issue of The Atlantic Monthly. It is about the Conficker worm. The article tells how, despite the intense efforts of today’s cyber Red Adairs (the “Conficker Cabal,” as they are called), Conficker still remains a major threat; in other words … the bad guys are winning (or possibly may have already won).

If you look into what made Red Adair a hero in his time, you learn that he had deep knowledge and intense courage, and he developed the very best equipment for the tasks at hand. There are some information security experts with extensive knowledge of cyber attacks and cyber warfare. However, in Bowden’s article, Rodney Joffe, SVP and Chief Technologist at Neustar, claims that “There aren’t more than a few hundred people in the world who understand this stuff.”

Most information security experts, in my experience, are really nice people, but they do not seem ready, willing or able to confront adversaries in a rough and tumble. They say what should be done, but do not seem able to muster the forces that are needed to succeed against our adversaries.

Richard Clarke has just published a book on cyber warfare, detailing vulnerabilities and sources of attack. But, in a recent interview with Rachel Maddow on MSNBC, which I described in my May 18, 2010 column “What Richard Told Rachel,” he appears too cerebral and objective … he doesn’t energize anyone to action. Howard Schmidt has the courage to take on the formidable task of White House cyber coordinator, but I haven’t seen him rant and rave about our needing to man the ramparts or charge the enemy (wherever they might be). Vice Admiral Michael McConnell (USN, Ret.) is low key when he states that nothing significant will be done by the public and private sectors until after we have experienced a major attack (see my March 29, 2010 column “Cybergeddon … Ho Hum”).

On the other hand, President Obama, to his credit, did make a major speech on the topic on May 29, 2009 and has shown that he can fight for causes that he truly believes in. However, I don’t think that he has been sufficiently coached on the imminent dangers that face our cyber livelihoods and existence. In fact, his recent statements about not knowing anything about iPods, iPads, PlayStations, etc. give one pause as to the degree to which he might be fully cognizant of the real potential impact of cyber attacks.

When academics address the topic, it is usually from the vantage point of an interesting challenge requiring some game-changing technologies … and these solutions are likely to be years away. No urgency there.

Our defense, intelligence and law enforcement agencies are heavily involved, but predominantly with respect to their own turf. So what is left?

What we need are convincing cybersecurity and cyber defense calls to action, leaving intellect and reserve behind. An appropriate exhortation, taken from William Shakespeare’s Henry V, might be: “Once more unto the breach, dear friends … !”

Once more …

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: "Red" Adair, "The Enemy Within", BP, British Petroleum, conficker, Howard Schmidt, Mark Bowden, Michael McConell, President Obama, Richard Clarke, spotlight, The Atlantic Monthly

Feed enhanced by Better Feed from Ozh

Eldridge Cleaver, 1968

The explanation for the 1000 point drop and bungee rebound in the Dow Jones on May 6, 2010 has been, and continues to be, the object of much scrutiny. Many explanations and combinations thereof have been put forward. Perhaps the most interesting, and in some senses the most disturbing, is the article “Did a Big Bet Help Trigger ‘Black Swan’ Stock Swoon?” reported by Scott Patterson in the May 11 issue of The Wall Street Journal  … and yes, Jacob Bunge did contribute to this article also.

As you may recall, I have long touted Nassim Nicholas Taleb’s book, The Black Swan, in which he warns about our lack of anticipation of low-probability but high-impact adverse events. Well, according to the WSJ article, Universal Investments LP, a hedge fund with  Mr. Taleb as an adviser, might have been instrumental in the huge stock market price decline by exacerbating an already-falling market through placing huge bets that the market would tumble further.  Apparently this action might have triggered a series of events leading to the “flash crash,” notwithstanding issues that arose in regard to how the various marketplaces invoked (or didn’t invoke) circuit breakers.

The above issues raise an interesting question in regard to cause and effect for information security professionals. The question relates to the degree to which we might actually be creating problems rather than reducing them. There is a frequent refrain by management, which is based on actual experience (mine included), about there not being a need to spend so much on security since nothing bad has happened. The converse to that is the statement that we only started being attacked after we had installed firewalls, since we could then observe a continuous series of attempts to break into the systems.

Of course, this is faulty reasoning … yet there is a modicum of truth to it. It is not that security professionals cause break-ins, but there is little doubt in my mind that, by raising the bar, we are cultivating smarter, more sophisticated and more effective forms of attack. Much as the excessive and inappropriate use of antibiotics often results in more virulent drug-resistant microbes, so we are seeing the growth of highly-professional technically-brilliant attackers against systems that have been well protected against earlier malware.

So, you might be thinking, are you suggesting that we back off the escalating security arms race and give the bad guys free reign? Clearly that makes no sense. In fact, what we try to do is to get ahead of the attackers, which to date has been a discouraging effort, to say the least. It is human nature to step up to such challenges. But we should do it in ways that make sense and have some chance of success. The U.S. government and its agencies are looking for “game-changing” technologies to beat the criminals and terrorists, but my response to that is that the bad guys don’t play by the rules anyway, so why change the game? They’ll only break the rules again.

No, the answer isn’t fully a technical one. Humans are very good at getting around or subverting preventative technologies, if not directly then through social engineering and threats of physical harm (most movies on the topic favor the latter for its visual entertainment value). So we really have to address the human and social factors. Avoidance and deterrence remain when preventative approaches have been exhausted. We must cajole or pressure individuals to behave in a manner that protects and preserves information assets. And we must limit exposure of such assets to potential damage and compromise.

I wrote about the human role and its impact on security in the chapter, “An Adaptive Threat-Vulnerability Model and the Economics of Protection,” in the book Social and Human Elements of Information Security: Emerging Trends and Countermeasures, edited by Manish Gupta and Raj Sharman (IGI Global, 2008).

It is likely that, when the post mortem of the May 6 dive in the Dow Jones is eventually completed, we will discover that human elements were just as great contributors as were technology and processes. It will also become more apparent that computer systems multiplied human activities by orders of magnitude, thereby greatly exacerbating their impact for good and for evil. Many were part of the problem … and few, if any, were part of the solution. Whether an effective solution can be implemented remains to be seen.

© BlogInfoSec.com, 2010. | Permalink | One comment | Add to del.icio.us
Post tags: breaches, Dow Jones, Human Elements, Nassim Nicholas Taleb, spotlight, The Black Swan, Universal Investments LP

Feed enhanced by Better Feed from Ozh

As an aside, I, probably with many others, had thought of the bungee-jump analogy on seeing graphs of the price movements and, sure enough, the term was used in an article by Benyamin Appelbaum in The New York Times of May 9, 2010 with the title “Thursday’s Stock Free Fall May Prompt New Rules.” Quite by coincidence, a person by the name of Jacob Bunge contributed to several articles on the bungee-jump market in The Wall Street Journal, although I didn’t see them using that particular term.

Back to the main theme … It appears that the downward spike in the market could well have been due to a number of “systemic” idiosyncrasies, to put it mildly. Why am I not surprised? As with the subprime fiasco, major disruptions have occurred in areas so complex that only immense high-speed computers can deal with them … sometimes with catastrophic results. Part of the problem is that, while very bright humans designed, developed and tested these systems, they did so in relative isolation. True, the creators of these systems had to ensure that each component system “talked” to others, but in this case no one had apparently grasped that the sum of the somewhat understandable parts was greater than the incomprehensible whole. There were reportedly prior rumblings about the inconsistencies among the various types of stock exchange, but it appears that no one in authority insisted on creating a stable overall computer-based marketplace.

Which brings me to my main point … In my earlier columns of January 11, February 16 and February 22, 2010, I pushed the concept of negative testing, which I choose to call “functional security testing.” This is a form of testing which determines that systems do not do what they are not intended to do. It is not good enough to verify that systems function as they are supposed to do. One must also make sure that such systems do not behave in inconsistent or dangerous ways when subjected to particular inputs and that, if forced into failure mode, they fail gracefully. I wrote about this topic in an article, “Investing in Software Resiliency,” which was published in the September/October 2009 issue of STSC CrossTalk: The Journal of Defense Software Engineering.

Recently I had a chat with Chris Wysopal of Veracode on the subject of functional security testing. It was at the May 7, 2010 CSO Breakfast Meeting, which is an outstanding forum created and run by Bill Sieglein. Chris rightly pointed out that such testing can be huge with a virtually unbounded number of test cases. I agree. However, I suggest that, if testing is limited to first-order cases (that is, what happens when a single input, rather than a sequence of inputs, is entered) and if statistical sampling methods are used, then it might be somewhat tractable. I have certainly found such an approach to be beneficial, even though not all-encompassing.

Now, this type of negative testing might be doable for systems under the control of a single entity, but what do you do about a complex of systems that span many entities? Well, there are several possible approaches. If, as in the case of U.S. stock markets, they are ultimately under the jurisdiction of the same government regulator, such as the SEC, then that regulator can mandate broad-based testing across all participating market operations. Otherwise, industry groups, consortia or coalitions might be able to coordinate such testing among their membership.

Of course, such multi-entity testing exhibits orders-of-magnitude greater complexity than that done by a single organization. I bemoaned the increasing complexity of computer systems as far back as April/May 1994 in my regular Technology column in the short-lived, though excellent, magazine Securities Industry Management. The title of the piece was “The Death of K.I.S.S.” Today, we have not only much more complex systems and networks than 16 years ago, but the security technologies, which we use to protect them, are similarly complex.

So, what is the answer for determining system behavior across many organizations? One approach is to develop more macro-level simulation models of the various marketplaces. Such models can be used to determine impact for higher-level scenarios, such as the one that occurred on May 6, 2010, rather than at the individual program level. I personally helped initiate a project to create such a model for financial transactions. I am hopeful that models, such as this one, will provide insights that will both anticipate and explain unforeseen behaviors in response to major events.

One cannot expect organizations and their regulators to understand and control complexities developing in financial services and other sectors without the aid of sophisticated methods, such as advanced testing tools and simulation models. Yes, we can opt to remain in reactive mode, suffer the damaging consequences of catastrophic events, and then try to come up with solutions after the fact. How much better it would be if we could understand the impact of various disasters before they actually happen and modify our systems in advance so as to avoid the pain.

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: Dow Jones Index, functional security testing, negative testing, spotlight

Feed enhanced by Better Feed from Ozh