Following the White Rabbit Blog

Automated Security Testing - Can't I Just Point-n-Click? (Part 3)

RafalLos — Fri, 16 Oct 2009 21:49:00 GMT

So now that you've got the background from my other 2 posts in this series, you know the options and you have some background. Let's talk about the limitations of technology and why your brain is still required to do your job. Many folks continue to try and push the boundaries of technology, and while I applaud this effort greatly, I for one can't see us security analysts ever being replaced entirely by technology as some would have you believe. The analytical mind still trumps technology ... although I think there are some limitations based on levels of experience, etc. Read on for more ...

Technology & Automation's Limitations

Let's face it, there are some very serious limitations to technology today, even in the product-filled world of web app security. I think you will probably agree that there are many products that solve non-existant problems ... or what I would refer to as "brilliant solutions without purpose"... but we'll save that conversation for another time. Right now if we look at automation logically we can simply state that automation ,more specifically software, has its limitations at pattern-matching, for the most part (more in a minute). Immediately we can say that pattern-matching is a severe limitation to any technology, just look at the failed anti-virus installation on your computer. Does it protect you from every strain of every virus? what about new malware? Of course not ... that's why everyone pretty much agrees anti-virus in present form is a dead concept. Moving this into the web app sec world we can easily say that pattern matching is next to impossible when you look at static analysis (analysis of source-code) because thanks to the brilliance of the human mind we all do things just a little bit differently. To prove the point, ask 10 developers to write the same piece of code, even a simple function, you may find 2 that are the same ... maybe.

Static analysis is particularly difficult from the perspective of automation, although there are great attempts out there I will acknowledge, because you're dealing with code. As I've written previously static code analysis has enough of a hard time with dealing with producing theoretical vulnerabilities, much less trying to understand every developer's code. This is why there is no such thing as an "out of the box" tool that works on static code analysis. Let's be logical about it ... your "sanitization" function can't possibly be anticipated by the tool you just bought to analyze your code ... and while the tools available can make attempts to "learn" the way your developers code, and what functions are safe, which are scrubbers, etc in the end it's just hours and hours of "tuning" that require ... ta-da ... human intervention!

Taking the case to dynamic analysis doesn't make it any more pretty. Again, here we're pattern-matching against expected outcomes to "negative testing". We push javascript (such as the ever-popular pop-up) into a form field and expect it to come back to the browser in the same way that it was sent and execute a pop-up. Then we can determine that it's a vulnerability... right? It's not that simple though - because when you look at code coming into the browser you have to analyze the context it's being piped into! If you're pushing code you have to make sure it will actually execute first ... which is the challenge. Next, for your consideration think about how we test for database manipulation (SQL Injection). We send database command syntax appended or injected into the regular application fields to try and elicit a database response. Of course ... if the developers suppress databse responses to the end-user this makes it very difficult to detect injection when it's "incorrect"... Concepts like Blind SQL Injection are even more tricky because you're injecting database commands and not expecting a direct response but a change in page-state, or a positive/negative response which is also extremely difficult to script and contextualize. You'll notice that a lot of this comes back to context and while software can do catagorization pretty efficiently a la pattern matching, it's impossible to account for all possible states, responses and configurations. Yikes! This is all enough to make your head spin!

I certainly don't envy those developers who are writing security analysis tools, and I can tell you first-hand that the folks that work in our HP Web Security Research Group are absolute geniuses. Scripting and pattern-matching gray-area responses is like walking a tightrope between false-positives and false-negatives ... and remember that no matter what you do people will attempt to discount the tools you build because they're either too noisy, or miss too much. This is especially why I am so big on human interaction in the process!

Your Brain Required

Now we get to it ... your brain will continue to be required for the forseeable future in security, more specifically the analytical part of web app security. While technology and innovation will continue to drive better and "smarter" engines for analyzing and attacking web applications, my crystal ball tells me that people will always be necessary. Actually, not just people. People with a clue will always be necessary - there is a huge distinction! Let's venture into why [intelligent] humans are necessary, and why anyone selling "a point-n-click" security tool should be laughed out of the building. You see, people build software. Even the smartest people make mistakes. Therefore, even the best software will have mistakes which often manifest themselves as security vulnerabilities. Given that, why would you trust the analysis of this potentially vulnerable software with more potentially buggy software? Make sense yet?

Software-based testing, even software-driven testing is fine as long as there is someone who is schooled and reasonably accomplished in the art and science of interpreting results and analyzing them. What is required here is a 2-step method we like to refer to as "validation" of findings. You see, automated tools continue to get better at finding more and more complex defects yet the analysis of findings will always be the trickiest part of a security testing strategy. Looking at what an program/script/tool has uncovered and being able to critically deduce whether this is a positive vulnerability, a false-positive, or whether it simply requires more attention is critical to a security analyst's position and job description. The power of the human mind often kicks in where software leaves off, and can trigger a multitude of findings that would otherwise go undiscovered.

A great example of this type of need for a human analyst is from a penetration test I was a part of a while back. The automated tool uncovered a treasure trove of low-hanging vulnerabilities including some cool SQL Injection and Cross-Site Scripting issues, as well as a crossdomain.xml issue that was pertinent to our attack. On their own these attacks could do some damage but it wasn't until the analyst actually dug into these attacks and noticed that they could be chained together to produce an incredible attack vector that there was (at the time) no solution for! You see, we could test for XSS, and SQLi, and even the crossdomain.xml vulnerability ... but the software couldn't string those together and notice a gaping flaw in the design of the application that allowed for a complete compromise of the online application.

So the bottom line here is that I want you to walk away from this series being able to not only understand but intelligently speak about why a "point-n-click" security testing tool will never suffice, and why you have to have the human intellect to back it. That being said, there are a number of offerings such as HP's Web App SaaS offering which mix the automation and tools approach with an augmentation of the human factor for when you find yourself in a situation where you just don't have the in-house expertise! What I'm saying is don't trust your web application security to a tool, or even a collection of them - because alone they aren't telling you the whole picture. Throw away the notion that you can just point and click your way to being secure ... it's never going to happen that way.

The answer then? Education, first and foremost, is key. Make sure you either educate or hire smart & intelligent security analysts. Make sure that you have people who understand how attacks work, why they work, and how to detect them manually. Your analysts should be able to spot basic attacks like SQLi and XSS in a site by hand, and execute (or know where to get cheat-sheets for) the more complex attacks. You don't have to hire the uber-hax0r, just know enough to call one when you need one. The next thing is to ensure you've got the best tools in your toolbox... often this means mixing open-source and closed-source apps together into something that works best for you. Know your applications and which attacks apply ... PHP-style attacks certainly won't work against IIS-based ASP.Net apps ... usually. Be ready to raise your hand when you're in over your head. There's no shame in asking or acknowledging when you don't know ... I do it all the time and it's quite liberating.

I hope I've managed to convince you that point-n-click security is a failed prospect. What do you think? If you're interested in a further conversation please feel free to email me (via this blog) or get a hold of me through your HP sales rep (you probably already have one!)

Automated Security Testing - Can't I Just Point-n-Click? (Part 2)

RafalLos — Fri, 16 Oct 2009 17:06:00 GMT

In the previous post - I tackled the question of automation, full automation, in web application security testing. We discussed the problem in great detail and underlined some of the issues that we will need to address and understand. In this post, I'm going to talk through the options and technological limitations that we face today and will continue to face deep into the future.

The Options

If you're going to attempt to test web applications with some measure of automation there are a few options you have available. There are full and partial automation opportunities, and application separation as well as multiple tools. Addressing them in order here...

Full automation is what most people still think of when it comes to security testing their web applications. Full automation involves simply putting a URL into a field and clicking GO and standing back to watch the action. There are times when this is practical but there aren't many of those times, unfortunately. I've spoke with many folks recently who feel that web application security testing should be done like vulnerability scanning was when it first kicked off. Point, click, and receive results. This isn't practical because of the fact that there are many possible ways that this option can fail. Sadly, the less people understand the more they want to push into full automation. Let's think about full automation for a minute. In order for a tool to be able to perform a fully automated scan you have to assume that the tool can analyze site structure and compute an attack strategy on the site without human intervention. Forget that you're asking a whole lot from a computer program ... think about what that actually means. You'll be asking the tool you're using to be able to understand every part of the application ... fully. Can you say you can understand every part of the applications you test fully? Remember software is only as good as the people who write it, and unfortunately the people who write testing software can only make it as good as the examples they have to work with. Herein starts to peek a problem we'll address later ... mounting complexity. Full-on automation requires that the tool analyze every AJAX call, every FLASH object, every piece of JavaScript, every nook and cranny and every workflow through the application. If you're heard me talk about the failure of automation on the frontier of workflows you already know why this is such a losing proposition without human automation - but it gets more complex than that. You're hoping that the automation component can do all the work in a pre-defined amount of time, right? Let's be realistic, most automated tools, if not properly tuned will run for days, hours or weeks before running themselves out of memory of stack space - hopefully completing the scan. The reasons this happens I will address later on in the technical limitations but you're asking an awful lot of software that's testing software. Say you do get a complete scan. Say for the sake of argument that the tool you're using manages to completely cover the web application attack surface and finds a whole mother-lode of vulnerabilities. What you're saying now is that you want that same piece of automation (or software) to be able to validate its own findings. Fail. You already know that automation isn't perfect at finding vulnerabilities ... and now you want validation for the same price? Consider that ask...

Partially manual testing is the next logical choice. Involving the human being as little as possible but still allowing for some intervention to do the set up and validation makes logical sense. The problem here is that the human being here has to understand what he or she is doing otherwise this process fails. Integrating a human being into web application security testing is a scary thing ... because now you're asking a human being to complement the software you're using but it certainly has its advantages. In fact, I would argue that it's better to have a human involved than to attempt to do everything with automation as you'll get better results 4 out of every 5 times. The problem is in the human part of this equation. Knowing what you're doing ("I'm testing a web application's security") and actually knowing what you're doing are drastically different. You also have to be trained in the tool you're using otherwise you'll fail with even more vigor. But here's the deal, partial automation involves the human being (tester) interfacing with the tool in order to provide it not only analytical insight but also guidance on what to test, what variables to use, what to tweak and what to avoid ... then analyzing the results. This is what most knowledgeable penetration testers and web application security experts do today with varying degrees of success. Don't let anyone fool you, it's a lot tougher than you'd think to get results particularly when they have to be consistent! Tweaking a piece of software and using it like a sledgehammer to find the low-hanging fruit is fairly easy ... getting deeper and better results than the tool could do on its own is a little more tricky. Lots of testers simply never master this craft and either end up blaming the tool, or simply giving up. Partially automating a testing tool, particularly one that's built to do evil, is an art-form and must be well-understood or the results could not only be catastrophic, but also inconsistent and more dangerous than when the tool is run fully automated.

Your other option, of course is do testing in a partially automated way. What you probably don't know is that tools like WebInspect can function in this capacity brilliantly. "Penetration tester assistance mode" is what the folks who do this all the time call it. As the penetration tester looks at different areas of the site a black-box scanning tool is used surgically, with a large amount of human guidance. This use-case really isn't a human being assisting an automated tool as much as an automated tool is used as a supplement to the human being's abilities to do the mundane and simple tasks. Furthermore, more advanced tasks can be performed such as advanced XSS or SQLi testing within the framework of the tool so the tester doesn't have to do it by hand. Using the tools as an extension of the tester is a great way for someone advanced in the art and science of breakage to function ...but that expertise has to be there first. You can't just jump feet-first into this type of usage model and expect to succeed.

So there we have it, 3 possible ways to engage in "automated" testing tools, a la black box security testing. The thing you must think about is which one is right for the situation you find yourself in, your knowledge level and experience, and specific use-case. What works for one may not work for others, your mileage may vary, batteries not included and some assembly is required.

Automated Security Testing - Can't I Just Point-n-Click? (Part 1)

RafalLos — Fri, 16 Oct 2009 16:14:00 GMT

I've been witness to an interesting phenomena. Several otherwise rational folks- customers, prospective customers, and pundits alike - have posed the question to me now over a the last several months. I've been thinking a lot about the topic and have some thoughts I think it's time I share.

The question for discussion is this: "Shouldn't a security testing tool (Web App security, black-box specifically) be able to just accept a URL and credentials and test my site, providing results without me having to intervene?"

The answer, quite simply is an unabashed "No"... but I think it needs more of an explanation than that. It's often all too simple to provide an answer without explanation; or worse with an explanation that not everyone can understand, so I'll both answer the question, explain it in detail and give some real-life examples of why I'm answering this way. Grab a cup of coffee, get comfortable and let's think this through rationally together. I'm going to do this as a multi-part blog entry ... I can already see this as taking a few hours to write much less to read and fully comprehend...

The Main Issue

The main issue in question here is not whether computers can replace humans entirely for security testing - which I hope we can all agree on is a solid no but whether computers and automation has come far enough to begin test automation to a point where a human can provide minimal input and have a test complete. The problem with this request is that we're asking automation to make decisions within the process of testing. Decision making, so far in evolution, is best left to the human analytical brain, rather than automation - and the primary rational is here is that humans possess the ability to reason rationally whereas computers ... cannot. At the core of the question is the ability to make decisions or reason which then either makes or breaks an automated test. Let's think about this in a different light... let's look at this from the viewpoint of a mechanic. What we're really asking here is for a computer to hook up to the vehicle, diagnose the entire system without human input and then provide a solution, testing the effectiveness without a human in the loop. Rationally we can already see where this would break down. A computer can hypothesize a problem, apply a solution successfully without actually solving the problem the driver had in the first place. Diagnosing a problem in a vehicle, as mechanics will tell you, is more than just something you can do from a text-book, or by taking a course. It takes years of experience to understand vehicular cause and effect, and why a rattle in the front of the car may actually be a bad bearing in your rear wheel... computers can't tell you these things, yet. The other issue here in the mechanical world is that not everything can be connected to a computer system for diagnostic yet - there are still limitations. The problem can be easily extended to the digital world for web applications. Not everything can be analyzed properly and we'll go into more detail in a minute for why that is.

Bringing this back to the question at hand and whether automation can simply "do the job" of assessing a web application's security viability ... we have to break the issue down into its bare components to further analyze. First, there's the identification and site functional analysis ... typically we call this the "crawler phase" or "discovery phase" depending on which tool you're using. Crawling the site (or application) means clicking buttons, inputting data, and traversing the site all while building a virtual map of what the site looks like, what the option trees are, and how traversal through the site is done legally without attempts to subvert the site. The next major step is the pre-attack analysis - whereby the tool attempts to build the attack sequences and tree for how the site will be attacked. This type of phase generally involves a lot of heavy memory and processor usage and building incredibly large and complex data structures (generally in machine memory). Once this is done the attack sequence can begin. Once the tool is confident that all attack patterns and plans have been laid out, the attacks are launched and the tool starts to do the heavy lifting it was built for. Inevitably during the attack process something new is discovered. Whether at attack pattern triggers some new function, or something breaks in a beautiful way ... the system has to put that newly found functionality back into the control-stack of the application for re-analysis and another pass. The tool will continue making the start -> discover -> attack-build -> attack -> repeat loop over and over as long as new things are discovered... until there is nothing new left on the discovery stack. Once the tool reaches that state it can be understood that the attack and discovery phases are complete and the tool moves to a final attack-analysis phase. At this point it will have to correlate, verify and validate the findings from throughout the process to make sure that there aren't any issues with these findings. The last step is to present it to the requester via a report. Whether the report is a dashboard, a PDF, or exposted XML or CSV the reporting piece is usually pretty standard and well understood. Having this process completely self-contained and automated is what some people seem to want - and I'm here to tell you that's a dangerous thing to ask for.

So now that we have the problem identified ... let's go talk about what options we have, why people are required and doing this completely in an automated fashion is a bad, bad idea.

...

There you have it ... the problem is now identified, unmasked, and ready to be discussed in detail. The upcoming post will detail some of the options we have for solving this issue and what technological limitations we are faced with today, and into the future. The last post in this series will go deep into the reasoning for why I continue to say that your brain will always be required. Until next time!

Is Anybody Listening?

RafalLos — Thu, 15 Oct 2009 16:22:00 GMT

Greetings, I am finally back home after an exhausting trip which had me speaking at 2 conferences back-to-back in separate countries and on opposite side of the coast! I did learn some valuable lessons from speaking at these two wildly different conferences thought, so I thought I would share them with you here for your benefit too.

First off, the Information Security conference I attended on Tuesday in Toronto called "SecTor" was brilliantly run and targeted towards Canadian-based information security professionals and wanna-be security professionals. It's OK to say it, there are plenty of people that attend these conferences who are looking to break into the business and want to learn about information security enough to get a grounding of what the industry is about... so they attend these conferences. My talk "When Web 2.0 Attacks" was well-attended and I even had some big names in my audience (thanks to RSnake, Hoff and a few others that wandered in and out) and I think the overall impression was that the stuff I presented was relevant to people's daily lives in Information Security. That's kind of the problem though...

You see, while I ordinarily wouldn't think twice about educating those in my field ... someone that's been doing this for a while longer than I reminded me a while back that this is what we would call "preaching to the choir". Sure, I tend to agree that even within Information Security not enough people understand Web App Sec well enough to build a program and actually reduce any real risks - but those folks have been hearing this talk for years upon years right? At some point I'm bound to hit the law of diminishing returns; and furthermore, people who didn't agree with me 6 months ago aren't likely to agree with me today. Great conference, great mind-share but it's definitely time to reach a broader audience.

That's where the next conference I spoke at comes in. Wednesday morning, at 4:00am Central time (yea, AM) while some of my colleagues were stumbling into their hotel rooms in downtown Toronto I was hopping into a car and being driven to the airport to head out west. My destination was Anaheim, CA where I would speak at StarWest later that day. I'm still not sure how through the delayed flight, sickness, and almost-missed connection I made it out to the West Coast by 2pm, but I did... and Star West was awesome.

StarWest (run by the SQE folks (www.SQE.com) is nicely put together and serves an entirely new audience of people. Here at StarWest (although I did find it strange that we were in the heart of DisneyLand!) the audience was almost entirely composed of software test engineers, managers and those related to the field. This was a completely different set of ears than what I'm used to ... this was a good thing.

The first thing I heard when I put my welcome slide up was "Hey, isn't security supposed to be done by the security people?" Love it. This is exactly the mentality and walls I was there to break down. I think as we went through the hour-long session on "Detective Work for Testers..." I managed to convince a few people in the audience that their jobs were closely tied to mine in Information Security. Maybe, maybe not. The bottom line is that there were many great folks who came up to me and talked afterwards and through the end of the conference about the absolutely missing component in their SDL that was security. I had one lady in the audience (although she fled before I could get more out of her, and had to track her down myself later on the show floor) tell me that her security team is the developers and that because they tell the bosses that they don't have security issues no one ever tests the code. I wish I could recall where she worked, hopefully no place important like a bank or anything ...

The point is - this was the right audience. If you were there and came to my talk, awesome! If you missed it, slides are posted and we can talk about it whenever you have some time.

Do you believe that Information Security and Software Quality testing is one and the same? Do you believe that a quality defect may as well be a security defect? Can you successfully explain the difference between a security and quality bug?

... I'm fairly sure I have my target audience for the next foreseeable future. Listen up quality testers - I'm coming to a conference near you!

SecTor - Meet n' Greet

RafalLos — Tue, 29 Sep 2009 22:54:00 GMT

Hey everyone ... I thought I'd consolidate all the thoughts around the SecTor Tweet-Up that have been floating around Twitter (via SecurityTwits and myself) into a single blog post... so here it is...

When:             Tuesday, October 6th at 10:00pm local time
Where:           The Loose Moose (Google it) - 146 Front Street West, Toronto, ON M5J 1G2, Canada‎ - (416) 977-8840‎
Who:               Everyone who's attending SecTor (or not) that's involved in Web Application Information Security (or security in general)
Why:               Meet me, and possibly Dennis Hurst of Hewlett Packard's Application Security Center ... and meet other InfoSec people!

There you have it. Nothing formal ... just come and meet... talk and have a good time!

Also.... my talk is on Wednesday ...

Time:          10:45 - Noon
Room:        203-D
Title:           “When Web 2.0 Attacks: Understanding Security Implications of AJAX, Flash and ‘Highly Interactive’ Technologies”

... see you at SecTor!

The Dangers of a Disaster-Driven Security Program

RafalLos — Thu, 17 Sep 2009 15:27:00 GMT

Reality check... at least 30% of the customers I have worked with this year use a "disaster-driven" security program.

Yes, it means exactly what you think. Nothing gets done, nothing gets approved until there is definitive proof that the $company has been hacked, stolen from, or otherwise compromised.

While we as security professionals often joke that this is the best way to make our point and get budgetary consideration - this is actually a very poor way to run things! Why you ask? Let's analyze this situation. There are many dangers to being reactionary and jumping on the emergency du-jour... not the least of which is money waste, catastrophic loss, and resource confusion and absolute loss of direction. I think it's best if we address each of those points individually to make everything nice and clear.

It would almost seem logical to only spend money when things go wrong- that way you know where your weakness is and you can patch the things that are broken. After all, you don't buy new tires because you think you'll be getting a flat, right? You buy a new tire when the old one blows out, or wears too thin. Same with the hot-water heater, your roof and pretty much anything else in real-life... There are serious logic flaws in that thought process. First off, we all know it costs many, many more pennies to "clean up" after a disaster than it would have taken to avoid the disaster in the first place... hrmm... or do we? You see, falling into this mental trap is easy... putting together the right logic to avoid it is quite difficult.

Let’s first talk through how you would measure these options, in order to provide empirical evidence. The important thing is to measure events which would be relevant to your business and model. So if you’re an industrial company, with very little web presence trying to substantiate the need for site security… good luck. Measuring involves accumulating the costs, all of them, of a disaster-driven approach. Inclusive costs would be things such as data-breach notification, legal fees, productivity loss, projected consumer confidence loss and other things that are very soft measurements … again making the empirical approach difficult here. Whether you do your own research or trust industry models – you will likely come to the conclusion that fighting fires with band-aids is more costly than being proactive… guaranteed.

The next important point against disaster-driven security is that catastrophic loss. Since I often liken InfoSecurity to life insurance let’s take that approach. We all know you can’t buy life insurance after the patient has crossed that line… I think we can all agree on that. The same is for security… sure you can beef up your defenses after a major disaster in security – but the damage is done! Whether you’re now dealing with untrusting customers or partners… you’ve got a tough hill to climb to win over those people again. This of course is completely ignoring how brutal the media can be… and then there’s the “Social media” that is merciless as well… Giving a press interview saying “Yes, we did everything we could pro-actively and still got breached” is much different than “Well, we were defenseless, but at least we’ll be ready for the same attack next time!”… obviously. Catastrophic loss leads to internal turmoil, profit shrinkage, people losing their jobs… and all sorts of nasty things… trust me, I know first-hand.

If you’re ever been a part of a data-breach or worked for a company that’s been hacked you know how difficult it is to work in that environment. Having leadership which either “follow the trends” or are “disaster-driven” means you’ll never actually successfully complete a project start to finish. This is true because odds are you start to plan, maybe even get into implementation before something strikes and you’re forced to drop everything and go do something else. Without continuity in your work life it gets confusing and you start to lose your place, projects are forgotten, and there is a lot, and I mean a lot of wasted everything.

Lastly, we address “loss of direction”… which at this point should be a self-evident outcome. When you’re chasing fireflies it’s very simple to run off the pier, since you’re looking not at where you’re going long-term but at all the pretty shiny lights all around you lighting up and dying off. Imagine, just imagine, if you had to chase one emergency after another. Imagine what that would do for your ability to resource plan, budget, and get a clear sense of direction for your department or company. It’s the perfect analogy for what’s going on in companies that have disaster-driven security practices.

Like it or not, many of you work in a company that believes security should be driven by incidents not strategy. Whether you want to or not, you’re enslaved by “running around putting out fires” and have very little sense of direction. Maybe it’s time you do something about that?

You’re #1 weapon against disaster-driven security is foresight, and metrics. You’ve got to anticipate, and measure carefully to prove that there is more risk in waiting for a disaster to occur, than being pro-active. After all, that is what everyone in this industry should be doing.

Good luck!

What are you delivering?

RafalLos — Wed, 26 Aug 2009 04:01:00 GMT

Caution: This post may make you uncomfortable

What business value are you delivering to your business?
.
.
.
... still trying to answer the question?

If you can't immediately answer the question of "What business value is your web application security program providing to the business?" then you're in for some serious trouble. The sad thing is, 2 out of every 3 security analysts that I talk to cannot answer that question without stumbling. This of course underscores the problem that we face as Information Security becomes more and more ingrained into the business. Security, just like every other part of the business, must justify its existence and value.

Should security have to justify itself? Does security have to have a value proposition? -- Of course!

When trying to answer that question consider one of the following answers...

Driving risk-reduction
Saving [the company] money
Making [the company] money
Contributing to compliance

Answering this question seems a lot easier said than done, doesn't it? Simply saying "we find vulnerabilities" brings about the inevitable "so what?" response from someone who is not intimately familiar with the principles behind security. Vulnerabilities in themselves aren't of any inherent value... consider that for a moment. Even making web applications your business publishes to the web, to partners or to customers "less hackable" doesn't really provide the business any value. Sometimes perspective hurts.

Try quantifying the work you or your team does in cost-savings, risk-avoidance, or other measures that the business understands and it's a whole new game. Making Information Security a key stakeholder in the business; however, requires exactly this type of answer to be given about every undertaking the security team has. There is no magic answer, no secret Jedi mind trick that will make your answers work every time. You'll have to work these out yourselves every single time but there is a single key to being relevant and getting funding.

Define what you do, what you're proposing and then ask yourself... "So what? What does this mean to the business?"

SaaS: The Definitive Cliff Notes on Web Security Delivered

RafalLos — Wed, 22 Jul 2009 00:53:00 GMT

Grab a cup of coffee, make some room on your calendar and read on.

...

This whole thing started earlier when, while reading through the mass of posts on every mailing list I belong to, I came across a question about SaaS services on the WebAppSec "Web Security" mailing list. This got me thinking and after someone responded I decided to chime in myself. Given that there was little discussion about the topic until another vendor stepped into answer the question from their marketing angle I thought it would be appropriate to pitch HP's SaaS solution as well... and that's when things got interesting. Immediately I got some responses from folks eager to understand what the differences were from SaaS to the newly labeled "Cloud Security" services (shoot me now, please?) and how that all differed from traditional service-oriented offerings. Folks also commented off-list about some of the challenges they think this poses... so I thought it only prudent to address much of that and throw in a few remarks for clarity to round this all out.

The Backdrop

Let me start this conversation off by saying that we'll be focusing on Web Application Security testing/scanning ... if that wasn't obvious. Now then, let me state that SaaS isn't really anything that hasn't been offered before, under a different name. Software as a Service used to be known as the ASP model (Application Services Provider) or just plain services before that. SaaS means that the customer gets a piece of software delivered, typically over the web, in a manner that does not require them to build out everything internally (but more on that in a minute) and in the process save on resources, time and money. Software as a Service (SaaS for short) is heavily mis-understood in that many vendors call it different things, and worse-yet they try and throw many of their offerings into the SaaS bucket without really giving much thought to whether it's appropriate or not! The SaaS offering from many vendors is a response to the recent economic downturn, and rightfully so as many companies simply can't afford the internal resources associated with web application security. What customers simply aren't getting is the bigger picture - that there is much more to SaaS than cost-savings, and the ability to use operating budget vs. capital budget - which is a big distinction in places I've worked before, and many other large enterprises. Allowing an enterprise to increase security and decrease risk by purchasing a service-based offering while drawing from the operating budget is a blessing many enterprise CISOs have been waiting for.

The SaaS Facts

The financial angle: As I hinted at above, the SaaS model has become very popular in the 18 months or so due to the rising need to cut costs, while trying not to cut [security] corners. Security managers are being asked to save the company dollars, and reduce their budgets while the increasing risks from doing business over the web continue to mount. Risks don't go away just because budgets shrink, so companies large and small are looking for ways to pinch the penny and figure out ways to quite literally do more with less. The financial angle on SaaS makes a world of sense when you consider most enterprises count SaaS as part of a run budget, or operational expense. The distinction of operational versus capital expenditures is a stick one. Capital expenses mean that there is some good being delivered by a vendor... that good depreciates over time and there are soft costs associated with capital expenditures that are added by savvy CFOs to calculate the true cost of acquisition of some product. Capital budgets, it's no secret, are drying up all over the place. This leads CISOs hungry for answers and ways to mitigate their security risks (or at least identify them!) without having to spend from their captial budgets. The true beauty of SaaS shows up when you start to address the product as a service. What if... you could get everything you want and not have to pay for the product - but rather simply have it delivered as a service over the web? This question often raises many CISOs eyebrows... What if...
Being able to purchase a web application security scanning tool without having to pay for it all at once (and not having to finance it) begins to make sense quickly when you realize that this is an operational expense. Much like keeping the servers powered and cool, or the Windows boxes up-to-date and patched... a SaaS expenditure means getting an on-going service that cheats and can actually deliver you product. Saving money is top-of-mind all over, and if you're crunched on budgets and still want to talk web security... SaaS may be the saving grace you've been waiting for.
The resource angle: Think this through - how many qualified web application security resources does your company have? If you've answered anything above 1 you're already in the minority. Given that most large enterprises have north of 1,500 web applications and less than 3 qualified web application security specialists that's a nasty 500:1 ratio. How, then, does the job get done? Your answer should be either automation or out-sourcing. The next question is can you afford to outsource all the work to an equally qualified web application security vendor? The answer is likely no... but then that usually doesn't matter because you pick the applications that are most critical, get those reviewed and move on thinking you're safe. Wrong. Remember that in all likelihood you're going to be in a mixed and shared hosting environment. Having just one of your lower-class applications owned by some "evil hacker" means that the rest of the dominos are likely to fall as well. This brings us back to the question - how do we get over the 500:1 ratio? The answer is a combination of automation, oursourcing, and services... known affectionately, here at HP, as SaaS. The ability to supplement an enterprise's internal staff, and provide on-the-fly scaled servics is something that most CISOs drool over, and if you can bundle that up with a world-class web application security scanning tool (we can say that, we have the awards to prove it) delivered over the web and accessible from anytime, anywhere... then you have a true winner. Being able to augment the work your over-extended security team has on their plates is something SaaS vendors all strive for, but few do well. In addition to your human resources, think of the investment in computer resources that have to be made to bring an enterprise-grade web application security scanning solution in-house. Do you have $50,000 - $200,000 to spend on the infrastructure behind the web application security scanning solution you choose? If you're like most enterprises... the answer is a firm "heck no". So what SaaS offers here is the complete out-sourcing of the infrastructure to handle such a huge deployment including servers, operating systems, storage and all. One complete bundle... pretty cool huh?
The unreal flexibility: Would you like to have 1 monitored application? How about several? Whether you are looking for a single quarterly scan of your entire public web presence of a daily scan of everything you own, or an on-demand project-based service which grows and shrinks as demand does, or simply a fully-managed daily-use platform for your internal teams... it's all here in a complete SaaS offering. Flexibility is one of the keys that makes a good SaaS offering something your company can live with, and grow with. Forget being stuck with the all-or-nothing approach... demand that you get this kind of flexibility from your vendor and if they can't deliver switch. Make sure you've got uptime guarantees, data storage redundancy, and 24x7 live human support... otherwise you're not getting your money's worth!

The Dangers

There are, of course, dangers to moving to this SaaS model. Nothing is free of any downside so make sure you understand the risks!

non-local data storage -make sure your company policy allows for the non-local storage of critical company information (including a direct map to your most vulnerable assets)
vendor dependence - it's no secret that once you become dependent on a vendor you choose it's very difficult to migrate off that vendor keeping your data and processes intact
3rd party access - since there will now be a 3rd party 'testing your web systems for vulnerabilities' make sure that your company policy allows for this, and has provisions for privacy and contingency for disclosure; your vendor of choice will have access to your most intimate secrets (where you're most vulnerable)

The Benefits

Let's face it, the benefits for a program like this are numerous so I'll attempt to name some of the major ones here...

cost savings - perhaps paramount to any good security program is the fact that SaaS has a high probability of saving you hard dollars
efficiencies - a SaaS solution from your vendor of choice, unlike your employees, does not sleep or require days off... it simply works and works to make sure that your company's web-based assests have defects identified and teed up for remediation
scalability - whilte your company can only scale so much internally with human and technology resources, your SaaS vendor does not operate on these same restrictions; this creates an extremely scalable solution
knowledge - a well-orchestrated SaaS solution will not only provide industry's finest technology but some of the best people out there as well, giving you access to knowledge, training, and security intelligence you simply could not afford to staff up internally
ROI - most security professionals don't talk ROI [Return on Investment] which is rather unfortunate as one of the nicest things about a SaaS offering [particularly from HP's Application Security Center] is that the ROI doesn't need to wait for servers to be built, software to be installed, people to be trained... you start getting real value out of your purchase almost immediately and that translates into a healthy return on investment calculation
value - everything considered... the amount you pay for a SaaS solution is generally a fraction of the cost of deploying this same solution internally... more for less translates to business value pure and simple!

If you've finished this, and are now wondering how you can get some more information on just how you can save money, and get more web security... feel free to contact me directly via the contact link in this blog... I'm happy to talk about our solution, or anything else floating around out there...

StarWest - Where QA and Security Will Collide

RafalLos — Mon, 06 Jul 2009 18:15:00 GMT

Is site security QA's problem too?!

Hi everyone, I can't wait for fall and the StarWest testing conference in Anaheim! I'm so psyched to be presenting "QA Techniques for Identifying Workflow-Based Security Defects" in what will hopefully be one of the better talks of the week. I've been promising many of you an explanation of why QA and IT Security cannot live without each other, as it pertains to web app security, and I aim to deliver.

This talk will be heavily focused on the reasons why IT Security still fails in many instances to find serious web application security defects - and what the Quality Teams can do about it. How about that... identifying security as more than just "security's problem" - it's an enterprise-wide problem that bleeds very much over into the QA testing organization. The days of the security teams doing "scans" and pitching the results over the cubicle wall to the developers are long, long over (were they ever really here?) and the days of collaborative defect mitigation throughout the application lifecycle are here. Come listen and learn some of the techniques that the QA testing teams can use to identify security-based defects in the web applications; and understand why it's not just security's problem anymore.

But wait! there's more! Just in case you're thinking to yourself... sure I'd love to go but I don't think I have the travel budget - there's a discount code yours truly has gotten made up just for you, my readers! Simply follow the instructions below - and I'll see you in Anaheim at StarWest.

Simply click this link: ( http://www.sqe.com/go?SW09Rafal ) and enter the code SWRL to get your discount...

Here's the abstract - just to get you psyched up too!

"Workflow-based web application security defects are especially difficult on enterprises, because they evade traditional, simple, point-and-scan vulnerability detection techniques. Understanding these defects, and how/why black-box scanners typically miss them is the key to creating a testing strategy for successful detection and mitigation. Rafal Los describes the critical role that application testers play in assessing application workflows and how business process based testing techniques uncover these flaws. Rafal demystifies the two main types of workflow based application vulnerabilities: business process/logic vulnerabilities and parameter-based vulnerabilities. As the complexity of web applications continues to increase, learn how to adjust your testing strategy to make sure you don’t miss these unique types of defects."

Blog Comments

RafalLos — Mon, 06 Jul 2009 18:09:00 GMT

OK, I give up boys and girls... the spammers have me out-gunned. When I sift through 1,000+ pieces of SPAM comments/day it's time to call it quits.

Admitting defeat isn't pleasant but that only means that I'll be turning OFF the ability for you anonymous folks to comment. I will continue to moderate and allow everyone to comment, but only if you register. Now, I have no misgivings that this will actually stop blog comment spam, but it should annoy them enough to leave my blog alone ... maybe.

Thanks for reading!

Quality Engineers & Testers - StarWest is Coming Up!

RafalLos — Thu, 02 Jul 2009 20:45:00 GMT

I'm thrilled to announce that I have been selected to speak at the StarWest 2009 Quality Conference (SQE) October 5-9th 2009, hosted at the DisneyLand Hotel in Annaheim, CA! Link to the conference website is here (http://www.sqe.com/starwest/Schedule/Default.aspx) and there are a number of awesome speakers as well!

The StarEast conference was chock-full of great speakers, vendors and of course yours-truly... speaking on Security topics and why the quality assurance teams are so crucial to the web application security process. That's right, I've been talking about Q/A engineering and testing teams and why they're so crucial to the success of any enterprise web application security program - but now for the first time you'll get the truth that the IT Security guys probably won't tell you - YOU are the key! My talk on this topic promises to be riveting and will certainly have an impact on formal testing and security organizations...

As an added bonus - if you sign up you'll get money OFF the price of your admission!

Normal 0 false false false EN-US X-NONE X-NONE

Register using special promo code SKWS and save up to $300! Register by September 4^th to add the Early Bird Discount for up to $600 in total savings! Call the client support group at 888.268.8770 or register online at: https://www.sqe.com/starwest/Register/SelectConference.aspx

I'll see you all there!

Blog Comment SPAM...

RafalLos — Fri, 26 Jun 2009 04:21:00 GMT

Hi everyone - I apologize in advance if you write a nicely thought-out comment to one of my posts and it gets "lost in moderation"... I have recently started getting north of ~250 SPAM comments every 12 hours or so (as quickly as I clean them up) so I may accidentally delete yours as part of the SPAM. My sincere apologies, and I trust you'll sympathize with me when I say I do my best - but sometimes I just can't beat the spamming machines out there. I make sure I don't get any SPAM published to my blog - but at the same time it's entirely possible that the baby goes out with the bath water.

Thanks for your patience and understanding.

~Rafal

The Problem of "Too Many Problems"

RafalLos — Wed, 24 Jun 2009 21:45:00 GMT

Hey everyone... now that I'm back to regularly posting I thought I'd address the issue I've faced with the last few customers we've gotten the pleasure of visiting. Speaking from experience you never want to introduce a tool or process that will overwhelm people - it tends to cause that "deer in the headlights" reaction... and we all know how well the deer normally fares, right?

That being said, I think I've found a problem that's plaguing companies that are lagging in the Web App Sec program department (yes, there are companies who are now just starting to get into the thinking of building a program to protect their web sites and applications) even more than the political problems they face, or which tool/service to purchase. Nope... there is even a bigger problem.

I will frame it for you by giving you an example of a recent product validation I was a part of... and what turned out to be a brilliant disaster. It was brilliant in that our suite of web application black-box testing tools performed beautifully, and uncovered way more than the customer (or the competing products) were expecting. This is where the disaster struck... As this was all being written up into a nicely packaged recommendations and findings document we sat down with the person who was in charge of the validation project. Immediately, my contact's face turned a few different shades of pale before going completely white - he was shocked at what we found and how simple it was to completely and fully compromise their main public-facing website. After the initial shock wore off and color returned to his face he was excited that we would be presenting this to the "senior security group" headed by the CISO the next day. Here is where things started to go sideways.

The presentation the next day kicked off as expected... we presented our executive summary, the methodology of our product validation and moved on to the specific findings. In this case, since there was so much wrong I stripped out only the Critical and Highly Important issues and bundled the rest into a "non-mission-critical" bucket for the sake of brevity. My goal was to move through that into our recommendations section where we would propose what the customer should do next, including building a security validation program and starting to integrate into the SDL; let's just say I never got that far...

As soon as I hit the Criticals section I noticed something wrong. Immediately the faces of the folks in the room started to look... befuddled I think is the correct word. Some got that glazed-over look I get when my wife tries to explain the complex relationships of her friends and such... they were overwhelmed, lost, and confused. I stopped and asked if there were questions... no one raised their hand or spoke up so I continued. I got about 1/2 way through the critical issues section when the CISO, hand half-raised, looked at me and said "This is way too much ... I just don't think we can handle it". Naturally I thought he was talking about the depth of the presentation... or the mountain of information I was giving them... nope - he was referring to the number of things that we had found that were wrong with the site.

What happened next is nothing short of terrifying... faced with the mountainous task of comprehending, addressing and remediating these 100+ critical issues the CISO was frozen in his place... like a deer in the headlights. I guess I can't really blame him given that any one of the 20+ SQL Injection vulnerabilities uncovered could entirely compromise their main system. The question on the table now was - what do they do? After thinking about it and talking amongst themselves for a few minutes (at this point the presentation was halted) they agreed that there was nothing they could do - and they would simply leave the production systems alone and simply focus on the websites which were currently in development. The deer had gotten so bedazzled with the oncoming car it didn't care that it was about to become an inevitable victim.

What just happened? What we have here is a failure to launch. Given the overwhelming nature of the issues, some companies simply choose to "do nothing" because the resource requirements of tackling all those issues seem impossible. Rather than sitting down and prioritizing sites, vulnerabilities and required effort they simply choose to be overwhelmed. Well, I'm not sure choose is the right word here... but you know what I mean. It's not trivial to look at over 300 critical and high-ranked vulnerabilities on your main site and resolve to prioritize and address everything in a sane fashion. It takes resources and a tough resolve to make that happen... and it's not a trivial task as I've said.

That being said... the lesson learned here is that I shouldn't take for granted that given a moutain of security risks - someone will be able to find a starting point and then be willing to push that boulder up hill. Going forward I'm definitely going to be offering more tactical and strategic assistance from the years of experience solving these sorts of partially-psychological problems in security.

If you're faced with one of these crises... don't despair! First remember that others run into the same headlights. Second - don't think you can't do it... with a good risk-based approach and a sound methodology you can transform that overwhelming moutain of security risks into a big win for you and your department.

Good luck!

Enterprise Web Application Security: Part 2 - The Policy

RafalLos — Tue, 23 Jun 2009 07:30:00 GMT

In the first part of this series titled "Enterprise Web Application Security: Part 1 - The Foundation" I left you with 6 foundational things you should consider before galloping head-long into building a web application security program. Going forward I'll take as a given that you've defined those 6 items, and you have good definition of all of those components.

After you've defined the baseline for your security program the next most important thing to do is exercise your patience. This second stop involves a lot of thinking, writing, and processing time. You'll be building a policy and framework for the entire program going forward, so don't spare the details.

Building a policy is difficult, let's face it. You have to have some sense of actual security inside, with a tint of business logic so that you have buy-in from your upper-management, and now you've got to think about how to keep the people who will actually have to read and use it engaged. I'll address each of the 3 major groups you'll need to impress and give you some pointers on how to do that... so let's get going.

First off, let's discuss the format of your policy document. Over time, what I've found that works well in organizations large and small is the "Cook Book" approach. Folks in your organization want to know what they have to do to be in compliance with security - or they'll simply find a way to avoid security altogether. That being said your responsibility is to lay out a simple cookie-cutter approach to performing common tasks and projects in such a way that minimizes the need for interaction with security while creating a simple approach to building security into the project. To illustrate the point consider a project where your delivery team will be building out a private customer-centric site to interface with your internal systems. The project may be repeated dozens or hundreds of times each year and by creating a one-time approved and documented process, procedure and architecture document you'll save yourself and the project teams many cycles and meetings. They'll thank you for it later. In order to make this work though, you'll have to make sure you cover as many of the common project types as possible.

The last thing you'll need to remember on the format is to give people as many options (that you've pre-approved) as possible. There will be no one-size-fits-all approach that will work all of the time - so you need to make sure you cover as many options as possible and provide possibilities for accomplishing the same goal given budgets, resources and time. Don't forget to factor in budget, resources and time into your documentation or you'll regret it later when your hard work starts to gather dust.

The first group you'll need to cover are the people who make projects go - the project managers (PMs). They will need easy to understand cheat-sheet style documentation which can be printed out and stuck into a binder. Maybe you'll get lucky and your document's simple pages will end up on the head of the project management office... or maybe it'll be required reading during the project manager training process. At any rate - the policy must be comprehensible by the project manager. Comprehensible can be broken down into a few simple requirements: simple, short, non-technical, and short. I mention short twice because PMs have enough to worry about without yet another complicated document. Your policy should include a very short executive summary (the "why" for your policy), it should contain a logical "if - then" organized section which describes at a high level (at the business level) the causal and effect of the security requirement. If the business requirement is X, the security requirement will be Y must be laid out in simple to understand, non-technical language. The Y (the requirement itself) must be broken down into a general solution and a technical directive. An example of this is as follows: If the business chooses to store credit card numbers (the business requirement) they must be protected by an approved form of encryption (high-level requirement); this requires utilizing pre-built module A, B, C or code snippets D, E, or F depending on implementation or technical requirements. This example provides adequate if/then for the PM to grasp the requirement and then provides some light technical guidance on how to solve the problem.

After you've created a section for each of the cookie-cutters you're creating that's designed for the project manager you're going to want to address the development resources. These technical resources often have no care for the business angle of the requirement only the technical solution which must be included to pass security audit later on. Quite simply put these are the technical options that security is providing. You can provide code snippets, pre-reviewed and approved modules or an architectural approach to the issue. Securing a project can often take on many forms and it's important to include as many options (within reason, and that are technically sound) as possible to ensure that at least one if not more of the approved methods is feasible. Feasibility is often overlooked when creating a document like this... security professionals simply bark out requirements without looking into whether it's even possible to execute them within the parameters of the project. I've often heard from project managers that IT Security is trying to force the business to spend a million to protect a thousand. Make sure you don't run into this type of obstacle - as it will definitely deter adoption.

Lastly, and this group is only last because I'm saving the most important group for last, you need to address management. Eventually someone from either the PM or the Development groups will challenge your requirements and solutions. Ensure that upper-management understands and can make sense of your documentation. Getting this level of understanding will usher a whole new level of cooperation among business and IT and security, I promise. Covering not only the technical approach, but also the project-management approach is one thing... but to make your policy document readable by someone in upper management who may or may not even understand what IT is doing... that's a whole new level of brilliant. Upper management has a surprising thirst to "do the right thing"... but often times they're constrained by two major factors - understanding and budget. You can't always address budget (which is where I tell you, sometimes you just have to let them accept crazy risks), but you can definitely address understanding. Overcoming that taboo between management and security technology can not only create a very cooperative relationship but it has been known to lead to random acts of proper risk management. I once shared the elevator with a CIO who, after being asked to read and approve the security policy I had set forth not a week before, let me know that in the last meeting of his direct reports when someone mentioned a new project that they were considering - he was able to throw out the security requirement for that "type" of project. He smiled in self-approval as he described how silent the room fell... and how everyone could simply not believe the CIO was spouting security requirements. It was quite an accomplishment.

So there you have it - we've gone through a lot of material - but the policy requires a lot of attention to get it right. It's like building a good, strong castle. The foundation is everything... but the cornerstone is the policy.

Next we'll be addressing the first real step you'll be making - taking a baseline - and this time I hope not to have a month in between posts... thanks for reading!

Timless Cliche..."Doing More with Less"

RafalLos — Fri, 08 May 2009 16:18:00 GMT

I believe it was the caveman who first say "Ug, unga, munga, ug ug" which roughly translated to "We need to figure out how to do more with less". Since those days man has struggled with that eternal issue of getting higher productivity with less resources.

Looking around at the economic situation it's obvious to see why this issue presents itself so prominently today... which is why I'm thrilled to post the link below. Yes, it's a blatant ad for HP App Sec Center's tools and services but it is helping my customers answer that problem that's as old as humanity.

Give it a go, it's a short but informative view on how you can leverage our latest tools to get your web app security teams to be able to "do more with less"... There is a registration required, so you've been warned :)

Link here: https://h30406.www3.hp.com/campaigns/2009/events/sw-05-05-09/index.php?rtc=3-2P5BKUK