Following the White Rabbit

Quality Engineers & Testers - StarWest is Coming Up!

RafalLos — Thu, 02 Jul 2009 20:45:00 GMT

I'm thrilled to announce that I have been selected to speak at the StarWest 2009 Quality Conference (SQE) October 5-9th 2009, hosted at the DisneyLand Hotel in Annaheim, CA! Link to the conference website is here (http://www.sqe.com/starwest/Schedule/Default.aspx) and there are a number of awesome speakers as well!

The StarEast conference was chock-full of great speakers, vendors and of course yours-truly... speaking on Security topics and why the quality assurance teams are so crucial to the web application security process. That's right, I've been talking about Q/A engineering and testing teams and why they're so crucial to the success of any enterprise web application security program - but now for the first time you'll get the truth that the IT Security guys probably won't tell you - YOU are the key! My talk on this topic promises to be riveting and will certainly have an impact on formal testing and security organizations...

As an added bonus - if you sign up you'll get money OFF the price of your admission!

Normal 0 false false false EN-US X-NONE X-NONE

Register using special promo code SKWS and save up to $300! Register by September 4^th to add the Early Bird Discount for up to $600 in total savings! Call the client support group at 888.268.8770 or register online at: https://www.sqe.com/starwest/Register/SelectConference.aspx

I'll see you all there!

Blog Comment SPAM...

RafalLos — Fri, 26 Jun 2009 04:21:00 GMT

Hi everyone - I apologize in advance if you write a nicely thought-out comment to one of my posts and it gets "lost in moderation"... I have recently started getting north of ~250 SPAM comments every 12 hours or so (as quickly as I clean them up) so I may accidentally delete yours as part of the SPAM. My sincere apologies, and I trust you'll sympathize with me when I say I do my best - but sometimes I just can't beat the spamming machines out there. I make sure I don't get any SPAM published to my blog - but at the same time it's entirely possible that the baby goes out with the bath water.

Thanks for your patience and understanding.

~Rafal

The Problem of "Too Many Problems"

RafalLos — Wed, 24 Jun 2009 21:45:00 GMT

Hey everyone... now that I'm back to regularly posting I thought I'd address the issue I've faced with the last few customers we've gotten the pleasure of visiting. Speaking from experience you never want to introduce a tool or process that will overwhelm people - it tends to cause that "deer in the headlights" reaction... and we all know how well the deer normally fares, right?

That being said, I think I've found a problem that's plaguing companies that are lagging in the Web App Sec program department (yes, there are companies who are now just starting to get into the thinking of building a program to protect their web sites and applications) even more than the political problems they face, or which tool/service to purchase. Nope... there is even a bigger problem.

I will frame it for you by giving you an example of a recent product validation I was a part of... and what turned out to be a brilliant disaster. It was brilliant in that our suite of web application black-box testing tools performed beautifully, and uncovered way more than the customer (or the competing products) were expecting. This is where the disaster struck... As this was all being written up into a nicely packaged recommendations and findings document we sat down with the person who was in charge of the validation project. Immediately, my contact's face turned a few different shades of pale before going completely white - he was shocked at what we found and how simple it was to completely and fully compromise their main public-facing website. After the initial shock wore off and color returned to his face he was excited that we would be presenting this to the "senior security group" headed by the CISO the next day. Here is where things started to go sideways.

The presentation the next day kicked off as expected... we presented our executive summary, the methodology of our product validation and moved on to the specific findings. In this case, since there was so much wrong I stripped out only the Critical and Highly Important issues and bundled the rest into a "non-mission-critical" bucket for the sake of brevity. My goal was to move through that into our recommendations section where we would propose what the customer should do next, including building a security validation program and starting to integrate into the SDL; let's just say I never got that far...

As soon as I hit the Criticals section I noticed something wrong. Immediately the faces of the folks in the room started to look... befuddled I think is the correct word. Some got that glazed-over look I get when my wife tries to explain the complex relationships of her friends and such... they were overwhelmed, lost, and confused. I stopped and asked if there were questions... no one raised their hand or spoke up so I continued. I got about 1/2 way through the critical issues section when the CISO, hand half-raised, looked at me and said "This is way too much ... I just don't think we can handle it". Naturally I thought he was talking about the depth of the presentation... or the mountain of information I was giving them... nope - he was referring to the number of things that we had found that were wrong with the site.

What happened next is nothing short of terrifying... faced with the mountainous task of comprehending, addressing and remediating these 100+ critical issues the CISO was frozen in his place... like a deer in the headlights. I guess I can't really blame him given that any one of the 20+ SQL Injection vulnerabilities uncovered could entirely compromise their main system. The question on the table now was - what do they do? After thinking about it and talking amongst themselves for a few minutes (at this point the presentation was halted) they agreed that there was nothing they could do - and they would simply leave the production systems alone and simply focus on the websites which were currently in development. The deer had gotten so bedazzled with the oncoming car it didn't care that it was about to become an inevitable victim.

What just happened? What we have here is a failure to launch. Given the overwhelming nature of the issues, some companies simply choose to "do nothing" because the resource requirements of tackling all those issues seem impossible. Rather than sitting down and prioritizing sites, vulnerabilities and required effort they simply choose to be overwhelmed. Well, I'm not sure choose is the right word here... but you know what I mean. It's not trivial to look at over 300 critical and high-ranked vulnerabilities on your main site and resolve to prioritize and address everything in a sane fashion. It takes resources and a tough resolve to make that happen... and it's not a trivial task as I've said.

That being said... the lesson learned here is that I shouldn't take for granted that given a moutain of security risks - someone will be able to find a starting point and then be willing to push that boulder up hill. Going forward I'm definitely going to be offering more tactical and strategic assistance from the years of experience solving these sorts of partially-psychological problems in security.

If you're faced with one of these crises... don't despair! First remember that others run into the same headlights. Second - don't think you can't do it... with a good risk-based approach and a sound methodology you can transform that overwhelming moutain of security risks into a big win for you and your department.

Good luck!

Enterprise Web Application Security: Part 2 - The Policy

RafalLos — Tue, 23 Jun 2009 07:30:00 GMT

In the first part of this series titled "Enterprise Web Application Security: Part 1 - The Foundation" I left you with 6 foundational things you should consider before galloping head-long into building a web application security program. Going forward I'll take as a given that you've defined those 6 items, and you have good definition of all of those components.

After you've defined the baseline for your security program the next most important thing to do is exercise your patience. This second stop involves a lot of thinking, writing, and processing time. You'll be building a policy and framework for the entire program going forward, so don't spare the details.

Building a policy is difficult, let's face it. You have to have some sense of actual security inside, with a tint of business logic so that you have buy-in from your upper-management, and now you've got to think about how to keep the people who will actually have to read and use it engaged. I'll address each of the 3 major groups you'll need to impress and give you some pointers on how to do that... so let's get going.

First off, let's discuss the format of your policy document. Over time, what I've found that works well in organizations large and small is the "Cook Book" approach. Folks in your organization want to know what they have to do to be in compliance with security - or they'll simply find a way to avoid security altogether. That being said your responsibility is to lay out a simple cookie-cutter approach to performing common tasks and projects in such a way that minimizes the need for interaction with security while creating a simple approach to building security into the project. To illustrate the point consider a project where your delivery team will be building out a private customer-centric site to interface with your internal systems. The project may be repeated dozens or hundreds of times each year and by creating a one-time approved and documented process, procedure and architecture document you'll save yourself and the project teams many cycles and meetings. They'll thank you for it later. In order to make this work though, you'll have to make sure you cover as many of the common project types as possible.

The last thing you'll need to remember on the format is to give people as many options (that you've pre-approved) as possible. There will be no one-size-fits-all approach that will work all of the time - so you need to make sure you cover as many options as possible and provide possibilities for accomplishing the same goal given budgets, resources and time. Don't forget to factor in budget, resources and time into your documentation or you'll regret it later when your hard work starts to gather dust.

The first group you'll need to cover are the people who make projects go - the project managers (PMs). They will need easy to understand cheat-sheet style documentation which can be printed out and stuck into a binder. Maybe you'll get lucky and your document's simple pages will end up on the head of the project management office... or maybe it'll be required reading during the project manager training process. At any rate - the policy must be comprehensible by the project manager. Comprehensible can be broken down into a few simple requirements: simple, short, non-technical, and short. I mention short twice because PMs have enough to worry about without yet another complicated document. Your policy should include a very short executive summary (the "why" for your policy), it should contain a logical "if - then" organized section which describes at a high level (at the business level) the causal and effect of the security requirement. If the business requirement is X, the security requirement will be Y must be laid out in simple to understand, non-technical language. The Y (the requirement itself) must be broken down into a general solution and a technical directive. An example of this is as follows: If the business chooses to store credit card numbers (the business requirement) they must be protected by an approved form of encryption (high-level requirement); this requires utilizing pre-built module A, B, C or code snippets D, E, or F depending on implementation or technical requirements. This example provides adequate if/then for the PM to grasp the requirement and then provides some light technical guidance on how to solve the problem.

After you've created a section for each of the cookie-cutters you're creating that's designed for the project manager you're going to want to address the development resources. These technical resources often have no care for the business angle of the requirement only the technical solution which must be included to pass security audit later on. Quite simply put these are the technical options that security is providing. You can provide code snippets, pre-reviewed and approved modules or an architectural approach to the issue. Securing a project can often take on many forms and it's important to include as many options (within reason, and that are technically sound) as possible to ensure that at least one if not more of the approved methods is feasible. Feasibility is often overlooked when creating a document like this... security professionals simply bark out requirements without looking into whether it's even possible to execute them within the parameters of the project. I've often heard from project managers that IT Security is trying to force the business to spend a million to protect a thousand. Make sure you don't run into this type of obstacle - as it will definitely deter adoption.

Lastly, and this group is only last because I'm saving the most important group for last, you need to address management. Eventually someone from either the PM or the Development groups will challenge your requirements and solutions. Ensure that upper-management understands and can make sense of your documentation. Getting this level of understanding will usher a whole new level of cooperation among business and IT and security, I promise. Covering not only the technical approach, but also the project-management approach is one thing... but to make your policy document readable by someone in upper management who may or may not even understand what IT is doing... that's a whole new level of brilliant. Upper management has a surprising thirst to "do the right thing"... but often times they're constrained by two major factors - understanding and budget. You can't always address budget (which is where I tell you, sometimes you just have to let them accept crazy risks), but you can definitely address understanding. Overcoming that taboo between management and security technology can not only create a very cooperative relationship but it has been known to lead to random acts of proper risk management. I once shared the elevator with a CIO who, after being asked to read and approve the security policy I had set forth not a week before, let me know that in the last meeting of his direct reports when someone mentioned a new project that they were considering - he was able to throw out the security requirement for that "type" of project. He smiled in self-approval as he described how silent the room fell... and how everyone could simply not believe the CIO was spouting security requirements. It was quite an accomplishment.

So there you have it - we've gone through a lot of material - but the policy requires a lot of attention to get it right. It's like building a good, strong castle. The foundation is everything... but the cornerstone is the policy.

Next we'll be addressing the first real step you'll be making - taking a baseline - and this time I hope not to have a month in between posts... thanks for reading!

Timless Cliche..."Doing More with Less"

RafalLos — Fri, 08 May 2009 16:18:00 GMT

I believe it was the caveman who first say "Ug, unga, munga, ug ug" which roughly translated to "We need to figure out how to do more with less". Since those days man has struggled with that eternal issue of getting higher productivity with less resources.

Looking around at the economic situation it's obvious to see why this issue presents itself so prominently today... which is why I'm thrilled to post the link below. Yes, it's a blatant ad for HP App Sec Center's tools and services but it is helping my customers answer that problem that's as old as humanity.

Give it a go, it's a short but informative view on how you can leverage our latest tools to get your web app security teams to be able to "do more with less"... There is a registration required, so you've been warned :)

Link here: https://h30406.www3.hp.com/campaigns/2009/events/sw-05-05-09/index.php?rtc=3-2P5BKUK

The Pros of a Con

RafalLos — Thu, 07 May 2009 18:22:00 GMT

My slides are now available for you to revisit (if you would like a copy for yourself, email me), here at SlideShare.net

Most of you already know how much I love to stand in front of fresh set of faces to deliver the message of web application security. Every once in a while I get the rare pleasure of venturing out of my element (the "security" audience) and talking to an audience that doesn't actually have any vested interest in security... yet. This past couple of days spent with QA engineers and software testers at StarEast has been absolutely priceless. I can't tell you how much joy it brings me to see the light bulbs go on as you slowly make the transition to knowing and understanding what should make you lose sleep - that security is everyone's problem.

The audiences that came out to the "Hacking Flash" talk, as well as the "Building Practical Test Cases..." talk were wonderfully interactive and really game me faith in the fact that the message is reading more and more of the right folks. I know I mentioned at least a few times that the QA teams are the key to security's success, and I'll further explain for everyone else why that's the case in a later post - but know it's absolutely, 100%, no-bs truth.

When you get back to your jobs stop by your security team's desks and tell them that you are the secret step in a successful security strategy and program. Take a snapshot of their reaction. Remember, there is generall a 10:1 ration of QA to security staff... and your effectiveness is measured in the depth of cooperation with your security teams in decreasing your enterprise's risk on the web.

Given all that, I want to post some follow-up stories. I would like to get some brave souls to reply to me (either privately or here as a comment) with how you've taken this information back to your organization and utilized it for the company's greater good. What kinds of reactions have you gotten? Please let me know and I would love to have you guest-blog either semi-anonymously, or identifying yourself if you'd like!

Thanks again! Good luck out there.

Raising the Bar? Flash Encryption, Obfuscation

RafalLos — Mon, 20 Apr 2009 17:37:00 GMT

On the heels of my OWASP talk regarding decompiling and analyzing Flash [see SWFScan link] files lots of you have asked "So what about Flash file encryption or obfuscation? Does that make my code any more secure?" I've done the research and talked to experts (including our very own Billy Hoffman) - and have a blog post just for those of you starving for this information.

There are a lot of Flash file obfuscators/encryptors out there... all of them hoping to raise the bar for attackers against your client-side Flash code. I'd like to make sure I properly set the background for you here - everything you'll read about is happening on the client side within the user's browser framework, meaning, it's running in potentially hostile territory.

Now let's move on and take a look at some of the ideas we're addressing. First when-ever you're discussing client-run code you have to understand that whether it's encryption, obfuscation, or magic you have one major problem: the client has to know how to un-do the magic. When the Flash! file comes to your client it has to be interpreted by the Flash! player, right? In order for it to do that it has to be readable and understandable by the Flash! player. Think about that. If the code is sent encrypted, say using some strong AES-256 encryption technology, then the player is unable to render it thus creating a quite secure, but completely unusable "blob". In order for the code to be worth anything to the client it has to be de-crypted (or obfuscated). For that to happen you have to have the routine to decrypt | de-obfuscate the blob located within that blob, likely as a pre-cursor to the whole piece of code. You should already see a huge gaping hole here. Here's what this all looks like in terms of process from developer's workstation to client player:

Developer writes some [potentially bad] ActionScript code
Developer "obfuscates | encrypts" the code
User hits page, downloads embedded "blob"
User starts to execute Flash! file
Decryption|De-obfuscation routine runs, produces valid (unprotected) Flash!
Flash! player executes code, movie runs

Immediately, in the above step-by-step you'll notice that step 5 the decryption|de-obfuscation routine has to run on the client to unprotect the SWF file. This essentially breaks down to mean that the deobfuscation|decryption code has to exist on the client, within the protected SWF file. Ask yourself what sort of security that buys you, if you're including the unprotect routine with the protected code.

After wading through many different SWF encryption|obfuscation tools I've come to realize that they're selling to folks who simply don't understand the full scope of the problem. Here is an interesting quote from one vendor's marketing material. I'm not identifying the vendor... mostly to protect them from the questions you would have about their effectiveness.

"PRODUCT X uses Advanced Obfuscation Techniques along side proven Encryption Technology to provide security and protection for your Adobe Flash® SWF Files. Put simply, PRODUCT X prevents other people from decompiling or reverse engineering your SWF movie and stealing the ActionScript Code." How can a vendor make a statement like that, consciously, knowing full-well that this is just like the obfuscation techniques that are being used on JavaScript right now... their effectiveness is only marginal to the determined attacker. You have to continue to put these types of technologies into context because if you're looking at "encrypted content" you'd think that it's secured, much like an "encrypted database" is secured from someone who steals it... until you realize the main difference is that generally the decryption routine is not included with the database but as a separate process. This is the main difference. Since Flash! player has no internal mechanism to decrypt|de-obfuscate flash files the work falls to the application itself, meaning it has to be included into the code blob.

The verdict? If you're depending on a code obfuscation|encryption tool to protect your Flash! files, you should probably re-think your strategy. First ask yourself why you're hoping to hide the client-side code. Intent here is key... because while the tools you're using may temporarily deter a simple Flash decompiler, in the long-run it will not protect your code. As Billy Hoffman notes "Client-code obfuscation|encryption is much like WAF (Web Application Firewall) technology, it's a temporary fix meant to increase the "time to hack" while not providing anything permanent." Including sensitive information on a client-side code blob is never a good idea. This should be self-evident but apparently there is a significant market for Flash! obfuscation|encryption tools so maybe I'm wrong. Here are a few pointers for those of you thinking about writing sensitive client-side Flash! apps...

Never put sensitive information on a client (passwords, 'hidden" URLs, validation routines, encryption routines, etc)
Understand that anything on a client can be compromised because you no longer have control
Any encryption|obfuscation of client-side code has to be un-done in order for the framework to process it... thus only providing marginal security improvement

When all is said and done, to quote Billy Hoffman "It's like boxing a 7 year-old... you're going to win it's just a matter of how hard do you want to try". --Thanks to Billy Hoffman of HP’s Web Security Research Group for his contribution to this blog, and his ongoing effort to protect developers from their own worst enemy... themselves.

OWASP, Security Bloggers Finalist and more

RafalLos — Thu, 09 Apr 2009 18:14:00 GMT

First off - let me say how absolutely honored and humbled I am right now... after this past week's events.

That out of the way, I want to wrap up an absolutely wonderful week up here in Canada with the 3 OWASP chapters I spent time with over the last few days. Specifically I want to thank Benoit Guerette of OWASP Montreal, Sherif Koussa of OWASP Ottawa, and Nish Bhalla of OWASP Toronto for having me and showing an out-of-towner a wonderful time in each of your beautiful cities. You three have some of the most vibrant, enthusiastic OWASP chapters I've ever had the pleasure of working with. I encourage you to keep up the good work, meet regularly and get good quality speakers so that your numbers grow. I feel honored and lucky to be among the speakers you've invited in.

Next there's the news I got today over my twitter feed (I'm RafalLos on twitter, in case you want to follow my random thoughts). Apparently you all voted Following the White Rabbit, this very blog right into being a FINALIST for the Security Bloggers awards at RSA 2009. I'm blown away. My goal is to write some quality posts that security professionals, practitioners and management can read and get value out of - and if I'm succeeding it's because of all the feedback, comments and off-line things that I receive. Let me say again how honored and humbled I am to be among the ranks of people like Tim Callan at VeriSign, the SunBelt blog, the F-Secure blog... really - this is awesome!

So for the above reasons - thank you, and I will work even harder to continue to provide content you'll want to read and share with your colleagues and friends.

See you in the bits!

Let's talk "Rich Internet Applications"

RafalLos — Thu, 02 Apr 2009 14:10:00 GMT

Hello again everybody! My apologies for going radio silent for a bit - it's been a rough March with travel and visiting many of you over the past several weeks... and guess what? it's not going to be any better in the near future! The good news is that for those of you living in Canada (Ottawa, Montreal, Toronto) I'll be seeing you next week at the OWASP meetings scheduled.

Monday - April 6th - Ottawa, ON, CA
Tuesday - April 7th - Montreal, QC, CA
Wednesday - April 8th - Toronto, ON, CA

The topic is "Rich Internet Applications" and while the talk isn't currently named (officially) you can go cast your vote thru tomorrow on the "Name That Talk" poll going on on my Preach Security blog, please take a moment to vote!

I will be hanging out afterwards to meet up with everyone... so if you can make it, let's chat!

My InfoSec 2009 Slides Are Available!

RafalLos — Fri, 13 Mar 2009 06:20:00 GMT

In case you missed it, InfoSec World 2009 is at a close here in Orlando, FL. Thanks to the standing room only crowd that came out to see the talks - it was really stellar!

As promised, here are the slides, posted online for your viewing. If you are lookingn for the demos we did during our Total Browser Pwnag3 talk, Josh Abraham, my co-presenter, will have the voice-overs done soon and posted. I will link to it from here when that's ready.

Without further ado... here are the slides! http://www.slideshare.net/RafalLos

Enterprise Web Application Security: Part 1 - The Foundation

RafalLos — Fri, 20 Feb 2009 15:10:00 GMT

The term "Enterprise Web Application Security Program" has been evolving. Generally referring to a corporate IT program which includes web application code in some way and has traditionally meant either a white-box approach or a black-box approach, either through the use of tools or the use of a 3rd party for the assessment.

No matter how you look at it, that's all completely wrong.

First off the thought that a "security program" would begin with code is a failure to launch, in my experience. Web application security deals so much more with non-code items than we'd like to believe, but rarely address. These topics include hosting, server hardening, user-management, and a few others I'm forgetting now. The point is before you can bulid a strong web application security program that withstands not only economic cycles but business trends you have to understand what it is you're building. Much like a home, you can't change plans after the foundation has been laid...or else you will fail.

There are some fundamental components you must consider before you start to lay the foundation for your Enterprise Web Application Security Program... here are some of the most important ones that have to be addressed from day one...

Intended purpose - In order to solve a problem you must know what that problem is; you must understand what the purpose of the program you're building is going to be in business terms.
Long-term vision - Define what you see this program evolving into 6, 12, and 18+ months down the road; clearly identifying a long-term goal will assure that you don't start straying in different directions as you progress
Success criteria - There must be a clear definition of how success or failure will be measured; if there is no way to measure failure you will never understand if you've succeeded (or failed) in your goals. Setting realistic success criteria in a concrete context (as opposed to "secure the company's web applications") makes it real to reach those goals and achieve success, while setting milestones helps you focus on making little changes over time that don't happen overnight
Metrics - Setting success criteria and having clear metrics go hand-in-hand when building a framework and foundation for a successful program. Just as you have to have a goal you must be able to measure that goal accurately, at any given point along your path in order to assess your rate of success
Scope - While it may sound rudimentary to say that your program must have scope it is not to be confused with vision or success criteria. Scope can keep your program in-focus and prevent creep into areas you are not equipped to handle. Scope-creep is one of the most widely identified preventatives to success... if the finishline is always moving you will never be able to reach it
Identified starting point - Yes, it's critical to identify where you are starting- this goes back to gathering metrics and measuring success. Very rarely does a program start at "nothing" actually; there is always some degree of movement already - you must quickly identify your starting point so as to build from that point forward

There you have it - there are six (6) identified components to a foundational approach to building an enterprise web application security program. If you're starting to put together a framework for such a program; no matter whether it's due to compliance needs or internal pressures, make sure you understand at least those six pieces. Write them down, remind yourself regularly of their existence.

As master Yoda said - "Do, or do not do, there is no try".

Next time we will address framing that first step of building your program - the policy.

An Unfortunate Case of Learned Behavior

RafalLos — Tue, 10 Feb 2009 16:25:00 GMT

One of my all-time favorite quotes is "When all you have is a hammer, the whole world is a nail"... but lately that quote has started to apply to the practice of web application security programs, and that's causing me to start losing sleep.

Speaking to large-scale enterprises about their budding web application security programs has seen its share of challenges - but when one of the problems is the program's implementation... then we're faced with a different kettle of fish altogether. The issue arises from a mis-guided approach to solving enterprise problems based on a single, vendor-inspired, point solution. Spotting this imminent failure is rather simple, here are some warning signs you're about to head down a dangerously failure-laiden path:

Your enterprise web application security program is based on a point solution from {insert vendor here}

"solution" can be a tool
"solution" can be a service

You determine risk rating based on a pre-defined risk rating of high | medium | low

if you're taking a vendor's risk rating without your own context
if you don't add context to your vulnerability rating

You believe that tools can replace people in security vulnerability detection
You believe that people are more effective than tools in security vulnerability detection
You believe your enterprise is safe because your {black-box scanner | white-box scanner | service-provider} says so

The foundation for any good web application security program, one that will stand the test of time and circumstance, is diversification in the methods for risk-detection and analysis. I've said it over, and over... a diversified program must be at the root of your long-term strategy. Remember, every good practicioner understands the place of tools, services, and internal processes ... and never trusts his company's security and personal success to any single one of the aforementioned items.

The reality is this - every tool and human tester has a shortcoming, a blind spot - something {it | they're} simply not great at. Flipping that around ... every penetration tester has a specialty - SQL Injection for Oracle, Cross-Site Scripting, whatever... and tools are the same way. The problem with all of these is that if only one of them is used you end up with a narrowly-focused spotlight, completely missing everything else. As an example, if your web application is tested by a human being who's great at exploiting Oracle-based flaws but isn't well-versed in CSRF and other flaws, you will only get part of the story. Just like if you were to scan your application with a white-box scanning tool... since the analysis is static-driven you won't actually know what's really going on... in a dynamic (data-driven) test. Either way, you're left with an incomplete picture. Relying on this incomplete picture as your security posture leads to a false sense of security, and a feeling that if you fix the identified flaws in {inserver vendor here} report, your enterprise will be secure. That logic simply does not hold water.

This type of failing approach will lead to a condition of "You don't know, what you don't know"... and if you assume there is nothing else to know... your web application security program just failed to mitigate the most important risk - that of the unknown. Therefore, the answer is unbelievably simple, and unfortunately uncomfortable for some of the vendors in the security practice out there... you have to dip into both buckets. This is precisely why people seek the second opinion of another doctor - to get another look at the same problem.

The irony is that if you give 3 excellent penetration testers the same exact web application, which is relatively complex in nature (as almost all enterprise apps these days are), you limit their exposure window, limit the amount of money you can spend on their efforts... odds are you will get 3 slightly varied results. Worse-yet... one of those results may be the vulnerability that causes your next appearence on the front page of the Wall Street Journal. The same goes for web application security scanners... black box, white box... doesn't matter. Given the complexity of enterprise applications you can start to see why these differences arise...

So in these trying times which is the right course of action? While your mileage may vary and there is no one-size-fits-all solution, here's my recommendation... these few simple rules have served me well in some of the most challenging enterprises in the world:

Layer your strategy on top of a baseline of education; this does not have to be in-depth security training only a base knowledge of the CWE Top 25, or some other list if you need something for reference
Implement a baseline of tools throughout your lifecycle because you simply can't go wrong with minimizing risk by removing "low-hanging fruit" off the tree
Adopt a major/minor release analysis model, which looks like this:

Major releases [Y.0, (Y+1).0, (Y+2).0] receive special attention from an outside party (your favorite penetration testing bunch) and your in-house set of vulnerability scanning and analysis tools
Minor releases [1.x, 1.x+y...] receive scrutiny from a set of in-house built tools, based on an SDL approach which gives greater coverage and better chance of catching bugs
Define major/minor releases as...you see fit for your organization's budget, release strategy, and SDL type

Good luck. Remember... when all you have is a hammer, everything is a nail.

Defining Security as a Business Requirement

RafalLos — Thu, 05 Feb 2009 04:53:00 GMT

This post is a follow-up to the previous one on QA: Defect vs. Vulnerability. All the highly-intelligent responses I received got me thinking further, and so here I present my additional thoughts.

This may not be revolutionary - but given the response I received regarding the terminology difference between defect and vulnerability I think the only logical conclusion we can reach is that if security is not a foundational business requirement, we're sunk.

To expand on this point a little more I think it's important to follow non-technical critical-thinking here. Anything that does not make it into the functional specification of an application [web or otherwise] is an afterthought. It has been conclusively [and repeatedly] proven that anything that is not "baked in" as a requirement is nearly impossible to fix later on, as an after-thought. So we're presented with a puzzler. Security must be a business-level requirement. So how then does one translate vulnerabilities into a business requirement, sanely? Simply stating "... the application shall be free of unintended design flaws and security vulnerabilities" is like asking an architect to build a structure that will withstand every known (and unknown) possible attack - it's simply illogical.

Strangely, program leads that manage these large-scale web applications at the heart of nearly every major breach want concise, identified things to not put into the code... but since that list is a moving target the security team gets penalized for the nature of security itself. This is the reason why black-listing input is a losing proposition... you're always going to be in an arms race with the bad guys... and you'll never win.

I've heard some recent conversations hit the wire around using the CWE Top 25 or some other list as a definitive list of coding errors to avoid in web applications but I'm not sure if that will actually solve the problem. The problem with this approach is and will be that these lists are exclusionary measures. These lists illustrate what we must exclude to be [more] secure. Turning it around and making statements like validate all input makes little more sense, especially given that input validation must be defined in the context of the situation, and there is never a one-size-fits-all answer. To illustrate the point further - input validation may mean excluding certain character sets/patterns and pre-defining acceptable input options ... but this does not account for things like free-form input or other use-case specific examples.

In the end, the crux of the problem lies in the nature of security vulnerabilities. Security vulnerabilities are a moving target and although they can be loosely defined and lumpted into Top 7/10/25 lists it is not logical to consider these lists complete or even functional for designing software. Will a web application be secure if it follows the CWE Top 25 and addresses those issues? What about the OWASP Top 10? I don't think anyone has that answer, or at least is willing to stake their reputation on it.

So back to defining security as a business-level requirement... can it be done? Can one clearly articulate requirements to secure data/transactions/processes/whatever *before* the technologists get involved; meaning, before the means to execution are defined? I will leave that up for debate.

QA Lesson - Defect vs. Vulnerability

RafalLos — Tue, 03 Feb 2009 20:34:00 GMT

Back in April 2008 one of my very first posts to this blog was titled "Security Vulnerability != Defect; Why?" and it stirred some discussion. Over the past year I've spoken to more QA teams than I can probably remember, and the message has been carried through - but a recent article by Robert Auger at CGI Security gave me a mental nudge and got me to think this through again. As I sat and wrote a reply to Robert's article, I was once again reminded of the example I use; I will share it here for everyone's comment.

Consider the following example of defect versus vulnerability usage... and perhaps you'll be able to use it in your next discussion.

The context for this is a conversation I had with the an IT leader at a very large auto manufacturer, sitting around a conference room table just after I had presented to them some security concepts and why security testing was a critical component to their organization. After I was done, the IT Lead in the room (who won't be named here) looked at me and shook his head. Curious, I couldn't help myself but to ask why he was spending so much money on quality-assurance, but not a penny on security; more importantly why I had not convinced him to spend security dollars. This was his answer, and I think this perfectly illustrates my point, and why I now use defect instead of vulnerability when speaking to QA groups.

paraphrasing... "Vulnerabilities and defects are not the same thing, in fact, vulnerabilities aren't, in my opinion, anywhere near as important as quality defects. In this industry, a defect is illustrated by a brake pedal that goes to the floor as your brakes fail when you mash them to stop in case of an emergency. The brake failure is a defect which can lead to the loss of human life. Defects are taken very seriously, and we do everything we can to avoid them. A vulnerability is much different. A vulnerability is likened to a car that is not theft-resilient. A thief being able to easily break into the car, hot-wire it bypassing the starter-kill mechanisms (obviously without the keys) and drive away is a vulnerability. Which do you suppose you would focus on if you were faced with both in front of you?"

That struck me, and stuck with me. While this IT Lead's analogy was spot-on for the automobile industry... unfortunately it didn't quite hold for the software development organization. A defect was, in most cases, arguably not quite as critical as s security vulnerability - but neither will likely cause a loss of life (one would hope). Anyway... I thought I would share this story so that the next time you're talking QA... remember to use the correct terminology... it makes the point that much more... understandable.

President Obama's Web 2.0 Campaign Hijacked

RafalLos — Wed, 28 Jan 2009 21:00:00 GMT

Congratulations Mr. President, your Web 2.0 campaign to be the "hip" president has just been hijacked. In an interesting news article published originally on CyberInsecure.com, someone has decided to use the President's popularity to hijack his potential, and unsuspecting, users and drump malware on their machines.

I won't go into details, you can read it all here, but the moral is simple... I've been telling you folks that hackers don't just want your databases and credit card data... they want your CLICKS too. The more popular your site is, the harder someone will try and break it (silently) to inject things like adware/malware for their own purposes... which by now should be obvious - make money.

Protect your sites... and your users.