More software testing words of wisdom from my fortune cookie. Like the last one I blogged about, this quote captures a fundamental truth about the testing profession: Many of the bugs you find will never be fixed.

This blog post helps explain why:

“We live in a world where there just aren’t enough software developers. No matter what stage of development you may be at, your team could always use just a few more developers to build that great feature marketing wants, fix that extra bug that’s been nagging technical support, help build some tools so that software development can work more efficiently, etc. But sadly, we live in a world of constraints and that means that the marginal cost of any investment has to be paired with the marginal benefit it will bring.

To  improve this situation (that is, getting more of your bugs fixed) it would be wise to consider this recent piece of advice from uTester Joseph Ours: Change the way you report bugs.

And to all of our friends from other nations, we just wish you a safe and happy Saturday!

uTest:  And when all is said and done what will be the professional accomplishment you’ll look back on with the most pride?

JW:  Creating an actual discipline around software quality. Note I said quality and not testing. I want software projects as a whole to run more smoothly and more predictably. I really think that’s what software testing is all about — reducing the uncertainty of software development and finding ways to muscle errors out of the process. A process in which mistakes are harder than doing the right thing is the ultimate goal. We can’t eliminate them, but we can make doing the right thing to be the easiest thing to do.

uTest:  What’s your first assignment at Google?

JW:  To raise the level of testing precision and diligence. Google has a lot of smart testers, my job is to help mold them into a serious fighting force and let our bugs beware. But this isn’t so much an individual commitment. Google has a culture of collaboration that I am fascinated by as a Noogler.

We share offices (which might explain their interview strategy), inhabit common areas, collaborate constantly and work as a community. If I am successful, there will be many people who can take credit and if I fail, I won’t go down alone! I think the whole free food thing is at the heart of this as food is often the centerpiece for bringing people together. Lots of work gets done while your mouth is full. I hope to succeed before I have to buy bigger clothes.

uTest:  Rumor has it that you have a new book coming out.  What’s it about and when will it hit Amazon’s shelves?

JW:  It’s in production now. I hope it’s available later this summer. The title is Exploratory Software Testing: Tips, Tricks, Tours and Techniques to Guide Manual Testing.

[uTest note:  Since James is too shy to promote it, we'll do it for him.  The new book is available for pre-order at Amazon... get your advance copy today!]

uTest:  Will you be implementing test tours at Google? If so, which ones in particular?

JW:  That’s going to be up to Google’s engineers. I will most certainly teach them everything I know and be there to work alongside them. Good ideas tend to stick, so this little Google adventure will help decide whether the tours work here or not!

uTest:  In your opinion, what does the future of software testing look like?

JW:  I actually just wrote it up. It’s chapter 8 of my new book. But I am working on a pre-publication version of this particular topic.  I’m not yet sure whether it will be out as an eBook form or a presentation that I do separately. Watch the Google test blog for updates on this!

uTest:  How do automated and manual testing coexist in the future - are the compliments or substitutes?

JW:  They will co-exist better than they do today, but this question requires a longer answer.  Suffice it to say that every good automated test I have ever seen started out as a manual test. How’s that for coexistence!

[uTest note:  this would probably make a good topic for a future webinar with James]

Have other questions for James?  Want to tell us which master of the testing universe we should interview next?  Drop us a comment or send a note.

Several of these questions came directly from our community of testers.  The whole exchange is fairly lengthy, so we’re splitting it into two posts.  Come back and check out the 2^nd half later this week.

uTest:  So the news is out about your move to Google. What prompted you to make this move?

JW:  I didn’t so much leave Microsoft and I did join Google. I was attracted by all the Googlers I met at conferences and what I read on their blogs about the way they test. When they offered me the opportunity to be a part of it, one might even argue an important part of it, I found it impossible to decline.

uTest:  Is there something about Microsoft you’ll miss the most?

JW:  Yes, the breadth of both products and expertise. You literally have every type of software imaginable and a chance to collaborate with the people who make that software. From an intellectual standpoint, Microsoft is mind-blowing.

uTest:  What specific work at Microsoft did you enjoy the most?

JW:  Mentoring - making other people better. I love working with smart people and finding their passions. When you take a smart person and help them find and focus their passions, you have a powerful mix. Smart people who aren’t passionate creates a bad vibe. I think I was pretty good at helping people find that combination and enabling them to become the company’s top performers.

uTest:  That’s interesting. What is your approach to mentoring? Can you give us an example?

JW:  To attract passionate people, you have to be a passionate person. I don’t go out of my way to select mentees, we just discover each other. When I was at Florida Tech I supported a lot of students. Some came looking for the wages or to improve their chances of landing a job at Microsoft or Google, but the ones who were drawn by their intellectual curiosity where the ones that saw the most success. It’s been the same at Microsoft. I mentored folks who came and went and others that really got it and became stars.

uTest:  Can you tell us a little about the interview process at your former and current employers? They are both famously difficult and, now that you’ve experienced them both, do you have any tips for our readers?

JW:  During my Noogler orientation I saw stats on the accept rate of Google applicants — it’s astoundingly low. And although I never saw such stats about Microsoft, I have to imagine its low too. But the interviews themselves were fairly similar.

A lot of hard technical questions, both from a coding and testing perspective and also problem solving. Google kept me in a single room and brought the interviews to me. Microsoft shuffled me around from office to office. I think I prefer Microsoft’s approach from that standpoint, but the actual interviews were fairly similar. I think Google took a page from Microsoft’s playbook on that one.

Anyone out there who plans to apply at either place should be ready for problems that test your knowledge and your creativity. Good luck!

uTest:  With this new role, will you still be able to write, teach and evangelize testing?

JW:  I wouldn’t accept a job where I didn’t have to teach! Googlers will have every reason to get sick of me and I am still doing my STAR tutorial and occasional keynotes.

uTest:  So where can everyone find the soothing, savvy writings of Doc JW?

JW:  On the Google Testing Blog, of course! http://googletesting.blogspot.com/

Stay tuned for  part two of this interview with James when we’ll cover his new book, the future of software testing and a few other topics about his new gig.

You see, I am extremely clumsy with my mandoline.  Every time I’ve used it, I have somehow sliced open one of my fingers.  My mandoline is literally covered in knives - there are knives for slicing, knives for dicing, and even more knives stored underneath and inside the thing for torturing vegetables in additional ruthless and unimaginable ways.  The whole thing is one big vegetable and finger slicing machine.

The crazy thing about my mandoline is that despite the fact that it cuts me every time I even look at it, it’s not actually broken.  In fact, it’s in great shape.  All of its knives are sharp, its frame is sturdy, and it slices and dices just like one would expect.  I’m sure there is some kind of spec somewhere that says “verify mandoline can smoothly cut onions exactly .125 inches (3.17 mm) thick.”  I’m sure mine passes with flying colors.  What the spec probably doesn’t say is “verify mandoline won’t cut fingers.”

Specs can’t capture these sorts of negative tests very well.  You can write them, but a spec like that is a lot like a spec that says “make sure your software doesn’t crash.”  It’s very nice to include, but also kind of worthless.  This is where good testers can really show their worth.  A tester can think beyond the spec and relay feedback that can be just as valuable as any bug or error.

What are some of the evil mandolines in your projects?  If you relayed them to your developers or product managers, do you think you could make your software better?

Testers and testing are viewed as a cost center in many organizations. If you look at the roles of other “main” players, you quickly see that testers face what I call an issue of intangibles. Here’s what I mean:

• Project managers - They are task masters driving a product to completion. Businesses absolutely want products created on time and under budget - which is why they are (correctly) viewed as an absolute necessity.

• Analysts - These guys get the great job of descriptively conceptualizing the idea. This is akin to a paper prototype, and gives the business the first real glimpse of how it might look and work.

• Developers - They are the cream of the crop. They get to create an actual product that businesses can see and feel.

• Testers - Well, we say if it works or not.

Everyone else gets viewed as making a positive contribution in getting a product to market. Unfortunately, testers are in the position of “finding” things that can mar the perception of what is being created. This can lead to confrontational situations that are, to say the least, unpleasant. So, how can we as an industry change this perception of our craft? There are many answers, to be sure, but  I would like to focus on a single element of our profession that elicits much of this ill will: defect reporting. Actually, I would rather call them observations, but that’s another post entirely.

The primary purpose of defects is to point out what we believe to be a notable observation about the application. Typically, a defect report is about something that doesn’t work the way we think it should.  This inevitably leads us to write a report where we layout the steps used to encounter the defect. However, one thing we often leave off of such reports is why we logged it. After all, a defect is an observation. As with all observations, there is a point of reference. For software, it can be a requirements document, UI standards, historical experience, etc. Without our reference point included in the defect, the receiving party is left to guess why we thought it was wrong in the first place.

This can lead to all sorts of issues, but it generally leads to at least one meeting for clarification. But the bigger impact is by NOT saying why we think it is wrong, as we can lose credibility and the opportunity to demonstrate tangible value.

Consider this: In addition to recreating the steps, what if we included a reference point as to why something is a defect AND articulated the loss of value to a stakeholder (usually an end-user)? Would others view our observations, and us, as more valuable?  Of course they would, and isn’t that the point?

For example, instead of saying:

“I entered in ‘John Smith’ in the name field, hit the tab key, and received a javascript alert”

We said:

“I entered in ‘John Smith’ in the name field, hit the tab key, and received a javascript alert. The javascript alert contained an error message regarding null objects. End users experiencing this behavior may be inclined to believe the registration process has failed and may leave the website.”

We have stated an observation, a point of reference, and a value item related to a stakeholder. But it is not just end-users that have a value concern. Let us take another example.

Instead of saying:

“I entered ‘<script>alert(’me’);</script> into the name field and the application accepted it on submission. On the confirm registration page it displayed an alert message with the text ‘me’ and allowed me to successfully register”

We said:

“I entered ‘<script>alert(’me’);</script> into the name field and the application accepted it on submission. On the confirm registration page it displayed an alert message with the text ‘me’ and allowed me to successfully register. The system allowed un-encoded fields to be submitted and subsequently executed commands. This could allow a malicious person to execute a cross-site script attack against the application; leaving our system and users vulnerable.”

Here we stated our observation, our reference, and our concern. This makes our observations more useful to stakeholders. Ultimately, this makes testers more valuable as well. We get the opportunity to demonstrate our value; for example, how our efforts improved end-user experience, the security of our system, and adherence to the conceptualized idea, to name a few.

This, of course, is precisely what we want. And one of the easiest, most effective ways to demonstrate the tangible benefits we testers bring to the table is to change the way think of - and write - defects in software.

Joseph Ours has more than twelve years of information technology experience in areas of requirements definition, analysis and design, program/ project management, full system life-cycle development, training, testing, and quality assurance. He has been a project manager, team lead, and team member, continually striving to provide the very best service and expertise in response to an ever-changing and increasingly complex IT environment. You can read his blog here.

Yes, the results of the Battle of the Desktop Twitter Apps are final. The week-long QA competition - in which more than 600 testers  searched for bugs in five of the most popular Twitter apps - was a  success. As expected, we received hundreds (320 to be exact) of interesting GUI, technical and functional bugs, so selecting a handful of winners was obviously no easy task.

In fact, it was so difficult that we decided to expand our winners list. Also, be sure to download our free special report (PDF), detailing the results of the post-battle usability survey.

As you’ll see from the results page, the Battle of the Desktop Twitter Apps saw several repeat winners, along with some fresh, new uTest talent. Here’s a top-line list of the winners:

• Top Tester: Bernard Lelchuck (Israel)
• Top Novice: Tyler Ritchie (USA)
• Top US Tester: Joseph Ours

Tyler and Bernard also finished 1st and 2nd in the “Best Bug” category. Other winners for “Best Bug” included:

• Claire Pelayo (Philippines)
• Bryan Fisk (New Zealand)
• Aymen Chehaider (India)

As far as the apps were concerned, the uTest community ranked Tweet Deck #1 in terms of usability, feature set and overall quality. Coming in second (also in every category) was Seesmic Desktop, followed by Twirl, Tweetr and Twitteroo.

The results have already been featured in a Mashable story yesterday, ase well as a bunch of other news outlets, and of course, hundreds of individual tweets.

Once again, congratulations to all of the Bug Battle winners, and thanks to all those from our community who participated. If you’re aleady looking forward to our next competition - and we know you are - be sure to send us your ideas. We can be reached at marketing@utest.com, and of course, on Twitter.

We’re sincerely honored (and yes, a tad bit surprised) to win this award when we launched our software testing marketplace just 10 short months ago. The “top new company” category was full of worthy and well-established finalists, including Aster Data Systems, Carpio, Evolve IP, HubSpot, Jivox, Mazooma and Zuora.

The 2009 Stevies saw more than 2,600 entries submitted from companies of all sizes and industries, and more than 200 executives across the country participated in the judging process. The ceremonies were hosted by Cheryl Casone of Fox Business Network and broadcast nationwide on radio by the Business TalkRadio Network. We’ll try to track down some pics and vids from the festivities and post them later.

As Doron said when he accepted the award, this is gratefully accepted on behalf of 20 uTest employees, nearly 18,000 testers in our community and more than 120 software companies that turn to us to help get their apps tested. We couldn’t do it without you all!

Discussion Threads

Starting today, testing managers have a new tool for working with the uTest community.  With discussion threads, it’s now possible to quickly and easily create a virtual meeting room so they can communicate with their testers and the uTest project manager.

Testers who are working on a project can “meet” in real-time, ask & answer questions and share information with each other about that particular release.  And customers have a great new way to share critical information or requirements while a release is underway.

Discussion threads will enable testing managers to better manage and interact with their virtual testing teams and ultimately create even higher quality apps.

(Click to see a larger screenshot.)

Application Profiling

Another great feature we’re introducing is a tool to help testing managers better match their applications to testers expertise through refined application profiling.  When creating a new release, testing managers will have the ability to classify their application and testing needs.  Our system will use then use this information to automatically identify testers who are more likely to match the release’s exact testing needs.

The drop-down menus will allow testing managers to select:

• Application Type - Web, Desktop, Mobile
• Testing Type - Functional, Usability, or Load & Performance
• Industry Type - Social Networking, Retail, Entertainment/Gaming, Financial Services, etc.
• Audience - Consumer, Business, or Both

This matching process is just the beginning, and coming soon we will release several additional features for both testers and testing managers.  In the coming month, we will be requiring uTesters to update their profiles for even stronger project matching.

In addition to these new features, this release applied fixes to more than 40 bugs (all found by our uTester community).

Some might think once the global financial crisis has come to an end these marketplaces will dwindle as people go back to work.  Instead, I believe that they will evolve further, supplying a more efficient and creative way for companies to engage with and harness the crowd for help.

uTest is included in BusinessWeek’s article on crowdsourcing leaders and the related slideshow uses a great quote from our CEO.

You can probably guess what we think about this topic:  open marketplaces and crowdsourcing are the most efficient, frictionless, collaborative way to deliver professional services like software testing!  But we’d love to hear from you.  Is crowdsourcing the natural evolution of outsourcing (as James Whitaker has said) and thus, here to stay?