Effective Software Testing Blog

The attitude of effective public speaking

2011-05-26T05:19:00.000-07:00

Ok, before I start talking, let me make one thing clear - I am not the best public speaker I know. But 9 out of 10 times, I am able to communicate my ideas across to other individuals or groups. So I am going to share what goes inside my brain when I am writing (emails, blog posts, test cases, defect reports) or making PowerPoint presentations or addressing a crowd.

In all these activities I am thinking "here's what you should know at the end of this conversation / email / whatever" not "this-is-what-I-want-to-say". The focus is "you", not "me".

It helps me go to the level of my juniors as well as seniors or a mix of both these folks.
It helps me project things in the dimensions my audience can relate to.
It helps me connect to each individual even when I am speaking to a crowd.
Yes, I am nervous before any public speaking opportunity, but I view it as just that - an "opportunity" to speak to many "individuals". When I am standing up there and my brain goes "OMG! All these people are staring at me, judging me", I am able to snap out of it by talking to one-person-at-a-time.
Similarly while writing presentations or defect reports, I am thinking about the consumers of the same. How can my defect report help the developer get to the problem quickly and fix it? How can I make the idea clear and acceptable to the viewers? If I were on the other side of the table, what are the questions I would like to ask.
This getting out of my head also helps me stay or at least appear calm.

So, next time you have to perform or communicate anything, think: "what-should-You-get", not "what-should-I-say".

Is there anything else that you do when you are in a similar situation? What's your favorite public speaking tip? I'm looking forward to hear. :)

As always, thanks for dropping by and do let me know your thoughts. Your kind words of encouragement and wisdom mean a whole lot to me!

Varada on Testing Circus

2011-05-19T22:55:00.001-07:00

Thank you to Testing Circus for publishing my article “3 essentials to setting expectations right” in their May 2011 issue. Click Here

Also, many thanks to Devyani Sharma for the follow up she did with them for getting this published.

Sweet day!! :)

Use “5 W" to write effective defect descriptions

2011-05-18T19:28:00.000-07:00

Effective communication, like effective testing is a skill. So, like all other skills, it can be learnt. Writing up a good defect report is certainly one area where learning effective communication can help tremendously.

For the scope of this post, let’s focus on the Summary (or Title) of your defect report.

What should a good defect summary include?A defect summary should provide brief insight into the problem you are reporting.

It can be as short as a single sentence or a couple of sentences, depending on the length of Summary field in your defect reporting.
It should be specific, clear and concise.
It should provide enough details for all stakeholders (or consumer of your defect report) to understand the defect and act on it.

What’s this “5 W” and how it can help?There are 5 possible questions that the readers of your defect report will like answers to:

What
Who
When
Where
Why

Let's call these "5W". If you answer these satisfactorily in the summary, it can convey necessary and sufficient data on the defect that you are reporting.

Let’s understand each one in depth:

What?
What is the actual result? What is the problem noticed?
Who?
Which user? Admin? Normal user? Someone with special privileges or role?
When (which action)?
When does the problem occur? By performing which actions can the defect be replicated?
Where?
In which part of the application - screen / module / user story?
Why?
If you are 100% sure of root cause of your issue, you could mention that upfront, so developers will have some pointers to start debugging on. Use this option with extreme caution. There’s a possibility that putting this information up front can mislead the developers, too.

We can combine all these questions in a meaningful sentence as:

Defect Summary Template

Using this template can tremendously simplify defect reporting task. Here are few more guidelines to go with it:

All of these questions are optional except for “What”. Obviously!
Be specific in your reporting: “Null reference exception” is much more informative than “an error occurs”.
Use active voice. The sentence becomes shorter and clearer.
Keep “What” not “when” as your focus, as the subject of your sentence. Compare “Null reference exception occurs on trying to search for a value that does not exists in Products table.” with “When user provides a non-existent value in search field on products table, she gets a null reference exception.”

For some examples, check these:

“Object not found” Javascript error when trying to save appointments using the web version of e-Organizer.
Application crashes with *** exception when Admin tries to inactivate multiple users from User Listing screen.
“Could not load your list” error when user tries to add a to do list item for today on the Android e-Organizer app.

When we realized in our big teams that different reporting styles often led to confusion and multiple cycles of defects going back and forth between qa and dev teams for lack of understanding, we resolved to use this template and are happy with the results:

Our defect summaries are more informative and follow a standard pattern across all levels.
Defects are easily searchable as there are similar keywords being used.
Management gets better picture of the criticality of issue just by going through the summary.

Do try this trick yourself and revert back with your experience or comments.

Thanks for dropping by!

Make your defect reports stand out in two steps

2010-06-09T02:50:00.000-07:00

Defect reporting is as much art as it is science. You have to convey just the right info, with the precision of a scientist for approvers to approve your defect, for coders to easily understand & replicate it and then to fix it.

Can't stress enough on how important it is for your defect report to be good, so that it's chances of being fixed increase. Think about your defect report as something that's not much different from applying to a bank for a grant to finance your business venture. Only if the bank sees value in your proposal - it clearly understands what problem are you trying to solve for which market sector (and how, of course), - will it consider to give you monies.

Step 1. That gives us point#1 of this post. The artistic part. One essential point your defect report needs to have is a business case.

I am not talking about straightforward crashes or unexpected error messages. They will be fixed without needing any convincing. This is about requirements level issues or feature requests or some portions that need to be changed in the functional workflow. I've seen a lot of such defect reports termed as "Fix in later release" (read never-fix), which could have really added value to the system had they been fixed.

The common ingredient missing from these reports was - they failed to convince the managers / approvers of their value.

So, how do you make a hard-to-refuse case for your defect. First step is to do your homework. Answer some of the questions to yourself to get you started.

Why do you think something needs to be changed?
- Is there a conflict between two requirements?
- Is the current way cumbersome? Can some of the steps in a lengthy workflow be reduced?
- Is the current way compromising on security? Or performance?
Research your competitors. Are they doing things in an easier, more intuitive way? Are there better ways of achieving your objective? Faster? Simpler?
What benefits are your suggested changes offering to the end user? More accuracy? Ease of use? Freedom to use your application on multi-platforms (web / desktop / mobile)?
Think of yourself as the end user. How would you like the application to be? Why?

Once you get answers to some of the questions here, state it out clearly. Once. Cut the clutter.

(If you think you need help with concise writing, check out this book. Written by author Robert Fiske, this one is definitely for keeps with over 10,000 alternatives to lengthy phrases. Here’s where you can get it:

Step 2. Alright. Moving on to the science part. The one that will help coders to understand, replicate and fix your defects - adequate information. And how do you get this info across quickly and effortlessly? (Yeah, I count typing in 500 words of how-to article in the description for the coder to be able to replicate a crash as serious (and wasted) effort!) Use Media! As the old adage goes, "a picture is definitely worth a thousand words". Show the readers how to replicate an issue, don't tell.

Grab any one of the screen capture software tools around. You can try:

SnagIt from Camtasia (Love it! It’s so simple to use, and provides output which has people questioning me how I did it so well?)
Captivate from Adobe (Heard good things about this, but never tried)

Here are some ideas how these tools can help your organization in terms of productivity and you as a tester:

Use any one of these software tools to capture the video of steps to replicate the defect.
You can enhance the videos by adding annotations. Text bubbles for remarks, highlighted circles, arrows etc. are some of your choices.
Tools like “Click! Recorder” from GlueSoft can even prepare the list of steps for you alongside recording the video. How cool is that?
If your steps are not so complicated and it’s a straightforward defect, you can use these recorders to capture just the still image / text from error message or complete object information. Often this gives a very good hint to the coders on what could have gone wrong.

Easy? And so very effective!

There you go. Try these tips to experience the change for yourself. Don’t forget to drop a line and let us know your thoughts.

Thank you for reading!

Enjoyed reading this article? You’ll like receiving more tips and testing related information in your e-mail. Please subscribe by clicking here. You’ll also be helping me in reaching my goal of having 1000 subscribers by Dec 2011. You have my word for no spam and complete privacy of your information.

7 habits software coders wish the testers had

2010-06-01T23:41:00.000-07:00

In my tenure as a tester and a test manager, I've worked with both types of settings where:

my company provided testing as a service to some development firm, so testing team was the service provider and the coding team was the client.
my company developed the product itself, so the testers were a part of SDLC team

Interestingly in both cases the coding teams had similar wish list for testers. Here are some common comments and how we can do something about them:

Good functional coverage: This is the first, most basic and a must-have requirement. Everyone – coders, managers, clients, all stakeholders – expect you to understand requirements inside out.

Make sure you question each line of requirements documentation, find positive as well as negative scenarios to test.
Test from each stakeholder’s perspective. Cover most user scenarios.

Test environment to resemble end user’s setup as closely as possible:
1. I’ve seen more than one case when the testing was done well but the environment on which the system was used at the user’s end was not well replicated. Result? Disappointments and wasted efforts! So beware.
Early reporting of defects:
1. Coders wish the testers would tell them about a lurking defect as quickly as they have found it (so that they can fix it soon). That does not mean testers should should “I’ve got one” each time something unexpected happens. No. You should definitely take your time to dig deeper, isolate your findings, find possible causes of the problem (and probable solutions, even? Why not?!). Just don’t wait till the last moment to fling a defect over the wall when the product is about to roll out.
Adequate information in defect reports:
1. A good defect report consists of all the info required to isolate and fix a defect, in a succinct, crisp way.
2. Here your skills of observation are going to be utilized the maximum. Mention the exact scenario(s) when something failed. Mention your test environment. Did you have any network services or antivirus program running when the system stalled? When you executed a file more than 2 GB did you get a crash? Paint the exact picture.
Surrounding testing while closing defects:
1. Verifying and closing defects is another crucial activity, that is often overlooked even by experienced testers and that results in dollars lost. One trick I tried while closing a defect was to try and replicate it again. If I fail after reasonable number of attempts, the defect is verified closed.
Performance numbers for products / websites:
1. Coders more often than not rely on you to understand the health of the system they are building. Set aside time and make sure you perform load, stress testing. Do your numbers and report how good, bad or ugly your system is. After all you might have given the best UI features, the coolest animation and stuff on your web page, but if it takes forever to load your users will be turned off. So, get serious about reporting these numbers.
2. Make sure you also make your managers aware of the importance of these readings, so they can plan time for performance testing, finding related defects and getting issues fixed, system optimized.
(Last but not the least) Respect for other person’s work:

One of my greatest lessons from serving s/w industry is that “it’s the man that maketh the machine”. And men (and of course women, too) have human flaws. Egos. If you spent your 8 waking hours of a day away from your family (and TV) to write a web page, no matter how sloppy, you’ll still be expecting others to look up to it with awe. Ok. I agree there’s a bit of exaggeration there. But get the point clear. Don’t make defect counts proportional to the amount of respect you show for others. Be diplomatic when it comes to announcing faults in someone else’s work. “Do unto others what you would have others to do unto you.”

Influence of stakeholders on testing

2010-05-19T23:04:00.000-07:00

Wikipedia defines Software testing as "an investigation conducted to provide stakeholders with information about the quality of the product or service under test."

A stakeholder is someone who has a 'stake' or 'interest' in the product being developed. The stakeholders of a project influence functionality being tested, non functional aspects of the product, resources (tools & technology) used for testing and even time and team size. Somethings like the time and team size aside, understanding expectations of stakeholders from any product can really help testers add value to the quality of product.

For adequate coverage the testing team should ‘think’ like a stakeholder while writing and executing test cases. I’ve seen issues discovered by the end users after the product went live, which should have been caught by the team who tested requirements. One of the primary reasons for this is not ‘thinking’ like a stakeholder.

Let's take an example to understand this. Consider a scenario where ABC company is launching a website to manage their sales orders and delivery. This website will allow customers of ABC to place orders on the website, ABC users to track deliveries of goods and payments received. The management reports section will allow senior managers to view revenue specific reports. SuperDevs company has been awarded this development project and they are further outsourcing the testing to BestTest company.

Some groups to look for while identifying stakeholders in this case would be:
End users - People who'll actually use this product. The customers of ABC Inc., staff and senior management. Most often these are the people who’ll give project requirements. People from this group influence core functionality of the product. More often than this this the only stakeholder group whose needs are validated by the testing team in their testing.
Customers would need to

Search for products and place orders. Might involve logic for discount calculation, shipping charges calculation etc.
Make payments over the internet securely
Edit details like shipping address etc., sometimes after placing the order
Track production status and delivery schedule (expected date of delivery)
Cancel an order. This might involve cancellation fees to be levied on the customer

Staff members from various departments would have different needs like

Production dept staff would need to know open orders requirement to plan production schedule
Marketing people would need to know which items are in demand and which are not, so they can create discount schemes etc to move the slow items
Customer support people would need to know the status of open orders and product details of orders that have been delivered so they can answer support queries, handle returns etc.

Executive staff would need special reports to give them revenue details as well as forecasts about orders so they can take intelligent decisions regarding budget and strategy.

Project Sponsors – Project sponsors have a value-for-money attitude towards a project. They influence timelines, team strength, budget and quite often tools & technology used by the dev and testing teams.

They do have an interest in the competitiveness of the product in their own niche market, though. This area is often overlooked while testing. It is a good idea to have someone research the available market tools or ideas so that the product can have solid requirements and better features. A good way to log those ‘feature requests’ during requirements testing phase is to do this home work and come up with ideas. Wear an entrepreneurs hat while you do the research, so you can get most of your feature requests approved! :)

ABC company’s IT staff – These people would typically co-ordinate between ABC company, SuperDevs team and BestTest team. They’ll have a stake in making sure the project runs smoothly and is delivered on time. They might not influence the functional requirements as much as they would lay down the non-functional ones.

Go talk to some of the IT team members before you write non-functional test cases like expected response time, number of concurrent users etc. They’ll also be your best guides in setting up test labs. Another overlooked area of testing. Several projects run into major / critical issues because the end users’ environment is different from the ones used by testing folks to test.

The development team (SuperDevs in our scenario) – Yes! They have an interest in the quality of product delivered, therefore they are also project stakeholders. Though they do not influence the functional aspects of a product, they’ll definitely make a difference to it.

Be it requirements testing, design & code reviews, buddy (pair) testing - there are several ways you can work alongside developers to improve quality of your product.

The testing team (from our example, the BestTest folks) – Are they one of the stakeholders? What do you think?

Are there any other stakeholders you can think of?

I’ll leave these questions open for you to mull over. Do share your thoughts in the form of a comment so we can discuss and debate.

This article enlists some of the areas which a testing team can better cover. Testers and test managers, next time you are assigned a project you can create a similar list of stakeholders and make sure you cover everyone’s interests. You’ll surely see some reduction in issues that escape to the production environment.

Happy testing :)

Use what you have

2010-05-10T01:38:00.000-07:00

In one of the development projects where I was the test manager, I faced with a challenge. My team size varied through out the development cycle. Here’s how:

Between three test engineers and myself we analyzed requirements, tracked issues thereof, designed test scenarios and test cases. Three additional test engineers joined us during execution and release phase. Since the dev and test were independent business units, the test engineers were typically loaned resources who were assigned on availability basis.

We were faced with the challenges of frequently changing test engineers besides the package deals that come with testing a data intensive, high impact financial auditing software.

Here is how we two tools available in our development environment to help us improve productivity:

1. SQL Server:

Our software accepted data input in the form of XML files. We created bulk test data (based on some sample data provided by our clients) in SQL server tables. Then used simple “For XML” clause to generate basic XML file. Minor find-and-replace tweaks generated our actual input file. We then attached these test data files with each test case in our test case management tool for use during the execution cycle. Now the newbie engineers joining us had simply to use this data to verify the expected results.

2. Share point server

We were given a project website on SPS for our internal collaboration. We created lists on this site:
- Queries list: Queries regarding requirements or a behavior were logged and tracked here. Each query had a user to whom it was currently assigned to. Test manager monitored aged queries and made sure they were resolved satisfactorily. Often these queries ended up simplifying requirements that were conflicting or ambiguous, or helped improvise technical design. Sometimes the query turned out to be a genuine issue, so it was transferred to our defect tracking tool. We made sure to add the test case / defect id from other system into this list so we could establish traceability.

- Test scenarios list: Before we created detailed test cases, we nailed down the top level test scenarios. This helped us gauge the extent of testing needed and get a feel of changes being done to the system in each iteration. This list also served as the introductory chapter for new members.
Each item in this list had 3 owners assigned to it – the business analyst owner, the dev owner and the QA owner. The QA owner was typically the one who created this scenario. The dev owner was the one who would use this scenario during his / her preliminary testing before marking a piece of code as ready-to-test. The BA owner would come into the picture is anyone from the team asked clarifications on an item or if the requirements document needed updates due to a scenario.

The advantage SPS provided over .xls sheets was that we did not have to juggle shared workbooks and private copies. All info was kept in one location for every one to see. We also had our team members set up alerts on each of these lists so as to get notified of any changes.

These are just two of the ways we used tools in an unusual way to meet our specific needs. Do you have any more of such examples you’ll like to share with our readers? Feel free to add your comments.