Software Testing

2012-02-11T21:51:00.001-08:00

Jokes on Software Testing

http://softwaretestingfundamentals.com/software-testing-jokes/

http://extremesoftwaretesting.com/Humor/anecdotes.html

http://extremesoftwaretesting.com/Humor/SoftwareTestingJokes2.html

Some Good Quotes

2010-11-08T09:02:00.000-08:00

If debugging is the process of removing bugs, then programming must be the process of putting them in. ~Author Unknown

"Quality is free, but only to those who are willing to pay heavily for it." (Lister, DeMarco: "Peopleware")

"The only person who likes change is a wet baby." (Carr, Hard, Trahant: "Change Process")

"A day lost at the beginning of a project hurts just as much as a day lost at the end. ... There are infinitely many ways to loose a day ..but not even one way to get one back." (DeMarco, "The Deadline")

“Before software can be reusable it first has to be usable.” – Ralph Johnson.

“If you don’t care about quality, you can meet any other requirement” – Gerald M. Weinberg

“Irreproducible bugs become highly reproducible right after delivery to the customer”, Michael Stahl’s derivative of Murphy’s Law

Article info : Wideband Delphi Estimation Technique

2010-08-27T07:56:00.000-07:00

Article info : Wideband Delphi Estimation Technique

Daily Status of Test Execution

2010-03-14T05:03:00.000-07:00

There are a couple of important items that we should keep in mind when we communicate the status of the daily test execution to the higher or middle management. The important items would be the following
a) Status of test case execution:
This can include the total number of test cases planned, executed, passed, failed, blocked and not executed.
b) Defect status
This can include total number of defects reported vs. the severity. Also the information on Open Sev 1 and Sev 2 defects would be a subject of interest as most of the projects the Sev 1 and 2 defects get fixed where as the Sev 3 and 4 would be open.
c) Also mention about the risk and issues that the team is facing during test execution and the plans to mitigate the risks.
d) The status report also should state whether the current rate of test execution is enough to achieve the planned end dates. If no it will be good to mention the plans that are in place to increase the execution rate.

Testing in Sessions..

2009-12-31T23:05:00.001-08:00

When all of us were studying at school or college (and even now) we have periods of 45 minutes or 60 minutes in length. Have you ever thought why this is limited to 60 minutes or 45 minutes? The answer is simple students will loose concentration after 45 minutes. Testers are like students and they will loose concentration after a certain amount of time. When you are testing an application make sure that you take a break after 45 minutes and then come back with a fresh mind to resume testing. When you are writing test cases ensure that the test cases does not contain too many steps. If your test case is lengthy it may take hours or days to execute. I have seen testers writing test cases with 400 or 500 steps which may take more than a day to execute.
The tester may start the execution of the test cases may be with Build 1 and by the time he reaches 50% of the execution you may have Build 2 from the development. You may be forced to start the test case execution from Step 1 again if the feature or requirements under test underwent a change in Build 2. It will be a good practice to limit the number of steps in your test case to 20 or 25 steps so that you can complete the test case execution in less than an hour's time. It may be not be possible to limit the number of steps in test cases for example when you are testing a complete end to end flow of an application. So try to execute test cases or perform adhoc testing in sessions of shorter duration and find as many defects as possible.

Changes that I saw in Software Testing

2009-11-28T23:59:00.001-08:00

Software testing or Quality Control was not a separate department in most of the companies till the year 2000. I never knew that I can turn into a software test engineer or whether there was a career path in the testing stream while I was studying. In the early 2000, Software professional were not interested to move from development to testing. Many of the fresh graduates who joined companies during the 2000-01 time frame very forced to do testing jobs .After the 2001 recession many of the IT Companies started facing competition and quality became one of the key service differentiators. The demand for testing jobs increased during this time and companies started setting up independent testing teams. Anyone who knew how to write test cases or execute test cases got a job. Developers who were not good at coding or logic decided to give a try and many of them failed miserably in testing too. Things went on like this till 2003 after which there was a boom for automation and performance testing. Companies started investing in tools and had planned to automate everything they can. Engineers who have attended basic training on tools and with record/play back experience became automation engineers. In the year 2006 there was a focus on domain knowledge for testers, like a healthcare IT company wanted to recruit testers only with healthcare background. Since there was a huge demand for good testers this move was not very successful. Now I think I have talked enough about the past. What are companies thinking about....most of them wants testers with specialized skills like database testing, security testing, etc? But do we have right people in our market. ?

Business Process Testing

2009-10-03T05:27:00.000-07:00

Business Process Testing deals with the testing of end to end or business
workflow of an application. The concept of BPT was in existence long before HP introduced in their Test management tool Quality Centre. This technique can be used to develop manual and automated components. There are accelerators provided by HP for applications like SAP, Siebel etc. This technique will help companies to reduce the cost and time required for test case construction and automation. I have tried this myself and was convinced with the concept and its implementation.

Tester's Desk

2009-04-11T19:37:00.000-07:00

There has been a shortage of good test case design tools. The site www.testersdesk.com has solved this. The site has tools which can be used to identify test data and test cases. Have a look at this site where more of the options are available for free. The site or the company is run by an individual with the name Ashwin who was working as an AVP in AppLabs.

Keyword Test Case Design

2008-12-09T23:11:00.000-08:00

An Article on Keyword Driven Test Case Design:-
http://www.stickyminds.com/s.asp?F=S14393_ART_2

Virtual Test Lab

2008-10-31T08:26:00.000-07:00

Introduction

I would like to explain how testing on clean PC will help you to overcome the issues that may arise due to presence of additional software or improper settings on the Test PC.

Many of the test engineers perform testing on PC that have lot of additional programs installed like messengers, browser toolbars and products that your potential clients or customers may not be using. Also, a test engineer involved in functional testing may install different versions of the AUT during the testing life cycle. Most of the install or uninstall programs prompt the user whether to uninstall a shared file or dll during the uninstall process. However, the fact is that most of the users ignore this message due to lack of knowledge or awareness. Also, the install program written for your software may fail to remove some of the registry entries.

I have across situations where the client was unable to use the software due to the absence of some files in his machine during the starting of my career. After a little bit of research we came to know that the PC was used for testing had third party software, which had the missing file. But for our bad luck, the customer did not have the same software installed.

Test Environment

Many companies have adopted a phase in their testing lifecycle called installation testing or ignition testing where the software is tested in a clean environment. However, this is normally carried out after regression or system testing. A clean environment refers to a fresh PC loaded with the required OS and with the software(s) your customers will be using.

It will be a good idea to figure out the actual environment used by the client during the requirements phase. This applies to both client PC’s, Application and database servers. In addition, some of the settings of your PC like regional settings and browser settings also play a major role in identifying issues or defects. It is a good practice to maintain a checklist for the PC that you will be using for your testing and configure the same. The usage of test automation tools has increased in the recent past. Many tools available in the market have issues identifying GUI objects inside the AUT if one change the settings like display properties in the OS. Therefore, it is important to maintain the same settings across all the PC’s used for testing.

Also, consider an AUT that has to be tested on three different OS (Win 98,XP and 2000) with IE 6,IE 7 and Netscape 7. If the customer that you are working for requires a weekly build then the time that one will be allocated for testing will be limited. In this case, one has to have nine fresh PC’s with OS and browsers to deliver the product after testing. Normally the process of setting up a PC with OS and configuring the same with the checklist takes a minimum of 3 hours time. The time required to setup the PC’s each iteration of testing will be close to 27 person hours. One should not forget the overhead of maintaining the nine PC’s. So, this approach is not feasible for performing testing on a clean PC during the time constraint.

Ghosting

So how do we overcome this problem? Most of the branded PC’s in the market is sold with the OS installation in today’s market. Have you thought how PC manufacturing companies install OS in the PC’s that they are selling? Any Windows OS installation takes a minimum of 1.5 hours. So a company that manufactures 500 PC’s a day has to spent 750 person hours to install an OS. Most of the companies will not be ready to spent time and valuable resources for this.

I got an opportunity to visit one of the hardware-manufacturing units in India a few years back. The company had a mechanism to install OS on a new PC within 7-8 minutes. They were using ghosting software called Drive Image Pro for this. All the PC ‘s manufactured in a department required almost the same display or sound card drivers. A PC will be loaded with the OS and the required drivers and a ghost image of the hard disk is taken. The same images are restored to all the PC’s later within minutes. The same concept can be adopted for setting up PC’s for testing.

There are many tools available in the market for this purpose like Norton Ghost, G4U, and G4L etc. Install the required OS and software’s and capture an image of the same. On has to make sure that your image is void of your AUT. You may have to maintain multiple copies of images based on the browser versions or OS that one is going to test. Restore the image, install your AUT and use them whenever you do your testing.

But this approach has its own limitations too. I have listed down some of them below.

One has to maintain different copies of images based on the OS, browser etc.

All the images have to be updated once a patch or service pack is released for your OS.

One should have sufficient privileges to rename PC’s if your try to use the same image for all test pc’s in a networked environment.

Most of the ghosting tools operate from a bootable CD or startup floppy and one may require physical access to the PC.

System Restore

Many of Microsoft OS likes ME and XP is having a feature called System restore. The System Restore feature enables administrators to restore their computers to a previous state without losing personal data files (e.g. Word documents, graphic files, e-mail). System Restore actively monitors system file changes and some application file changes to record or store previous versions before the changes occurred. Nevertheless, this method also has its own limitations.

System Restore monitors only a core set of specified system and application file types (e.g. .exe, .dll etc).

System Restore does not monitor changes to or recover users' personal data files such as documents, graphics, e-mail, and so on.

System data contained in System Restore's restore points are available to restore to for only a limited period.

System Restore does not completely uninstall any program if restoring to a point prior to the program installation.

You can find more information regarding system restore in the Microsoft site.

Virtual Machine Technology

VM technology helps you to run multiple OS environments on a single server or physical hardware system. Current technology like the MS Virtual Server 2005 R2 has added a layer on top of the OS in the physical environment. In addition, one can allocate virtualized hardware resources to Virtual Machines.

For example consider a high end desktop PC with 4 GB RAM and 100 GB HDD. One can install the host operating system with 1 GB RAM and 40 GB HDD space. After this, one can go ahead and create six VM ‘s with each VM having 512 MB RAM and 10 GB HDD space. . The technology also can package an instance of an OS and the configure applications into a VHD file. These files can be rapidly deployed on to your test environment very easily. One has to maximize test hardware to reduce costs and to cover more additional test scenarios. This approach is easy to use and has gained popularity fast. You can find more information on this in the site Microsoft Site. Also, keep in mind that the one should have separate licenses for the VM’s installed on Host OS. Some of the limitations are given below: -

If the Virtual Server is down due to hardware issues all the VM’s installed on the Host will be unavailable.

In addition, one has to invest money to purchase high end desktop PC’s.

This approach can be used only for non-production purposes.

Conclusion:

In this article, I have explained the advantages of using a clean PC for testing. I then went on and explained the different approaches that can be used to setup a Clean PC. One should evaluate the various options explained before implementing the same in their organization.

Test Efficiency

2008-10-31T08:25:00.000-07:00

What is Test Execution Efficiency?
It is generally very difficult to measure the efficiency of the testing process or the testing team for a project. Test Efficiency helps to calculate the efficiency of testing i.e. how many defects were leaked to the customer as compared to number of defects reported by the testing team. Generally almost 10-15 % of defects will be leaked and is considered acceptable. In the recent years, Companies have stared spending huge amount of money for developing quality. Due to this defect leakage percentage has come down to less than 10%.

How to Measure?
The Excel Sheet Attached helps us to calculate the efficiency of a testing process based on the number of defects reported by Customer and to the number of defects identified by the Testing Team.

Steps

1) Provide Ranking to each severity.
In the excel sheet the severity rankings have been assigned as

a) Critical--4
b) Serious – 3
c) Moderate –2
d) Minor –1

2) Collect the list of defects reported by the Testing and Customer based on Severity.
For example:
The customer has Reported 1 Critical, 1 Serious, 2 Moderate and 5 Minor Defects. The Testing Team should have identified these defects. The defects that were not *******ble in a test environment but only in a live production environment should not be considered.

The Testing Team has Reported 10 Critical, 5 Serious, 10 Moderate and 10 Minor Defects.

The Test Efficiency is calculated as follows: (T/T+C) * 100
T=4*10+5*3+10*2+10*1=85
C= 1*4+1*3+2*2+5*1=16
So Test Efficiency is (85/85+16) * 100=84.16%

Suppose if the Customer had not identified any defects in above example then the Test Efficiency will be 100%.

Consider a small project in which the testing team and customer did not find any defects (Assume you had a good programmer who did unit testing properly) then also the Test Efficiency will be 100%.If the Testing Team Failed to find any Defects and the Customer were finding them then the efficiency of the testing will be 0%. The formula used required fine-tuning in this case

2008-10-31T07:35:00.000-07:00

What is a Smoke Test?
Smoke testing is non-exhaustive software testing, to prove that the most crucial functions of a
program work, but not bothering the finer details. A smoke test is similar to the basic type of
hardware testing, in which the device passed the test if it didn’t catch fire the first time when it
was turned on. The term is also used metaphorically, especially in computer programming, where
it is also called a "sanity test
Most of the software companies follow the “daily build and smoke test” process. Each day every
file is compiled, linked, and combined into an executable program and the program has to pass
through a “smoke test,” a relatively simple check to see whether the product “smokes” when it
runs. After performing a smoke test the tester can come to a conclusion that the software can be
accepted or rejected for a major round of testing.
Benefits of Smoke Testing
Reduces Integration Risks
I have encountered many instances in the past where the entire software gets
screwed up when the source code is integrated. The composite code does not
work well. The integration issues will lead to a delay in the delivery of the
product/project. The smoke tests may help to reduce the integrations issues to a
good extent.
Defect Diagnosis.
The product which worked on second day and failed on the third day .One can
easily find out what has gone wrong from the previously working build.
Improves Morale.
The morale gets boosted .Everyone will be happy to see the improvement in the
product working as one is seeing that in a day to day basis
Progress Monitoring.
When you build the every day, the features that are present and absent are obvious.
Once can get a sense of how close it is to completion.
How to Smoke Test?
1. Code Merge : The programmer compares his copy of the source files with the master
source files, checking for inconsistencies between recent changes made by other
developers. The programmer then merges the code changes with the master source files.
Source code control tools can be used for this purpose.
2. Private Build : The programmer builds and tests the software to ensure that newly
implemented feature works as expected.
3. Execute the smoke test: The developer runs the current smoke test against the private
build to be sure the new code won’t break the build.
4. Code Checkin:. The developer checks his or her private copies of the source code into
the master source files.
Article on Smoke Testing
4
5. Generate the daily build: The build team generates a complete build of the software from
the master sources.
6. Run the smoke test: The build team runs the smoke test to evaluate whether the build is
stable enough to be tested.
7. Fix problems immediately: If the build team encounters errors that prevent the build from
being tested (that break the build), the developer who checked in the code is notified, and
that developer fixes the problem immediately.