Testing Methodologies

Software testing fundamentals

2012-01-07T18:43:00.000+05:30

To start anything we need to have the fundamental idea behind it. Though we are testing, we may not aware of the fundamental idea behind testing. Let's have a look at the fundamental things of Software testing.

Basics

The basic things we have to be aware testing are,

Software quality
Dimensions of quality
Software quality assurance
Software quality control
Verification and Validation
Software development life cycle
Software testing life cycle

Models

Let it be testing or development each requires a model to follow. We can have following models for testing,

Water fall model
V model testing
Iterative model
Agile testing(Check out the previous post of Agile testing)

Levels of Testing

The testing can be done in the following levels,

Unit testing
Integration testing
System Testing
User acceptance testing

Methods of Testing

Black box testing
white box testing
Gray box testing
Manual testing
Automation testing

Types of Testing

Smoke testing
Installation testing
Functional testing
Look & Feel testing
Usability testing
Compatibility testing
Performance testing
Security testing
Confirmation testing
Regression testing

Test Artifacts

Test plan
Test cases
Test script
Test data
Test suit
Test harness
Test report

Defects

Defects
Defect severity
Defect probability
Defect life cycle
Defect report

Metrics

Defect age
Defect Density
Defect detection efficiency
Defect removal efficiency
Test coverage
Test effort variance
Test schedule variance
Cost of quality

Testing tools

Test Management tools
Defect tracking tools
Fundamental testing tools
Performance testing tools
Unit testing tools
Automation testing tools

We will see the fundamentals in detail in next blog.......Happy testing

2011-12-25T23:31:00.000+05:30

Types of Testing - Black box testing

Let's see about BLACK BOX testing.

Black box testing

Black box testing is to verify whether the software is functioning as expected or it fails to produce expected output for a defined input irrespective of the way how it is made to produce the result(say the code, programming language used, loops used) nothing matters but we need the expected output from the software. The tester does not need to have the basic knowledge about the programming language used to develop the software, code pattern, methods used but he expects the output for a given set of inputs. The black box testing is also called as closed box testing.

Black box testing does not consider the internal design but requires the functional specifications. The test cases, inputs to the system, expected output are defined based on the functional specifications.

Black box testing treats the system as a “black-box”, so it doesn’t explicitly use Knowledge of the internal structure or code. Or in other words the Test engineer need not know the internal working of the “Black box” or application.

For example, when we are using the calculator in windows, what we would expect while entering 7+2, we expect the output as 9, but we don't care about how it produce the result, what is the code pattern, how long it takes to produce the result and all. This is Black box testing

We can otherwise call the black box testing as closed box testing

Where we can use the black box testing - The black box testing can be used in System integration testing(IST), unit testing and User acceptance testing(UAT).

How long it is reliable to use black box testing - It is quite reliable in terms of functionality of a software, but when it comes to performance(Turn around time - The time between input given and output produced, quality of code) we can not rely on it.

Can it be used in all the applications - Of course the black box testing can be used in kinds of applications.

Why is it called as black box - Because we do not see/need what is happening inside an application but we just want expected output for an input.

Tools used for Black Box testing:

Black box testing tools are mainly record and playback tools. These tools are used for regression testing that to check whether new build has created any bug in previous working application functionality. These record and playback tools records test cases in the form of some scripts like VB script, Java script, Perl.

Methods of Black box Testing:

Graph Based Testing Methods:
Each and every application is build up of some objects. All such objects are identified and graph is prepared. From this object graph each object relationship is identified and test cases written accordingly to discover the errors.

Error Guessing:
This is purely based on previous experience and judgment of tester. Error Guessing is the art of guessing where errors can be hidden. For this technique there are no specific tools, writing the test cases that cover all the application paths.

Boundary Value Analysis:
Many systems have tendency to fail on boundary. So testing boundry values of application is important. Boundary Value Analysis (BVA) is a test Functional Testing technique where the extreme boundary values are chosen. Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values.

Extends equivalence partitioning

Test both sides of each boundary
Look at output boundaries for test cases too
Test min, min-1, max, max+1, typical values

BVA techniques:

1. Number of variables
For n variables: BVA yields 4n + 1 test cases.
2. Kinds of ranges
Generalizing ranges depends on the nature or type of variables
Advantages of Boundary Value Analysis
1. Robustness Testing – Boundary Value Analysis plus values that go beyond the limits
2. Min – 1, Min, Min +1, Nom, Max -1, Max, Max +1
3. Forces attention to exception handling

Limitations of Boundary Value Analysis
Boundary value testing is efficient only for variables of fixed values i.e boundary.

Equivalence Partitioning:
Equivalence partitioning is a black box testing method that divides the input domain of a program into classes of data from which test cases can be derived.

How is this partitioning performed while testing:

1. If an input condition specifies a range, one valid and one two invalid classes are defined.
2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined.
3. If an input condition specifies a member of a set, one valid and one invalid equivalence class is defined.
4. If an input condition is Boolean, one valid and one invalid class is defined.

Advantages of Black box testing

Tester can be non technical in terms of technology used to develop the application(c, c++)
It is efficient when used in large systems
Since the developer and tester are independent, testing is balanced and unprejudiced.
Testing aids to identify the vagueness and contradictions in functional specifications.
The test is done from the user point of view, because the end user should accept the system.
Test cases can be designed as soon as the functional specifications are complete.
The testing can measure the functionality of the software completely.

Disadvantages of the Black box testing

It is difficult to identify tricky inputs if the test cases are not developed based on specifications.
Designing test cases are complex unless we are very clear about he functional knowledge of the
There are chances of having unidentified paths during the test process as we are not aware of the code and its accessible input methods.
There is high possibility of repeating tests if it is already tested by the developer.

Happy testing......!

Agile Software Development

2011-12-19T16:35:00.003+05:30

Agile software development is a group of software development methodologies based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It is a conceptual framework that promotes foreseen interactions throughout the development cycle. The Agile Manifesto introduced the term in 2001.

Agile Manifesto

In February 2001, 17 software developers met, to discuss lightweight development methods. They published the Manifesto for Agile Software Development to define the approach now known as agile software development. Some of the manifesto's authors formed the Agile Alliance, a nonprofit organization that promotes software development according to the manifesto's principles.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.

The meanings of the manifesto items on the left within the agile software development context are described below.

Individuals and Interactions – in agile development, self-organization and motivation are important, as are interactions like co-location and pair programming.

Working software – working software will be more useful and welcome than just presenting documents to clients in meetings.
Customer collaboration – requirements cannot be fully collected at the beginning of the software development cycle, therefore continuous customer or stakeholder involvement is very important.
Responding to change – agile development is focused on quick responses to change and continuous development.

Twelve principles underlie the Agile Manifesto, including:

Customer satisfaction by rapid delivery of useful software
Welcome changing requirements, even late in development
Working software is delivered frequently (weeks rather than months)
Working software is the principal measure of progress
Sustainable development, able to maintain a constant pace
Close, daily co-operation between business people and developers
Face-to-face conversation is the best form of communication (co-location)
Projects are built around motivated individuals, who should be trusted
Continuous attention to technical excellence and good design
Simplicity
Self-organizing teams
Regular adaptation to changing circumstances

In 2005, a group headed by Alistair Cockburn and Jim Highsmith wrote an addendum of project management principles, the Declaration of Interdependence, to guide software project management according to agile development methods.

How the Agile methodolgy Works

The Agile methodology is close to iterative model. Each task is broken into small task and must be completed within short period of time. The period denotes not more than 3-4 weeks. When the time limit goes beyond 3-4 weeks, the task has to be split into smaller tasks.
As said, the smaller tasks cannot provide a fully developed software or the software cannot meet its requirement, so iterations will purse till the software is performing its intended task.
Each task is having its requirement gathering, development, unit testing, System integeration testing which is used to develop stable software.
The Agile methodology comes by bringing collobration among team members, Customer collabration and by responding to change.
Once each part of the software is developed, we can integrate them as working software which performs its intended functionality.

Agile development model would go like this.

1. The software is split into smaller modules which could be done by 3-4 weeks
2. Team is assigned to each task where team is having 6-9 members
3. A representative is assigned to each team on behalf of the stake holders.
4. Development would be done in daily basis
5. Review on the development will be done by conductiong team meeting including differnt modules - All modules would gather together to discuss about their progress, clarifications and challenges
6. Development activity will be done and testing will begin on each modules.

Agile development method suitability

Suitability

Large-scale agile software development remains an active research area.

Agile development has been widely seen as being more suitable for certain types of environment, including small teams of experts.

Positive reception towards Agile methods has been observed in Embedded domain across Europe in recent years.

Some things that may negatively impact the success of an agile project are:

Large-scale development efforts (>20 developers), though scaling strategies and evidence of some large projects have been described.
Distributed development efforts (non-colocated teams). Strategies have been described in Bridging the Distance and Using an Agile Software Process with Offshore Development
Forcing an agile process on a development team
Mission-critical systems where failure is not an option at any cost (e.g. software for surgical procedures).

The early successes, challenges and limitations encountered in the adoption of agile methods in a large organization have been documented.

In terms of outsourcing agile development, Michael Hackett, Sr. Vice President of LogiGear Corporation has stated that "the offshore team ... should have expertise, experience, good communication skills, inter-cultural understanding, trust and understanding between members and groups and with each other.

Mobile Testing Automation

2011-12-12T22:21:00.010+05:30

Mobile testing Automation allows to automate the frequently used regression test cases. There are lots of difficulties that an Organization or a test faces while doing testing manually. Automation is done using Open source applications as well as Licensed applications. The following difficulties are faced usually while automating the mobile testing.

1. Technical difficulties
2. General difficulties

Technical Difficulties

1. Total number of test cases
2. Finding compatible application for automation
3. Using the same script for different Mobile operation systems is very critical
4. Evolution of mobile platforms/phone at rapid speed

General Difficulties

1. Hardware compatibility
2. Application complexity
3. Development schedule is very limited

How Automation testing overcomes these problems ?

Let me tell you, automation testing is not going to resolve all the above mentioned problems, still it solves critical issues. The very critical issues are

1. Large number of test cases
2. Diversity of mobile devices/platforms
3. Evolution of mobile devices/platforms

Let's consider as each of them as a critical issue and see how automation helps to overcome those issues.

Issues 1 : Large number of test cases : Automation as name implies we can automate all the regression test cases using different tools like QTP. We can automate all the possible test cases which does not require more interactions or support from human. While the applications that automates the test cases provide options to add plug-ins as mobile platform grows we can add plug-ins to support to its newer version(Really interesting right !).

Issues 2 : Diversity of mobile devices/platforms: As we are aware there are lots of mobile phones, mobile platforms, applications are emerging at a rapid pace. The automation provides solution to this problem also. There are automation tools available especially for mobile testing.

For example, the application "Seetest" and "M-eux test" is used to automate same scripts for various mobile platforms. The question arises here is , How is same script used for different platforms. Here comes the concept "Plug-ins".

What is plug-in ? Plug-ins are just a software component, it can be a single or a set of component that can be used with a particular application. For example , you are having Internet explorer, which would not usually come up with Flash player, but to play a video online or say an you tube video you need the flash player. To do this you would download flash player plug-ins for Internet explorer. I hope now you are clear what is Plug-in

As such, different types of plug-ins available for different mobile platforms. Its really amazing right !

Issue 3 : Evolution of mobile devices/platforms: It is almost related to what we have discussed in the issue 2. The solution for the issue 2 is the answer for this question also. But when a new functionality is introduced in an application or if a new application is developed newly for a platform, what would happen.

As we are aware that mostly only the regression test is likely automated, the functional testing is done manually. We will keep you posted with the new technology when we come across.

Ok ! we have an overall idea about what is mobile testing, issues faced and solutions for those issues. But what are all the applications are available in the market for Mobile Automation Testing. Let's see.

As we all aware that software applications can broadly be classified into 2 types.

1. Open source applications (Like Unix OS)
2. Closed source applications (Like windows OS)

Open Source applications

Though there are so many Open source applications are available, I have listed only few of them.
1. FoneMonkey5 (For standard mobile application preferably for Iphone)
2. Webdriver (Mostly for mobile webbased applications)

Closed Source applications

1. Iphoney
2. Adope device central CS5
3. QTP 9.5
4. Gomez

There are lots of closed source softwares are available in the market, I have listed only a few. One important think to notice is the applications are supporting from non-web based application to web based applications.

Happy Testing ! We will meet in the next blog with some awesome testing topics.....

Octofinder

RSS Aggregator