Software Testing and Quality Assurance Guidelines

2008 Survey results - Software quality issues due to variable level of skills between locations

2008-12-19T02:47:00.000-08:00

"...communication and collaboration was still the dominant challenge in distributed development with more than 85% respondents mentioning it, the surprise second, as per the survey, was software quality issues arising from too much variation in skill sets between sites" (see http://www.infoq.com/news/2008/12/distributed-development-quality for more information).
This was one of the conclusions from the survey (September 2008) described in the attached link, but the top identified issues are practically the same, even if some changes have occur in the hierarchy
- General communication and collaboration challenges
- Software quality issues due to variable level of skills between locations
- Political issues with the way organization is structured
- Quality concerns due to difference in processes/practices
- Project management issues caused by complexities of distributed development

Informal reviews - the inexpensive way to get benefits

2008-12-02T11:47:00.000-08:00

An informal review is always the best solution when you want to avoid too many corrections from the formal reviewers or to decrease the number of obvious mistakes you've done. And it is quite easy: you have to go to one of your team colleagues and kindly ask him to review your work. The informal reviews are also very useful for the junior software testers since their confidence and experience is low, so they need help from the experienced colleagues (especially senior testers or programmers) to quickly review their work and to identify the aspects which don't make sense, which need to be improved or better described, etc. .
The informal review is not a formal process, it may be documented if the reviewers consider it necessary (the material may be as informal as a computer listing or hand-written documentation), is the most inexpensive way to get benefits and the date and the time of the agenda for it will not be addressed in the project plan.
Depending of the nature of the review, the tester can ask the informal review also from a programmer and not only from someone within the testing team ( senior testers or technical leads are especially recommended for this, because they have the requisite technical or business expertise).
If you are a programmer you should be performing informal code reviews or having a colleague developer perform an informal code review to catch some of the smaller mistakes that are commonly made. A number of techniques are available for doing this, for example, use a parameter or constant in the code as a switch to display all parameters and/or print them to a log file. Become aware of what causes errors in program code and how errors can be detected and prevented. Also, learn to make the functionality understandable by using adequate comments.
One more important thing for the informal reviewer : critically reading the work of another software tester/programmer enables you to become more able to identify, diagnose, and solve some of your own issues.

Review process

2008-12-01T14:12:00.000-08:00

"Review" is one of the key words when talking about the software testing. The reviews should not be necessarily carried out by the testers, but by the people to whom the project manager will assign this task (especially by the quality assurance department, if this exists within the company) .
We can review a lot of aspects during the software development life cycle: development and design documentation, test plans, test cases, test results, defects, tests and development metrics, etc.
Even if we are doing informal, walkthrough, technical or inspections reviews (I'll try to detail each of these types in the next posts) it is important to distinguish between the factors which make a review to be successfull or not.
We need to make sure the right people are involved for the review objectives, the review's objective is clear for all participants, each issue found is expressed objectively without affecting the author (it has to be a constructive action and not a "witch hunt"), we use checklists in order to be more effective in identifying the issues, adequate time for reviews are allocated in the project shedules, etc.
In case that the responsibilities are not taken seriously or understood, the allocated resources are insufficient or the processes to support reviews are insufficient (or don't exist) then we can talk about unsuccessfull reviews.
Only by reviews we can make sure that the system specifications/requirements are correct and important faults are found before coding has started.

Software testing and Quality Assurance certifications

2008-11-08T22:40:00.000-08:00

The software testers and quality assurance specialists can achieve now one (or more) of the testing/quality assurance certifications available. These Software testing certification are basically of 2 types: an exam type based (you'll need to pass an exam: e.g. ISTQB (International Software Testing Qualifications Board)) and an education based (you'll need to pass each course: e.g. IIST (International Institute for Software Testing)) )
Read below the list with the most popular testing certifications and see if you can find one suitable for you (the information displayed for each certification are the ones from their official sources):

- ISTQB Certified Tester, Foundation Level (CTFL) offered by the International Software Testing Qualification Board
CTFL is an exam-based certification, based on a Certified Tester - Foundation Level Syllabus (see http://www.istqb.org/downloads/syllabi/SyllabusFoundation.pdf).
Objectives for the CTFL: The Foundation Level qualification is aimed at anyone involved in software testing. This includes people in roles such as testers, test analysts, test engineers, test consultants, test managers, user acceptance testers and software developers.
This Foundation Level qualification is also appropriate for anyone who wants a basic understanding of software testing, such as project managers, quality managers, software development managers, business analysts, IT directors and management consultants. Holders of the Foundation Certificate will be able to go on to a higher level software testing qualification (more details on http://www.istqb.org)

- ISTQB Certified Tester, Advanced Level (CTAL) offered by the International Software Testing Qualification Board
CTAL is an exam-based certification, based on a Certified Tester - Advanced Level Syllabus (use http://www.istqb.org/downloads/syllabi/CTAL_Syllabus_V_2007.pdf).
Objectives for the CTAL :
The Advanced Level qualification is aimed at people who have achieved an advanced point in their careers in software testing.
This includes people in roles such as testers, test analysts, test engineers, test consultants, test managers, user acceptance testers and software developers. This Advanced Level qualification is also appropriate for anyone who wants a deeper understanding of software testing, such as project managers, quality managers, software development managers, business analysts, IT directors and management consultants. To receive Advanced Level certification, candidates must hold the Foundation Certificate and satisfy the Exam Board which examines them that they have sufficient practical experience to be considered Advanced Level qualified.
The courses ISTQB Advanced Level cover the following main areas:
- Advanced Level Test Manager ( the Advanced Level Test Management professionals should be able to:
define the overall testing goals and strategy for the systems being tested, plan, schedule and track the tasks, describe and organize the necessary activities, select, acquire and assign the adequate resources to the tasks, select, organize and lead testing teams, organize the communication between the members of the testing teams, and between the testing teams and all the other stakeholders, justify the decisions and provide adequate reporting information where applicable
- Advanced Level Test Analyst ( the Advanced Level Test Analysts should be able to: structure the tasks defined in the test strategy in terms of business domain requirements, analyze the system in sufficient detail to meet the user quality expectations,evaluate the system requirements to determine domain validity, prepare and execute the adequate activities, and report on their progress, provide the necessary evidence to support evaluations, implement the necessary tools and techniques to achieve the defined goals
- Advanced Level Technical Test Analyst ( the Advanced Level Technical Test Analysts should be able to: structure the tasks defined in the test strategy in terms of technical requirements, analyze the internal structure of the system in sufficient detail to meet the expected quality
level, evaluate the system in terms of technical quality attributes such as performance, security, etc, prepare and execute the adequate activities, and report on their progress, conduct technical testing activities, provide the necessary evidence to support evaluations, implement the necessary tools and techniques to achieve the defined goals (more details on http://www.istqb.org)

- Certified Software Test Professional (CSTP) offered by the International Institute for Software Testing (IIST):
CSTP is an education-based certification, based on a Body of Knowledge that covers areas essential for every test professional to effectively perform their job in testing projects. Objectives of CSTP Certification: help individuals develop their software testing skills through formal education, establish a common skill set for software testing professionals according to a well-defined Body of Knowledge, create a pool of qualified software testing professionals, prepare candidates for a wider range of software testing assignments, complement company in-house and on-the-job training programs, provide professional recognition and career enhancement. There are two requirements that must be satisfied before the CSTP certification can be granted. These are: Formal Education Requirement and Job Experience Requirement (more details on http://www.testinginstitute.com/cstp.php )

- CATe offered by the International Institute for Software Testing (more details on http://www.testinginstitute.com

- Certified Test Manager (CTM) offered by the International Institute for Software Testing (IIST):
CTM is based on the Test Management Body of Knowledge (TMBOK) developed by IIST through its Advisory Board.
Objectives of CTM Certification:
The CTM Certification was developed to fill the gap in the management skills required by test managers and test leads to effectively manage the test process, the test project and the test organization. Specifically, CTM aims at achieving the following objectives: help individuals develop their test management skills through formal education, establish a common skill set for software test managers and test leads based on a well-defined Test Management Body of Knowledge (TMBOK), create a pool of qualified software test managers Prepare test professionals, especially those who achieved the Certified Test Professional (CSTP) designation for management and lead positions in software testing projects, provide professional recognition and career enhancement for those who manage test projects.
There are two requirements that must be satisfied before the CTM certification can be granted. These are: Formal Education Requirement and Job Experience Requirement (more details on http://www.testinginstitute.com/ctm.php)

- Certified Software Tester (CSTE) offered by the Quality Assurance Institute (QAI):
Objectives of CSTE Certification:
The Certified Software Tester (CSTE) certification is intended to establish standards for initial qualification and provide direction for the testing function through an aggressive educational program. Acquiring the designation of Certified Software Tester (CSTE) indicates a professional level of competence in the principles and practices of quality control in the IT profession. CSTEs become members of a recognized professional group and receive recognition of their competence by business and professional associates, potentially more rapid career advancement, and greater acceptance in the role as advisor to management.
To qualify for candidacy, each applicant must meet one of four prerequisites: a 4 year degree from an accredited college-level institution and 2 years experience in the information services field, a 3 year degree from an accredited college-level institution and 3 years experience in the information services field, a 2 year degree from an accredited college-level institution and 4 years experience in the information services field, six years experience in the information services field AND Are working, or have worked at any time within the prior 18 months, in the field within covered by the certification designation (more details on http://www.softwarecertifications.org/qai_cste.htm )

- ISEB offered by the Information Systems Examinations Board (see more on http://www.bcs.org/server.php?show=conWebDoc.2064 )
The following are the ISEB certification for software testing:
- Foundation Certificate in Software Testing
This certificate provides a good overview of the basics of software testing (more details on http://www.bcs.org/server.php?show=nav.6942).
- Intermediate Certificate in Software Testing
This certificate is the next level of competency in software testing after foundation level. A pass in this certificate provides evidence that the candidate has proven analytical ability, having gained knowledge and practical skills in the key topics required for practicing software testing. (more details on http://www.bcs.org/server.php?show=nav.001010002003021).
Practitioner Certificate in Test Management
This certificate is a high level qualification which examines knowledge and skills required for the management of Software Testing. It is the next level of competency in software testing after the ISEB intermediate level. A pass in this certificate provides evidence that the candidate has gained a high level of competency in topics specific to the management of software testing, having proven ability to analyse, synthesise and evaluate. (more details on http://www.bcs.org/server.php?show=nav.001010002003022)
Practitioner Certificate in Test Analysis
This certificate is a high level qualification which examines knowledge and skills required for the technical analysis of Software Testing. It is the next level of competency in software testing after the ISEB intermediate level. A pass in this certificate provides evidence that the candidate has gained a high level of competency in topics specific to the analysis of software testing, having proven ability to analysis, synthesis and evaluate.(more details on http://www.bcs.org/server.php?show=nav.001010002003023)

- CBTS offered by the Brazilian Certification of Software Testing (ALATS) (more details on http://www.alats.org.br)

A similar list is described below for the Quality assurance certifications:

- CSQE offered by the American Society for Quality (ASQ)
Objectives of CSQE Certification: The Certified Software Quality Engineer understands software quality development and implementation, software inspection, testing, verification and validation; and implements software development and maintenance processes and methods (more details on http://www.asq.org/certification/software-quality-engineer/index.html).

- CQIA offered by the American Society for Quality (ASQ)
Objectives of CQIA Certification: The Certified Quality Improvement Associate has a basic knowledge of quality tools and their uses and is involved in quality improvement projects, but doesn't necessarily come from a traditional quality area (more details on http://www.asq.org/certification/quality-improvement-associate/index.html)

- CSQA offered by the Quality Assurance Institute (QAI) (more details on http://www.softwarecertifications.org)

Even if some of these certifications may be a bonus when trying to get a job in the software testing/ quality assurance area, they can't measure an individual's productivity, their skill, or practical knowledge, and cannot guarantee their competence, or professionalism as a tester.

Software testing certifications - ISTQB reached 100000

2008-11-07T13:15:00.000-08:00

The ISTQB Certified Tester program will soon become the most popular software testing certification.
Few words about ISTQB (see more details on their official link : http://www.istqb.org) - it is a "program that provides certification for software testers internationally. There are currently two levels of certification: The Foundation Level and three separate Advanced Level certificates (ISTQB Certified Tester Foundation Level (CTFL) and ISTQB Certified Tester Advanced Level (CTAL)). For both levels, international working parties develop and maintain internationally uniform curricula and exams. One of the core principles of the program is a strict separation between the administration of exams and the training to those who would like to prepare for the exam in ISTQB compliant courses".
What is interesting is that they've already reached to a number of 100,000 Software Testing Professionals ISTQB Certified, which I find to be a really great achievement (more details on http://www.istqb.org/october_24_2008.htm - BRUSSELS, BELGIUM (24 October 2008) – The International Software Testing Qualifications Board (ISTQB) announced that, through their National Boards and Exam Boards, they have granted more than 100,000 certifications at the ISTQB Certified Tester Foundation Level (CTFL) and ISTQB Certified Tester Advanced Level (CTAL). "ISTQB software tester certification is by far the most widely recognized and fastest-growing software tester certification in the world," said Rex Black, CTAL, president of ISTQB. “Reaching the 100,000 certification milestone validates ISTQB’s leadership in helping software testers to demonstrate their knowledge, skill and professionalism.” ISTQB certification traces its origins to the UK, where industry demand led to its development. “Software testing is critical to the reliable functioning of the world’s software systems. We realized that certification can help to improve the capability of those that test systems, while moving the profession forward,” explained Geoff Thompson of the UK Testing Board (UKTB). The appeal of ISTQB software testing certification quickly led to its expansion. Horst Pohlmann of the German Testing Board (GTB) continued, "Companies tell us they choose ISTQB certification because it is practical, focusing on the software testing knowledge and skills their testers need every day." In recent years, ISTQB has become a global standard for software testing certification. Vinay Baid of the Indian Testing Board (ITB) explained, “ISTQB software testing certification is now essential to software testing in India. As is true throughout much of the world, many employers prefer to hire ISTQB certified software testers or have a program that encourages their software testers to obtain ISTQB certification.” The rapid growth of ISTQB software testing certification can also be attributed to its not-for-profit nature. “Companies have found ISTQB certification to be credible because we freely offer the ISTQB certification syllabi and software testing glossary, and let you choose your own education,” said Patricia McQuaid, PhD, CTFL, president of the American Software Testing Qualifications Board (ASTQB). This not-for-profit quality extends to ISTQB’s leadership. The organization is composed of representatives from national boards throughout the world, all of whom are volunteers. ISTQB decides on the standards for certification and the national boards develop and maintain the country-specific examinations. “While we’re proud of our status as the leading global software testing certification, we won’t rest on our laurels,” said Black. “Companies look to ISTQB to help them deliver quality software, and software testers look to ISTQB to help them build their profession. ISTQB’s volunteer leadership is already developing the vision that will take software testing to a new level.”).

The Cost of Software Errors

2008-11-02T10:11:00.000-08:00

Which are the causes for the software errors? Well, we can enumerate a variety of reasons as to why errors occur during the software development: developers do mistakes, the requirements for the software are not clearly defined, the allocated time and bugetary have deadlines which can be hardly met (because more pressure will lead to more mistakes) , the communication inside the project is poor and need lot of improvements, the requireents changes are not documented as they should, the objectives are changed during the project lifecycle, etc. The pressure with time, budgetary constraints and deadlines are increasing the probability of making mistakes for the development project's members.
And the software errors are important for us because of their derived cost of errors. An error can lead to from a small cost to a huge one. For a wrong report (e.g. incorrect version of application or wrong displayed dates: dd-mm-yy instead of dd/mm/yy) the cost is little, because the impact of this error is minimal.
For an error which occur in a safety critical software, the cost can be huge or in extreme cases it may cost lives. E.g.:
- death resulted from inadequate testing of the London Ambulance Service software
- several 1985-7 deaths of cancer patients were due to overdoses of radiation resulting from a race condition between concurrent tasks in the Therac-25 software
- a China Airlines Airbus Industrie A300 crashes on April 26, 1994 killing 264
- the British destroyer H.M.S. Sheffield was sunk in the Falkland Islands war. According to one report, the ship's radar warning systems were programmed to identify the Exocet missile as "friendly" because the British arsenal includes the Exocet's homing device and allowed the missile to reach its target, namely the Sheffield, etc.
The cost of efects depends of the stage where the errors were detected (it increases proportionately with the passing of each successive stage in the development process before they are detected) . If an issue is detected early, in the requirement phase, then its correction will cost little (few dollars). But to correct a problem in the post-implementation phase may cost thousands of dollars.
The cost of correcting a defect during progressively later phases in the system development life cycle rises quickly, so it is generally accepted that the cost of fixing a defect will increase by a factor of ten for each successive project phase (user requirements -> functional specifcations -> technical specifications -> code -> unit -> system -> user acceptance testing).

Recommended Testing Books - 5th list

2008-11-01T06:32:00.000-07:00

...adding more recommended books, in this 5th list (use the Books label to find easily and read the previous 4 lists) :

31. Jakob Nielsen and Hoa Loranger - Prioritizing Web Usability (use link)
Product Description:
In 2000, Jakob Nielsen, the world’s leading expert on Web usability, published a book that changed how people think about the Web—Designing Web Usability (New Riders). Many applauded. A few jeered. But everyone listened. The best-selling usability guru is back and has revisited his classic guide, joined forces with Web usability consultant Hoa Loranger, and created an updated companion book that covers the essential changes to the Web and usability today. Prioritizing Web Usability is the guide for anyone who wants to take their Web site(s) to next level and make usability a priority! Through the authors’ wisdom, experience, and hundreds of real-world user tests and contemporary Web site critiques, you’ll learn about site design, user experience and usability testing, navigation and search capabilities, old guidelines and prioritizing usability issues, page design and layout, content design, and more!

32. Steve Krug - Don't Make Me Think: A Common Sense Approach to Web Usability (2nd edition) (use link)
Product Description:
Five years and more than 100,000 copies after it was first published, it's hard to imagine anyone working in Web design who hasn't read Steve Krug's "instant classic" on Web usability, but people are still discovering it every day. In this second edition, Steve adds three new chapters in the same style as the original: wry and entertaining, yet loaded with insights and practical advice for novice and veteran alike. Don't be surprised if it completely changes the way you think about Web design.
Three New Chapters!
Usability as common courtesy -- Why people really leave Web sitesWeb Accessibility, CSS, and you -- Making sites usable and accessibleHelp! My boss wants me to ______. -- Surviving executive design whims
"I thought usability was the enemy of design until I read the first edition of this book. Don't Make Me Think! showed me how to put myself in the position of the person who uses my site. After reading it over a couple of hours and putting its ideas to work for the past five years, I can say it has done more to improve my abilities as a Web designer than any other book.
In this second edition, Steve Krug adds essential ammunition for those whose bosses, clients, stakeholders, and marketing managers insist on doing the wrong thing. If you design, write, program, own, or manage Web sites, you must read this book." -- Jeffrey Zeldman, author of Designing with Web Standards

33. B. M. Subraya - Integrated Approach to Web Performance Testing: A Practitioner's Guide (use link)
Product Description:
The performance testing of Web applications is not understood properly and no skill sets are available to address the challenges faced by practitioners. In addition, there is currently a lack of available research on the performance aspects of the application. "Integrated Approach to Web Performance Testing: A Practitioner's Guide" fills this void and provides an integrated approach and guidelines to performance testing of Web-based systems. Based upon a mix of theoretical and practical concepts, this book provides a detailed understanding of the various aspects of performance testing in relation to the different phases of the software development life cycle. Using a rich mixture of examples, checklists, and templates, this book illustrates the different facets of application performance. "Integrated Approach to Web Performance Testing: A Practitioner's Guide" applies a practical approach to making appropriate choices of tools, methodologies and project management for performance testing.

34. Stacy Joines, Ruth Willenborg, and Ken Hygh - Performance Analysis for Java(TM) Websites (use link)
Product Description:
Performance is a serious issue. A website must be able to handle the demands of users today, as well as adapt to increasing demands in the future. A website's responsiveness to customers has a direct effect on the success of an organization. Many developers struggle with the complex issues of website performance. This book is a timely guide for enterprise website developers and QA teams. The authors combine their experience analyzing and improving hundreds of websites and show how to conduct an effective performance analysis. They demonstrate how to solve common yet difficult problems, how to monitor tests, and analyze the data collected. Exploring common website types such as brokerage, e-commerce, and B2B, they discuss different performance needs, as well as how to design a performance test for each type of website. They use IBM WebSphere as the example Java application server, yet the content transfers easily to others such as BEA's popular WebLogic.

35. Anne Mette Jonassen Hass - Configuration Management Principles and Practice (Agile Software Development Series) (use link)
Product Description:
Configuration management (CM) is frequently misunderstood. This discipline is growing in popularity because it allows project participants to better identify potential problems, manage change, and efficiently track the progress of a software project. CM is not easy, but at the same time, it need not be difficult. This book gives the reader a practical understanding of the complexity and comprehensiveness of the discipline. Many current CM practitioners rely too heavily on commercial CM tools, and fail to understand the concept as a whole. With the deeper knowledge of CM principles taught in this book, readers will be better able to manage and deliver their next project. The book is included in the Agile Software Development Series because there is growing recognition that an effective configuration management strategy is the cornerstone of a truly agile project.

36. Tom Arnold - Professional Software Testing with Visual Studio 2005 Team System: Tools for Software Developers and Test Engineers (use link)
Product Description:
Visual Studio 2005 Team System (VSTS) features a robust, integrated suite of test tools that work seamlessly in the Visual Studio development environment Covering all phases of the development lifecycle, this book shows readers how to implement best practices for software testing using the appropriate components of VSTS Readers will learn how to use the tools effectively, directly from those who helped design and develop the software Shows how to catch common programming mistakes every time code is compiled, helping to shorten the time to market

37. Lasse Koskela - Test Driven: TDD and Acceptance TDD for Java Developers (use link)
Product Description:
In test driven development, you first write an executable test of what your application code must do. Only then do you write the code itself and, with the test spurring you on, you improve your design. In acceptance test driven development (ATDD), you use the same technique to implement product features, benefiting from iterative development, rapid feedback cycles, and better-defined requirements. TDD and its supporting tools and techniques lead to better software faster. Test Driven brings under one cover practical TDD techniques distilled from several years of community experience. With examples in Java and the Java EE environment, it explores both the techniques and the mindset of TDD and ATDD. It uses carefully chosen examples to illustrate TDD tools and design patterns, not in the abstract but concretely in the context of the technologies you face at work. It is accessible to TDD beginners, and it offers effective and less well known techniques to older TDD hands.
What's Inside
- Learn hands-on to test drive Java code
- How to avoid common TDD adoption pitfalls
- Acceptance test driven development and the Fit framework
- How to test Java EE components-Servlets, JSPs, and Spring Controllers
- Tough issues like multithreaded programs and data access code

38. Andreas Zeller - Why Programs Fail: A Guide to Systematic Debugging (use link)
Product Description:
Why Programs Fail is about bugs in computer programs, how to find them, how to reproduce them, and how to fix them in such a way that they do not occur anymore. This is the first comprehensive book on systematic debugging and covers a wide range of tools and techniques ranging from hands-on observation to fully automated diagnoses, and includes instructions for building automated debuggers. This discussion is built upon a solid theory of how failures occur, rather than relying on seat-of-the-pants techniques, which are of little help with large software systems or to those learning to program. The author, Andreas Zeller, is well known in the programming community for creating the GNU Data Display Debugger (DDD), a tool that visualizes the data structures of a program while it is running.
- Winner of a 2006 Jolt Productivity Award for Technical Books
- Shows how to reproduce software failures faithfully, how to isolate what is important about the failure, and to discover what caused it
- Describes how to fix the program in the best possible way, and shows how to create your own automated debugging tools
- Includes exercises and extensive references for further study, and a companion website with source code for all examples and additional debugging resources

39. Gerald D. Everett and Raymond, Jr. McLeod - Software Testing: Testing Across the Entire Software Development Life Cycle (use link)
Product Description:
Software Testing presents one of the first comprehensive guides to testing activities, ranging from test planning through test completion for every phase of software under development, and software under revision. Real life case studies are provided to enhance understanding as well as a companion website with tools and examples.

40. Vesna Hassler - Security Fundamentals for E-Commerce (use link)
Product Description:
This handbook offers an in-depth understanding of basic security problems and relevant solutions for those charged with maintaining the security of e-commerce sites. From designing secure Web, e-commerce and mobile commerce applications, to securing your internal network, to providing secure employee/user authentication, this book is structured to provide a comprehensive overview or help the reader pinpoint topics of immediate concern.

Recommended Testing Books - 4th list

2008-11-01T00:44:00.000-07:00

...extending the recommended books with the 4th list (use the Books label to find easily and read the previous 3 lists) :

23. Luciano Baresi and Elisabetta Di Nitto - Test and Analysis of Web Services (use link)
Product Description:
The service-oriented approach has become more and more popular, now allowing highly integrated and yet heterogeneous applications. Web services are the natural evolution of conventional middleware technologies to support Web-based and enterprise-level integration. The highly dynamic characteristics of service-oriented applications means their validation is a continuous process that often runs in parallel with execution. It is not possible to clearly distinguish between the predeployment validation of a system and its use, nor is it possible to guarantee that the checks passed at a certain time will be passed at a later time and in the actual execution environment as well. Baresi and Di Nitto have put together the first reference on all aspects of testing and validating service-oriented architectures, taking into account these inherent intricacies. The contributions by leading academic and industrial research groups are structured into four parts on: static analysis to acquire insight into how the system is supposed to work; testing techniques to sample its actual behavior; monitoring to probe its operational performance; and nonfunctional requirements like reliability and trust. This monograph is an initial source of knowledge for researchers in both academia and industry in the field of service-oriented architecture validation and verification approaches. They will find a comprehensive survey of state-of-the-art approaches as well as techniques and tools to improve the quality of service-oriented applications

24. Mark Fewster, Dorothy Graham - Software Test Automation (use link)
Product description:
Most importantly, this book debunks the myth that testing tools alone will automate the testing process successfully. It does a good job of showing the limitations of today's testing tools. Instead of providing quick fixes, the authors show that with proper planning, you can design and reuse test scripts in addition to automating many aspects of the testing process (such as comparing actual and expected results). With chapters on designing test scripts, creating a testing architecture that will allow you to reuse test cases, and even documentation on choosing the right testing tool for your needs, this book is filled with plenty of nuts-and-bolts advice. Later the authors explain software metrics for tracking defects and how to phase in automated testing for the real-world organization (with a good eye for your shop's return on investment). The book closes with over a dozen case studies from the field, in which testing professionals from large companies (including Microsoft) relate their successes (and failures) with real-world test automation. Accessibly written and filled with real-world expertise, Software Test Automation provides a worthy resource for any test engineer on the effective implementation of automated software testing

25. David J. Smith - Reliability, Maintainability and Risk (use link)
Product Description:
For over 30 years, Reliability, Maintainability and Risk has been recognised as a leading text for reliability and maintenance professionals. Now in its seventh edition, the book has been updated to remain the first choice for professional engineers and students. The seventh edition incorporates new material on important topics including software failure, the latest safety legislation and standards, product liability, integrity of safety-related systems, as well as delivering an up-to-date review of the latest approaches to reliability modelling, including cutsec ranking. It is also supported by new detailed case studies on reliability and risk in practice.
* The leading reliability reference for over 30 years
* Covers all key aspects of reliability and maintenance management in an accessible way with minimal mathematics - ideal for hands-on applications
* Four new chapters covering software failure, safety legislation, safety systems and new case studies on reliability and risk in practice

26. Herbert Hecht- Systems Reliability and Failure Prevention (use link)
Product Description:
Offers a comprehensive treatment of the techniques and practices of systems reliability and failure prevention, without the use of advanced mathematics. Features real-world examples from communication networks, aircraft and missile systems, the process industry, and satellite missions.

27. Wilbert O. Galitz - The Essential Guide to User Interface Design: An Introduction to GUI Design Principles and Techniques (use link)
Product Description:
Bringing together the results of more than 300 new design studies, an understanding of people, knowledge of hardware and software capabilities, and the author’s practical experience gained from 45 years of work with display-based systems, this book addresses interface and screen design from the user’s perspective. You will learn how to create an effective design methodology, design and organize screens and Web pages that encourage efficient comprehension and execution, and create screen icons and graphics that make displays easier and more comfortable to use.

28. Eric Schaffer - Institutionalization of Usability: A Step-by-Step Guide (use link)
Product description:
Institutionalization of Usability shows how to make user-centered design and development a routine practice within an enterprise. Other excellent books explain precisely how to make software usable; this book builds on that foundation, and focuses instead on how to get usability recognized and incorporated into an organization's values and culture. Based on author Eric Schaffer's extensive experience, the book provides a solid methodology for institutionalizing usability, guiding readers step by step with practical advice on topics like organizational change, milestones, toolsets, infrastructure, and staffing requirements needed to achieve fully mature usability engineering. Learn how to:
- Educate your organization about the importance of usability Hire and coordinate usability staff and consultants
- Plan the standards, design, and implementation phases
- Retrofit a method that has added user-centered activitiesRecruit participants for usability interviews and testing
- Select the right staff and project to showcase—by timeline, user impact, and visibility
- Evangelize, train and mentor staff, and support the community
Whether you are an executive leading the institutionalization process, a manager supporting the transition, or an engineer working on usability issues, Institutionalization of Usability will help you to build usability into your software practices.

29. Everett N. McKay - Developing User Interfaces Microsoft for Windows (use link)
Product Description:
Create better user interfaces by designing from the outside in. How important is user interface design? A poorly executed interface can frustrate users - and cost your organization time and money - while an effective design can improve productivity by helping users get their work done. So what makes good interfaces good and bad interfaces bad? Which design choices can improve the user's experience with a program? Emphasizing practical results over theory, this concise, Windows-focused handbook distills industry best practices and the author's 25 years of software development expertise into straightforward and effective methods you can apply right now to create more usable user-driven software. Coverage includes: Basic concepts of user interface (UI) design - knowing the standards; establishing consistency Practical development techniques - specific ways to improve the UI for applications built with the Win32 API or MFC Understanding the user experience - putting the goals of the user ahead of yours; focusing your creativity the "right" way Windows user interface components - mastering the fundamentals; making appropriate choices Testing and evaluation - ensuring software quality and usability

30. James McCaffrey - .NET Test Automation Recipes: A Problem-Solution Approach (use link)
Product Description:
If you develop, test, or manage .NET software, you will find .NET Test Automation Recipes: A Problem-Solution Approach very useful. The book presents practical techniques for writing lightweight software test automation in a .NET environment and covers API testing thoroughly. It also discusses lightweight, custom Windows application UI automation and teaches you low-level web application UI automation. Additional material covers SQL stored procedure testing techniques. The examples in this book have been successfully used in seminars and teaching environments where they have proven highly effective for students who are learning intermediate-level .NET programming. Youll come away from the book knowing how to write production-quality combination and permutation methods.

Suggestions for software testing and quality assurance discussions

2008-10-31T15:50:00.000-07:00

The software testing and quality assurance is a wide area and I'll try to provide the best guidelines for understanding these processes, but you can also help me to understand what type of guidelines you need.
I'll try to cover in my next posts and to provide useful information about:
- fundamental test processes (test planning and control, test analysis, test design, test implementation, test execution, exit criteria, reporting)
- software development models (V-model, testing within a life cycle model)
- test levels (component testing, integration testing, system and acceptance)
- test types (functional and non-functional testing, structural, confirmation, regression testing)
- test techniques (black box technique: equivalence partitioning, boundary value decision table testing, state transition; white box techniques: statement testing and coverage, decision testing and coverage; exploratory testing)
- test planning and estimation (test planning, exit criteria, test estimation, test strategies) and test progress monitoring and control
- determinng the project risks, product risks
- test tools and which is the best choice for specific application areas
- support information (recommended books, standards, courses)
- software testing tips & tricks
- glossary of terms
- and more

So please add your comments in this post and specify what areas of software testing and quality assurance you want to discuss here.

Recommended Testing Books - 3rd list

2008-10-31T12:56:00.000-07:00

...extending the recommended books with the 3rd list (use the Books label to find easily and read the previous 2 lists) :

16. Jeff Tian - Software Quality Engineering: Testing, Quality Assurance, and Quantifiable Improvement (use link)
Product Description:
The one resource needed to create reliable software
This text offers a comprehensive and integrated approach to software quality engineering. By following the author's clear guidance, readers learn how to master the techniques to produce high-quality, reliable software, regardless of the software system's level of complexity.
The first part of the publication introduces major topics in software quality engineering and presents quality planning as an integral part of the process. Providing readers with a solid foundation in key concepts and practices, the book moves on to offer in-depth coverage of software testing as a primary means to ensure software quality; alternatives for quality assurance, including defect prevention, process improvement, inspection, formal verification, fault tolerance, safety assurance, and damage control; and measurement and analysis to close the feedback loop for quality assessment and quantifiable improvement.
This is a superior textbook for software engineering, computer science, information systems, and electrical engineering students, and a dependable reference for software and computer professionals and engineers.

17. Paul Hamill - Unit Test Frameworks (use link)
Product Description:
Unit test frameworks are a key element of popular development methodologies such as eXtreme Programming (XP) and Agile Development. But unit testing has moved far beyond eXtreme Programming; it is now common in many different types of application development. Unit tests help ensure low-level code correctness, reduce software development cycle time, improve developer productivity, and produce more robust software. Until now, there was little documentation available on unit testing, and most sources addressed specific frameworks and specific languages, rather than explaining the use of unit testing as a language-independent, standalone development methodology. This invaluable new book covers the theory and background of unit test frameworks, offers step-by-step instruction in basic unit test development, provides useful code examples in both Java and C++, and includes details on some of the most commonly used frameworks today from the XUnit family, including JUnit for Java, CppUnit for C++, and NUnit for .NET. Unit Test Frameworks includes clear, concise, and detailed descriptions of: The theory and design of unit test frameworks, Examples of unit tests and frameworks, Different types of unit tests, Popular unit test frameworks. It also includes the complete source code for CppUnit for C++, and NUnit for .NET

18. William E. Lewis - Software Testing and Continuous Quality Improvement (use link)
Product Description
Software Testing and Continuous Quality Improvement, Second Edition, illustrates a quality framework for software testing in traditional structured and unstructured environments. It explains how a continuous quality improvement approach promotes effective testing, and it analyzes the various testing tools and techniques that you can choose. Section I explains the role of QA principles and best practices in software testing. It provides a detailed overview of basic software testing techniques and an introduction of Deming's concept of quality through a continuous improvement process. This section explores the Plan, Do, Check, Act (PDCA) process, which is applied to all aspects of software testing. Section II reviews the software development life cycle and describes how testing and continuous quality improvement are incorporated into each phase of development. Section III details continuous quality improvement as part of the testing process. It breaks down software testing into a series of tasks that apply Deming's PDCA cycle. Section IV discusses fundamental challenges of managing testing projects, whether they are on-site or offshore. You learn how to establish effective test estimations to ensure that testing projects are on track. It also covers strategies for monitoring and managing software defects. Section V contains a brief history of software testing, previews advanced futuristic testing tools, and provides guidance for choosing the proper tool for various environments. It provides examples of some of the most popular products and offers a detailed methodology for evaluating them

19. Jonathan Jacky - Model-Based Software Testing and Analysis with C#(use link)
Product Description:
This book teaches new methods for specifying, analyzing, and testing software; essentials for creating high-quality software. These methods increase the automation in each of these steps, making them more timely, more thorough, and more effective. The authors work through several realistic case studies in-depth and detail, using a toolkit built on the C# language and the .NET framework. Readers can also apply the methods in analyzing and testing systems in many other languages and frameworks. Intended for professional software developers including testers, and for university students, this book is suitable for courses on software engineering, testing, specification, or applications of formal methods.

20. Derk-Jan de Grood - TestGoal: Result-Driven Testing (use link)
Product Description:
Software testing is traditionally seen as a difficult and time consuming activity that is hard to embed in the software development process. This book provides a different view. It explains to stakeholders how testing can add value to software development and doing business, and provides the tester with practical information. TestGoal is not just another methodology. Several good testing methodologies exist. But, like any other profession, also testing encompasses more than the simple application of a methodology. After all, strict adherence to a specific methodology is no guarantee for success. Success stems from the mindset, enthusiasm, knowledge and skill of the tester. These factors determine whether a methodology is applied successfully and whether testing takes on a result-driven character. And that’s what TestGoal is about: a result-driven attitude, testing principles and expertise as fundament, and a hands-on six step plan to enable result driven testing

21. Marcus S. Fisher - Software Verification and Validation: An Engineering and Scientific Approach (use link)
Product Description:
The world needs an in-depth technical book describing the methods and techniques used to provide confidence in our system software. In today’s market, not only are governments more focused on software safety, but private industry and academia are as well. The methods and techniques used to provide such confidence is commonly called software verification and validation.
Software Verification and Validation: An Engineering and Scientific Approach, a technical professional book, performs software verification and validation on high-fidelity systems. This book includes crucial information for methods and techniques that not only develop the system, but ensure that the system was developed correctly.
Software Verification and Validation: An Engineering and Scientific Approach is structured for research scientists and practitioners in industry. This book is also suitable as a secondary textbook for advanced-level students in computer science and engineering.

22. Roland Groz - Testing of Communicating Systems (use link)

Product Description:
This book constitutes the refereed proceedings of the 16th IFIP TC6/WG6.1 International Conference on Testing of Communicating Systems, TestCom 2004, held in Oxford, UK in March 2004. The 14 revised full papers presented were carefully reviewed and selected for inclusion in the book. All Current issues in testing communications system are addressed, ranging from classical telecommunications issues to the testing of reactive systems and testing from UML.

Top testing mistakes

2008-10-30T14:47:00.000-07:00

Making mistakes is a natural thing and by testing we try to find them. But what if we do mistakes when trying to find mistakes. Well, I hope we all will learn from them and don't repeat them in the future. Until then, I'll try to enumerate some of those mistakes which I find the most important:
- to think that the only purpose of testing is to find bugs.
- to not find the most important bugs (and to be satisfied with finding a large amount of non-significant bugs)
- to fail to correctly identify risky areas
- to start the testing too late
- poor bug reporting
- to concentrate more on running tests than to design them

I recommend you to read the "Classic Testing Mistakes" article, written by Brian Marick (use the http://www.exampler.com/testing-com/writings/classic/mistakes.html link) if you want to know more about other testing mistakes.

Testing standards (IEEE, BS, CMMI, RUP, etc.)

2008-10-29T16:10:00.000-07:00

Now is time to specify several good "testing" related standards, which you'll probably find somehow interesting :

1. IEEE 1012 Software Verification and Validation
2. IEEE 829 Software test documentation (recommended)
3. BS 7925-2 British Std for Software Component testing (recommended)
4. US FDA General principles of software validation
5. US NIST Reference Information for the Software Verification and Validation Process
6. RUP Test discipline
7. CMMI v1.2 Verification and Validation

I'll try to comment some of these standards in one of the next posts, so you can decide if its can be important or not for you as QA/tester

Testing courses

2008-10-29T15:50:00.000-07:00

Since I've observed your interest for testing courses, not only for books, I've also put together a collection of very good testing courses which I recommend you to read:

1. Cen Kaner - BlackBox testing
2. Cen Kaner - Metrics and Measurement
3. Stale Amland - Exploratory testing
4. James Bach - Rapid Software Testing
5. Sita Ramakrishnan - lectures on testing
6. Brett Pettichord - Test Automation Architectures
7. Scott Barber - User experience, not metrics
8. Scott Barber - Beyond performance testing

Recommended Testing Books - 2nd list

2008-10-28T11:55:00.000-07:00

...extending the recommended books with the 2nd list:

8. Glenford J. Myers - The Art of Software Testing, Second Edition (use link)
Product Description:
This long-awaited revision of a bestseller provides a practical discussion of the nature and aims of software testing. You'll find the latest methodologies for the design of effective test cases, including information on psychological and economic principles, managerial aspects, test tools, high-order testing, code inspections, and debugging. Accessible, comprehensive, and always practical, this edition provides the key information you need to test successfully, whether a novice or a working programmer. Buy your copy today and end up with fewer bugs tomorrow.

9. Rick D. Craig (Author), Stefan P. Jaskiel - Systematic Software Testing (use link)
Product Description:
This textbook sets out to provide professionals with an in-depth understanding of the software-testing people and process issues that are critical for delivering high-quality software on time and within budget. The authors aim to give those involved in building and maintaining complex, mission-critical software systems a flexible, risk-based process to improve their software-testing capabilities. Whether an organization currently has a well-defined testing process or almost no process, this resource provides insights into better ways to test software. This guide is written for: software-test managers; testers; developers; quality-assurance managers; and software configuration managers

10. Elfriede Dustin - Effective Software Testing: 50 Specific Ways to Improve Your Testing (use link)
Product Description:
The knowledge of what constitutes a successful, end-to-end software testing effort is typically gained through experience. In this new book, noted testing expert Elfriede Dustin imparts the best of her collected wisdom. She presents fifty specific tips for a better testing program. These fifty tips are divided into ten sections, and presented so as to mirror the chronology of a software project. Using this book as a guide and reference, quality assurance professionals will be better able to insure the quality of their next application. While testing has historically been viewed as an afterthought in the grand scheme of software engineering, the success of an application, and possibly an organization, can rest on the shoulders of the testing team. That's because the testing program functions as the final "quality gate" for an application. Testing allows or denies the transition of an application into the market. There are a multitude of seemingly minor tasks that must be performed and managed by the testing team.

11. Ron Patton - Software Testing (2nd Edition) (use link)
Product Description:
Software Testing, Second Edition provides practical insight into the world of software testing and quality assurance. Learn how to find problems in any computer program, how to plan an effective test approach and how to tell when software is ready for release. Updated from the previous edition in 2000 to include a chapter that specifically deals with testing software for security bugs, the processes and techniques used throughout the book are timeless. This book is an excellent investment if you want to better understand what your Software Test team does or you want to write better software.

12. Paul C. Jorgensen - Software Testing: A Craftsman's Approach (use link)
Product Description:
Since the last publication of this international bestseller, software testing has seen a renaissance of renewed interest and technology. The biggest change comes in the growing prominence and acceptance of Agile Programming. Software Testing—A Craftsman’s Approach, Third Edition extends the combination of theory and practicality of the first two editions to include agile programming development and discusses the serious effect this emerging area is having on software testing

13. Lydia Ash - The Web Testing Companion: The Insider's Guide to Efficient and Effective Tests (use link)
Product Description:
* Written by one of Microsoft's key testing trainers, this hands-on tutorial and reference explains why, when, what, and how to test
* Teaches new and experienced testers how to analyze and properly test Web applications
* Filled with practical advice that can be immediately applied to any Web-testing task, on any browser running on any platform
* Concentrates on proven solutions and presents the material in a way that will help develop a professional skill set in novice testers and will improve the productivity of all testers
* Companion Web site includes dozens of valuable templates and test patterns that will allow readers to rapidly conduct tests in multiple languages against all browser and operating-system combinations

14. David Astels - Test-Driven Development: A Practical Guide (use link)
Product Description:
This book teaches programmers how to make Test Driven Development (TDD) work in their organization. TDD is unique because it forces the programmer to write tests for code before the code is actually written. This process is the reverse of how software testing has traditionally been conducted, but TDD ensures that software is produced more efficiently. A test-first mentality allows the programmer to define, specify, illustrate, limit, and drive the code, resulting in documented, tested, code that is as simple and lean as possible. The book also presents tools and techniques, and all major points are supported by numerous examples (including an entire project, end-to-end) and exercises.

15. Kent Beck - Test Driven Development: By Example (use link)
Product Description:
Quite simply, test-driven development is meant to eliminate fear in application development. While some fear is healthy (often viewed as a conscience that tells programmers to "be careful!"), the author believes that byproducts of fear include tentative, grumpy, and uncommunicative programmers who are unable to absorb constructive criticism. When programming teams buy into TDD, they immediately see positive results. They eliminate the fear involved in their jobs, and are better equipped to tackle the difficult challenges that face them. TDD eliminates tentative traits, it teaches programmers to communicate, and it encourages team members to seek out criticism However, even the author admits that grumpiness must be worked out individually! In short, the premise behind TDD is that code should be continually tested and refactored. Kent Beck teaches programmers by example, so they can painlessly and dramatically increase the quality of their work

...to be continued

Recommended Testing Books - 1st list

2008-10-26T08:52:00.000-07:00

I've put together a collection of my favorite testing books which I recommend you to read ( I'll also try to specify a direct link for each of these recommended books, especially to the appropriate page on Amazon.com).
I'll add additional books in other lists order to extend the recommended books.

1. Rex Black - Critical Testing Processes: Plan, Prepare, Perform, Perfect (use link)
Product description:
Text presents twelve testing processes that are critical to success. After each process is introduced, a case study demonstrates its use in various organizational, operational, and technological contexts. For computer software test professionals. Softcover. DLC: Computer software--Testing...

2. Lee Copeland - A practitioner's guide to software test design (use link)
Product description:
Here’s a comprehensive, up-to-date and practical introduction to software test design. This invaluable book presents all the important test design techniques in a single place and in a consistent, and easy-to-digest format. An immediately useful handbook for test engineers, developers, quality assurance professionals, and requirements and systems analysts, it enables you to: choose the best test case design, find software defects in less time and with fewer resources, and develop optimal strategies that help reduce the likelihood of costly errors. It also assists you in estimating the effort, time and cost of good testing...

3. Yu k Kuen Wong - Modern software review: Techniques and Technologies (use link)
Product description:
Software quality is vitally important to the success of a business, as a single undetected error or defect during the software development process could have disastrous consequences during a business operation. Software review is one of the methods used to detect defects. This process maintains the quality of the product by reviewing interim deliverables during development. Modern Software Review: Techniques and Technologies provides an understanding of the critical factors affecting software review performance and gives practical guidelines for software reviews...

4. Jerry Zeyu Gao - Testing and Quality Assurance for Component-Based Software (use link)
Product description:
Presenting the state-of-the-art in component-based software testing, this new, cutting-edge resource offers you an in-depth understanding of the current issues, challenges, needs, and solutions in this critical area. The book discusses the very latest advances in component-based testing and quality assurance in an accessible tutorial format, making the material easy to comprehend and benefit from no matter what your professional level...

5. Cem Kaner, Jack Falk, Hung Quoc Nguyen - Testing Computer Software (use link)
Product description:
The Testing Computer Software set the standard for the emerging field of test engineering with a full tour of the state of the art in managing the testing process. The reissued text makes this classic out-of-print text available once again. Though it relies heavily on older examples (including MS-DOS) and has not been updated, this text is still a worthwhile resource for practical-minded advice on the realities of testing. The best thing about Testing Computer Software is its practical point-by-point guide to everyday software testing, from creating a test plan, to writing effective bug reports, to working with programming staff and management to fix bugs...

6. Ilene Burstein - Practical Software Testing (use link)
Product Description:
Software testing is rapidly evolving as a critical sub-discipline of software engineering. The industry needs professionals educated in the modern methods and tools of software testing and quality assurance. Based on the needs of software professionals, "Practical Software Testing" takes a unique approach to teaching readers how to effectively plan for testing, design test cases, test at multiple levels, organize a testing team, and optimize use of testing tools. It introduces testing concepts that are managerial-, technical-, and process-oriented, using the Testing Maturity Model (TMM) as a framework....

7. Rex Black - Pragmatic Software Testing: Becoming an Effective and Efficient Test Professional (use link)
Product Description:
A hands-on guide to testing techniques that deliver reliable software and systems
Testing even a simple system can quickly turn into a potentially infinite task. Faced with tight costs and schedules, testers need to have a toolkit of practical techniques combined with hands-on experience and the right strategies in order to complete a successful project. World-renowned testing expert Rex Black provides you with the proven methods and concepts that test professionals must know. He presents you with the fundamental techniques for testing and clearly shows you how to select and apply successful strategies to test a system with budget and time constraints...

...to be continued

Testing terminology - Defect ( Bug, Fault or Problem)

2008-10-25T11:33:00.000-07:00

"Defect" definitions (same with Bug, Fault or Problem, but with some "nuances" ) :
- an imperfection that causes inadequacy or failure
- the lack of something necessary or desirable for completion or perfection
- an imperfection or weakness; fault; flaw; blemish
- any variance from a desired attribute, state, or functionality
- anything that either adversely impacts the expected functionality to the user, or otherwise reduces the quality of a work product
- (bug) fault or defect in a computer program, system, or machine
- (bug) a defect or difficulty, as in a system or design
- (bug) a defect in the code or routine of a program
- (problem) a state of difficulty that needs to be resolved
- (problem) a source of difficulty, etc..

The defect is a "flaw" in a component, that can cause it to fail to perform its requirements and it may cause the failure of the component while using it in real-time.
I've specified that the meaning of defect - bug - fault - problem is about the same, but with certain nuances; "Bug" implies that there is a fault made in a software, by developers, while the "Defect" is perceived as being more severe.
Also, we must not get confused by the sense of the "defect" and the "error" terms.
E.g. :
- for a calculation component of an application is indicated ( in its specifications) that x - x = 10 x;
this means that the one who wrote it in the specifications would have made an Error and by doing this he would have created a Defect.

... please comment if you have other suitable definitions

Testing terminology - Error

2008-10-25T10:52:00.000-07:00

"Error" definitions:
- a mistake
- a wrong action attributable to bad judgment or ignorance or inattention
- a misconception resulting from incorrect information
- a human action that produces an incorrect result
- the difference between a computed or measured value and a true or theoretically correct value
- the act or an instance of deviating from an accepted code of behavior, etc.

The errors are not restricted only to the code wrote by the developer (errors can occur also in the technical, functional specifications or design stages, so anywhere in the software development life cycle)

... please comment if you have other suitable definitions

Testing - definitions

2008-10-25T09:46:00.000-07:00

So, what is testing? We can find and give various of simple or complete definitions for this question:
- "...the process of executing a program with the intent of finding errors"
- "... testing is an empirical investigation conducted to provide stakeholders with information about the quality of the product or service under test, with respect to the context in which it is intended to operate. This includes, but is not limited to, the process of executing a program or application with the intent of finding "
- "... testing is finding out how well something works. In terms of human beings, testing tells what level of knowledge or skill has been acquired. In computer hardware and software development, testing is used at key "
checkpoints in the overall process to determine whether objectives are being met"
- "... the process of executing a program with the intent to certify its Quality"
etc.
But testing is mainly the process which consist of the following activities:
- review requirements
- planning
- preparation and evaluation
- defect detection,
in order to satisfy the specified product requirements (it involves operation of a system or application under controlled conditions and evaluating the results).

Testing - a necessity

2008-10-24T14:45:00.001-07:00

...software testing is a need. Why? Because the softwares built today are so complex and released with a stunning speed, due to the tough market competition. This implicitly leads to failures/mistakes/oversights, which will end up with incomplete user requirements (missed out somewhere during the development process) or not functional areas.
So, by software testing we reduce the failures occurance and provide to the user a product which meet all the specified user requirements.
But in the main the answer should be - because of the cost of error.
Each failure involves a variable cost, depending on how important it is, the area of activity, etc. Imagine an important failure in a safety critical software, for example a control system used by an airplane or by a medical device. Exactly, besides millions it can also cost lives.
Very important on the cost estimation is to understand that the cost of defects increases with the passing of the development process stages (in the first stages - e.g. requirements review stage - it may cost very little, but in a post-implementation stage it will cost a lot, for sure).
The benefits of software testing can be resumed at the followings: to reduce the number of defects (contained in the released version), to improve the quality of the software and to increase the reliability of the product.
We won't be able to test everything, because of the time and resources constraints on the level of testing performed, but we can determine the specific project risk factors based on the testing performed during the software development process.