SQTESTER - Software Testing Arena

Question Bank - 2

2008-07-07T06:25:00.000-07:00

NOTE: Only one answer per question
1-When what is visible to end-users is a deviation from the specific or expected behavior, this is called:
a) - An error
b) - A fault
c) - A failure
d) - A defect
e) - A mistake

2-Regression testing should be performed:
v) - Every week
w) - After the software has changed
x) - As often as possible
y) - When the environment has changed
z) - When the project manager says

a) - v & w are true, x – z are false
b) - w, x & y are true, v & z are false
c) - w & y are true, v, x & z are false
d) - w is true, v, x y and z are false
e) - All of the above are true

3-IEEE 829 test plan documentation standard contains all of the following except:
a) - Test items
b) - Test deliverables
c) - Test tasks
d) - Test environment
e) - Test specification

4-Testing should be stopped when:
a) - All the planned tests have been run
b) - Time has run out
c) - All faults have been fixed correctly
d) - Both a) and c)
e) - It depends on the risks for the system being tested

5-Order numbers on a stock control system can range between 10000 and 99999 inclusive. Which of the following inputs might be a result of designing tests for only valid equivalence classes and valid boundaries:
a) - 1000, 5000, 99999
b) - 9999, 50000, 100000
c) - 10000, 50000, 9999
d) - 10000, 99999
e) - 9999, 10000, 50000, 99999, 10000

6-Consider the following statements about early test design:
i. Early test design can prevent fault multiplication
ii. Faults found during early test design are more expensive to fix
iii. Early test design can find faults
iv. Early test design can cause changes to the requirements
v. Early test design takes more effort

a) - i, iii & iv are true. Ii & v are false
b) - iii is true, I, ii, iv & v are false
c) - iii & iv are true. i, ii & v are false
d) - i, iii, iv & v are true, ii us false
e) - i & iii are true, ii, iv & v are false

7-Non-functional system testing includes:
a) - Testing to see where the system does not function properly
b) - Testing quality attributes of the system including performance and usability
c) - Testing a system feature using only the software required for that action
d) - Testing a system feature using only the software required for that function
e) - Testing for functions that should not exist

8-Which of the following is NOT part of configuration management:
a) - Status accounting of configuration items
b) - Auditing conformance to ISO9001
c) - Identification of test versions
d) - Record of changes to documentation over time
e) - Controlled library access

9-Which of the following is the main purpose of the integration strategy for integration testing in the small?
a) - To ensure that all of the small modules are tested adequately
b) - To ensure that the system interfaces to other systems and networks
c) - To specify which modules to combine when and how many at once
d) - To ensure that the integration testing can be performed by a small team
e) - To specify how the software should be divided into modules

10-What is the purpose of test completion criteria in a test plan:
a) - To know when a specific test has finished its execution
b) - To ensure that the test case specification is complete
c) - To set the criteria used in generating test inputs
d) - To know when test planning is complete
e) – To plan when to stop testing

11-Consider the following statements
i. An incident may be closed without being fixed
ii. Incidents may not be raised against documentation
iii. The final stage of incident tracking is fixing
iv. The incident record does not include information on test environments
v. Incidents should be raised when someone other than the author of the software performs the test

a) - ii and v are true, I, iii and iv are false
b) - i and v are true, ii, iii and iv are false
c) - i, iv and v are true, ii and iii are false
d) - i and ii are true, iii, iv and v are false
e) - i is true, ii, iii, iv and v are false

12-Given the following code, which is true about the minimum number of test cases required for full statement and branch coverage:
Read P
Read Q
IF P+Q > 100 THEN
Print “Large”
ENDIF
If P > 50 THEN
Print “P Large”
ENDIF

a) - 1 test for statement coverage, 3 for branch coverage
b) - 1 test for statement coverage, 2 for branch coverage
c) - 1 test for statement coverage, 1 for branch coverage
d) - 2 tests for statement coverage, 3 for branch coverage
e) - 2 tests for statement coverage, 2 for branch coverage

13-Given the following:
Switch PC on
Start “outlook”
IF outlook appears THEN
Send an email
Close outlook

a) - 1 test for statement coverage, 1 for branch coverage
b) - 1 test for statement coverage, 2 for branch coverage
c) - 1 test for statement coverage. 3 for branch coverage
d) - 2 tests for statement coverage, 2 for branch coverage
e) - 2 tests for statement coverage, 3 for branch coverage

14-Given the following code, which is true:
IF A > B THEN
C = A – B
ELSE
C = A + B
ENDIF
Read D
IF C = D Then
Print “Error”
ENDIF

a) - 1 test for statement coverage, 3 for branch coverage
b) - 2 tests for statement coverage, 2 for branch coverage
c) - 2 tests for statement coverage. 3 for branch coverage
d) - 3 tests for statement coverage, 3 for branch coverage
e) - 3 tests for statement coverage, 2 for branch coverage

15-Consider the following:
Pick up and read the newspaper
Look at what is on television
If there is a program that you are interested in watching then switch the the television on and watch the program
Otherwise
Continue reading the newspaper
If there is a crossword in the newspaper then try and complete the crossword

a) - SC = 1 and DC = 1
b) - SC = 1 and DC = 2
c) - SC = 1 and DC = 3
d) - SC = 2 and DC = 2
e) - SC = 2 and DC = 3

16-The place to start if you want a (new) test tool is:
a) - Attend a tool exhibition
b) - Invite a vendor to give a demo
c) - Analyze your needs and requirements
d) - Find out what your budget would be for the tool
e) - Search the internet

17-When a new testing tool is purchased, it should be used first by:
a) - A small team to establish the best way to use the tool
b) - Everyone who may eventually have some use for the tool
c) - The independent testing team
d) - The managers to see what projects it should be used in
e) - The vendor contractor to write the initial scripts

18-What can static analysis NOT find?
a) - The use of a variable before it has been defined
b) - Unreachable (“dead”) code
c) - Whether the value stored in a variable is correct
d) - The re-definition of a variable before it has been used
e) - Array bound violations

19-Which of the following is NOT a black box technique:
a) - Equivalence partitioning
b) - State transition testing
c) - LCSAJ
d) - Syntax testing
e) - Boundary value analysis

20-Beta testing is:
a) - Performed by customers at their own site
b) - Performed by customers at their software developer’s site
c) - Performed by an independent test team
d) - Useful to test bespoke software
e) - Performed as early as possible in the lifecycle

21-Given the following types of tool, which tools would typically be used by developers and which by an independent test team:
i. Static analysis
ii. Performance Testing
iii. Test Management
iv. Dynamic Analysis
v. Test Running
vi. Test Data Preparation

a) - Developers would typically use i, iv and vi; test team ii, iii and v
b) - Developers would typically use i and iv; test team ii, iii, v and vi
c) - Developers would typically use i, ii, iii and iv; test team v and vi
d) - Developers would typically use ii, iv and vi; test team I, ii and v
e) - Developers would typically use i, iii, iv and v; test team ii and vi

22- The main focus of acceptance testing is:
a) - Finding faults in the system
b) - Ensuring that the system is acceptable to all users
c) - Testing the system with other systems
d) - Testing for a business perspective
e) - Testing by an independent test team

23-Which of the following statements about the component testing standard is false:
a) - Black box design techniques all have an associated measurement technique
b) - White box design techniques all have an associated measurement technique
c) - Cyclomatic complexity is not a test measurement technique
d) - Black box measurement techniques all have an associated test design technique
e) - White box measurement techniques all have an associated test design technique

24-Which of the following statements is NOT true:
a) - Inspection is the most formal review process
b) - Inspections should be led by a trained leader
c) - Managers can perform inspections on management documents
d) - Inspection is appropriate even when there are no written documents
e) - Inspection compares documents with predecessor (source) documents

25-A typical commercial test execution tool would be able to perform all of the following EXCEPT:
a) - Generating expected outputs
b) - Replaying inputs according to a programmed script
c) - Comparison of expected outcomes with actual outcomes
d) - Recording test inputs
e) - Reading test values from a data file

26-The difference between re-testing and regression testing is
a) - Re-testing is running a test again; regression testing looks for unexpected side effects
b) - Re-testing looks for unexpected side effects; regression testing is repeating those tests
c) - Re-testing is done after faults are fixed; regression testing is done earlier
d) - Re-testing uses different environments, regression testing uses the same environment
e) - Re-testing is done by developers, regression testing is done by independent testers

27-Expected results are:
a) - Only important in system testing
b) - Only used in component testing
c) - Never specified in advance
d) - Most useful when specified in advance
e) - Derived from the code

28-Test managers should not:
a) - Report on deviations from the project plan
b) - Sign the system off for release
c) - Re-allocate resource to meet original plans
d) - Raise incidents on faults that they have found
e) - Provide information for risk analysis and quality improvement

29-Unreachable code would best be found using:
a) - Code reviews
b) - Code inspections
c) - A coverage tool
d) - A test management tool
e) - A static analysis tool

30-A tool that supports traceability, recording of incidents or scheduling of tests is called:
a) - A dynamic analysis tool
b) - A test execution tool
c) - A debugging tool
d) - A test management tool
e) – A configuration management tool

31-What information need not be included in a test incident report:
a) - How to fix the fault
b) - How to reproduce the fault
c) - Test environment details
d) - Severity, priority
e) - The actual and expected outcomes

32-Which expression best matches the following characteristics or review processes:
1. Led by author
2. Undocumented
3. No management participation
4. Led by a trained moderator or leader
5. Uses entry exit criteria

s) - Inspection
t) - Peer review
u) - Informal review
v) - Walkthrough

a) - s = 4, t = 3, u = 2 and 5, v = 1
b) - s = 4 and 5, t = 3, u = 2, v = 1
c) - s = 1 and 5, t = 3, u = 2, v = 4
d) - s = 5, t = 4, u = 3, v = 1 and 2
e) - s = 4 and 5, t = 1, u = 2, v = 3

33-Which of the following is NOT part of system testing:
a) - Business process-based testing
b) - Performance, load and stress testing
c) - Requirements-based testing
d) - Usability testing
e) - Top-down integration testing

34- What statement about expected outcomes is FALSE:
a) - Expected outcomes are defined by the software’s behaviour
b) - Expected outcomes are derived from a specification, not from the code
c) - Expected outcomes include outputs to a screen and changes to files and databases
d) - Expected outcomes should be predicted before a test is run
e) - Expected outcomes may include timing constraints such as response times

35-The standard that gives definitions of testing terms is:
a) - ISO/IEC 12207
b) - BS7925-1
c) - BS7925-2
d) - ANSI/IEEE 829
e) - ANSI/IEEE 729

36-The cost of fixing a fault:
a) - Is not important
b) - Increases as we move the product towards live use
c) - Decreases as we move the product towards live use
d) - Is more expensive if found in requirements than functional design
e) - Can never be determined

37-Which of the following is NOT included in the Test Plan document of the Test Documentation Standard:
a) - Test items (i.e. software versions)
b) - What is not to be tested
c) - Test environments
d) - Quality plans
e) - Schedules and deadlines

38-Could reviews or inspections be considered part of testing:
a) - No, because they apply to development documentation
b) - No, because they are normally applied before testing
c) - No, because they do not apply to the test documentation
d) - Yes, because both help detect faults and improve quality
e) - Yes, because testing includes all non-constructive activities

39-Which of the following is not part of performance testing:
a) - Measuring response time
b) - Measuring transaction rates
c) - Recovery testing
d) - Simulating many users
e) - Generating many transactions

40-Error guessing is best used
a) - As the first approach to deriving test cases
b) - After more formal techniques have been applied
c) - By inexperienced testers
d) - After the system has gone live
e) - Only by end users

Answers - Question Bank -2

Requirements Analysis Process

2008-07-07T02:43:00.000-07:00

Requirements Analysis Process: Requirements Elicitation, Analysis And Specification

Requirements Analysis is the process of understanding the customer needs and expectations from a proposed system or application and is a well-defined stage in the Software Development Life Cycle model.

Requirements are a description of how a system should behave or a description of system properties or attributes. It can alternatively be a statement of ‘what’ an application is expected to do.

Given the multiple levels of interaction between users, business processes and devices in global corporations today, there are simultaneous and complex requirements from a single application, from various levels within an organization and outside it as well.

The Software Requirements Analysis Process covers the complex task of eliciting and documenting the requirements of all these users, modeling and analyzing these requirements and documenting them as a basis for system design.

A dedicated and specialized Requirements Analyst is best equipped to handle the job. The Requirements Analysis function may also fall under the scope of Project Manager, Program Manager or Business Analyst, depending on the organizational hierarchy.

Software Requirements Analysis and Documentation Processes are critical to software project success. Requirements Engineering is an emerging field which deals with the systematic handling of requirements.

Why is Requirements Analysis necessary?

Studies reveal that inadequate attention to Software Requirements Analysis at the beginning of a project is the most common cause for critically vulnerable projects that often do not deliver even on the basic tasks for which they were designed. There are instances of corporations that have spent huge amounts on software projects where the end application eventually does not perform the tasks it was intended for.

Software companies are now investing time and resources into effective and streamlined Software Requirements Analysis Processes as a prerequisite to successful projects that align with the client’s business goals and meet the project’s requirement specifications.

Steps in the Requirements Analysis Process

I. Fix system boundaries

This initial step helps in identifying how the new application integrates with the business processes, how it fits into the larger picture and what its scope and limitations will be.

II. Identify the customer

In more recent times there has been a focus on identifying who the ‘users’ or ‘customers’ of an application are. Referred to broadly as the ‘stake holders’, these indicate the group or groups of people who will be directly or indirectly impacted by the new application.

By defining in concrete terms who the intended user is, the Requirements Analyst knows in advance where he has to look for answers. The Requirements Elicitation Process should focus on the wish-list of this defined group to arrive at a valid requirements list.

III. Requirements elicitation

Information is gathered from the multiple stakeholders identified. The Requirements Analyst draws out from each of these groups what their requirements from the application are and what they expect the application to accomplish.

Considering the multiple stakeholders involved, the list of requirements gathered in this manner could run into pages. The level of detail of the requirements list is based on the number and size of user groups, the degree of complexity of business processes and the size of the application.

a) Problems faced in Requirements Elicitation

Ambiguous understanding of processes

Inconsistency within a single process by multiple users

Insufficient input from stakeholders

Conflicting stakeholder interests

Changes in requirements after project has begun

A Requirements Analyst has to interact closely with multiple work-groups, often with conflicting goals, to arrive at a bona fide requirements list. Strong communication and people skills along with sound programming knowledge are prerequisites for an expert Requirements Analyst.

b) Tools used in Requirements Elicitation

Traditional methods of Requirements Elicitation included stakeholder interviews and focus group studies. Other methods like flowcharting of business processes and the use of existing documentation like user manuals, organizational charts, process models and systems or process specifications, on-site analysis, interviews with end-users, market research and competitor analysis were also used extensively in Requirements Elicitation.

However current research in Software Requirements Analysis Process has thrown up modern tools that are better equipped to handle the complex and multilayered process of Requirements Elicitation. Some of the current Requirements Elicitation tools in use are:

Prototypes

Use cases

Data flow diagrams

Transition process diagrams

User interfaces

IV. Requirements Analysis Process

Once all stakeholder requirements have been gathered, a structured analysis of these can be done after modeling the requirements. Some of the Software Requirements Analysis techniques used are requirements animation, automated reasoning, knowledge-based critiquing, consistency checking, analogical and case-based reasoning.

V. Requirements Specification

Requirements, once elicited, modeled and analyzed should be documented in clear, unambiguous terms. A written requirements document is critical so that its circulation is possible among all stakeholders including the client, user-groups, the development and testing teams. Current requirements engineering practices reveal that a well-designed, clearly documented Requirements Specification is vital and serves as a:

Base for validating the stated requirements and resolving stakeholder conflicts, if any

Contract between the client and development team

Basis for systems design for the development team

Bench-mark for project managers for planning project development lifecycle and goals

Source for formulating test plans for QA and testing teams

Resource for requirements management and requirements tracing

Basis for evolving requirements over the project life span

Software requirements specification involves scoping the requirements so that it meets the customer’s vision. It is the result of collaboration between the end-user who is often not a technical expert, and a Technical/Systems Analyst, who is likely to approach the situation in technical terms.

The software requirements specification is a document that lists out stakeholders’ needs and communicates these to the technical community that will design and build the system. The challenge of a well-written requirements specification is to clearly communicate to both these groups and all the sub-groups within.

To overcome this, Requirements Specifications may be documented separately as

User Requirements - written in clear, precise language with plain text and use cases, for the benefit of the customer and end-user

System Requirements - expressed as a programming or mathematical model, addressing the Application Development Team and QA and Testing Team.

Requirements Specification serves as a starting point for software, hardware and database design. It describes the function (Functional and Non-Functional specifications) of the system, performance of the system and the operational and user-interface constraints that will govern system development.

VI. Requirements Management

Requirements Management is the comprehensive process that includes all aspects of software requirements analysis and additionally ensures verification, validation and traceability of requirements. Effective requirements management practices guarantee that all system requirements are stated unambiguously, that omissions and errors are corrected and that evolving specifications can be incorporated later in the project lifecycle.

Types Of Reviews

2008-07-05T09:25:00.000-07:00

Formal review:

A typical formal review has the following main phases:

1. Planning: selecting the personnel, allocating roles; defining the entry and exit criteria for more

formal review types (e.g. inspection); and selecting which parts of documents to look at.

2. Kick-off: distributing documents; explaining the objectives, process and documents to the participants; and checking entry criteria (for more formal review types).

3. Individual preparation: work done by each of the participants on their own before the review meeting, noting potential defects, questions and comments.

4. Review meeting: discussion or logging, with documented results or minutes (for more formal review types). The meeting participants may simply note defects, make recommendations for handling the defects, or make decisions about the defects.

5. Rework: fixing defects found, typically done by the author.

6. Follow-up: checking that defects have been addressed, gathering metrics and checking on exit criteria (for more formal review types).

Roles and responsibilities

A typical formal review will include the roles below:

Manager: decides on the execution of reviews, allocates time in project schedules and determines if the review objectives have been met.
Moderator: the person who leads the review of the document or set of documents, including planning the review, running the meeting, and follow-up after the meeting. If necessary, the moderator may mediate between the various points of view and is often the person upon whom the success of the review rests.
Author: the writer or person with chief responsibility for the document(s) to be reviewed.
Reviewers: individuals with a specific technical or business background (also called checkers or inspectors) who, after the necessary preparation, identify and describe findings (e.g. defects) in the product under review. Reviewers should be chosen to represent different perspectives and roles in the review process, and should take part in any review meetings.
Scribe (or recorder): documents all the issues, problems and open points that were identified during the meeting. Peer Review is generally a one-to-one meeting between the author of a work product and a peer, initiated as a request for import regarding a particular artifact or problem. There is no agenda, and results are not formally reported. These reviews occur on an as needed basis throughout each phase of a project.

Informal review

Key characteristics:

no formal process;
there may be pair programming or a technical lead reviewing designs and code;
optionally may be documented;
may vary in usefulness depending on the reviewer;
Main purpose: inexpensive way to get some benefit.

Walkthrough

Key characteristics:

meeting led by author;
scenarios, dry runs, peer group;
open-ended sessions;
optionally a pre-meeting preparation of reviewers, review report, list of findings and scribe (who is not the author);
May vary in practice from quite informal to very formal;
Main purposes: learning, gaining understanding, defect finding.

Technical review

Key characteristics:

documented, defined defect-detection process that includes peers and technical experts;
may be performed as a peer review without management participation;
ideally led by trained moderator (not the author);
pre-meeting preparation;
optionally the use of checklists, review report, list of findings and management participation;
may vary in practice from quite informal to very formal;
Main purposes: discuss, make decisions, evaluate alternatives, find defects, solve technical problems and check conformance to specifications and standards.

Inspection

Key characteristics:

led by trained moderator (not the author);
usually peer examination;
defined roles;
includes metrics;
formal process based on rules and checklists with entry and exit criteria;
pre-meeting preparation;
inspection report, list of findings;
formal follow-up process;
optionally, process improvement and reader;
Main purpose: find defects.

Need Of Testing

2008-07-03T06:18:00.000-07:00

Why do software faults occur?
All of the products of software development (specifications, source code, and test documents) are written by human beings. All human beings are prone to make errors regardless of how experienced or skilled they are. So a number of faults in software are inevitable/expected.

Furthermore, many of the computer systems being developed today are so large and complex that it is not generally possible for any one person to understand every aspect of a system. Many people have specialized in particular areas or aspects, for example we have database experts, software design experts, algorithm experts, and network and operating system experts. Also, there are experts in specific areas of the businesses that the systems support. Good documentation (specifications and the like) are necessary to communicate all the relevant information each person needs to complete their parts of a system. However, people are fallible and so make errors causing faults in the specifications. When dealing with one part of a system, a systems designer or programmer may think they understand some other aspect outside of their particular area of expertise but they may not. Assumptions are often made because it is easier and quicker to assume one thing than to find out what the real answer should be.

What do software faults cost?
The cost of software faults that are not detected before a system is put into live operation varies, depending on a number of factors.

Some of the more spectacular software failures are well known. For example the European Space Agency’s Ariane 5 rocket exploded 37 seconds into its launch because of a software fault. This fault cost $7billion. Although it had been tested a series of errors meant that out-of-date test data was used. Another example is the Mariner Space probe. This was meant to be going to Venus but lost its way as a result of a software fault. The FORTRAN program had a full stop instead of a comma in a FOR (looping) statement that the compiler accepted as an assignment statement.

A couple of more recent examples involve Mars probes. In the first example the one organization’s software calculated the distance to landing in inches while another calculated the distance in centimeters. This resulted in the loss of the probe at a cost of $125 million. In the second example, a probe was lost shortly before it should have landed at a cost of $250 million. It turned out to be spurious signals from hardware sensors were interpreted by the software as landing, so antennas were deployed, protective covers removed, etc. This fault had been identified in earlier testing, and changes were made to the hardware and the software. However, there was no final regression test, due to time and budget pressure. (The lack of a final test was also responsible for the Hubble telescope being myopic for its first years of operation.)

One final example is where the expected results were not calculated beforehand and in this instance cost American Airlines $5Omillion. By way of background to this example, airlines never want to fly with empty seats - they would prefer to offer discount seats than to fly empty. However they also do not want to discount the seat when they could receive the full price. Airlines run complex ‘yield management programs’ to achieve the balance right. But these programs often change a lot to increase profitability. One such change was catastrophic for American Airlines. The program gave reasonable -but wrong - results for the number of discount seats! A spokesman said ‘We’re convinced that if we would have done more thorough testing, we would have discovered the problem before the software was ever brought on-line’.

Of course not every software fault causes such huge failure costs (fortunately!). Some failures are simple spelling mistakes or a misalignment in a report.

There is a vast array of different costs but there is a problem, software is not linear. Small faults can have large effects. For example a spelling mistake on a screen title said “ABC SOFTWEAR”, a simple mistake but it cost the company a sale. Another spelling mistake was when the programmer spelt an insurance company’s name wrong — instead of “Union” they spelt it “Onion” — only one letter, but this was a major embarrassment.

The consequences of faults in safety critical systems can be much more serious. The follow examples are all real cases.

· Therac-25 (Canada) - this machine was used in the treatment of cancer patients. It had the capability of delivering both x-rays and gamma rays. The normal dose of x-rays is relatively low while gamma ray dosage is much higher. Switching from one type of ray to the other was a slow process but the operators discovered that if they typed fast it could be made to switch more quickly. However, it only appeared to have switched. Six people died as a result of having too high an x-ray dose. This was a design fault.

· A train driver in Washington DC was killed when an empty train failed to stop in a stabling siding. The train was fully automated; driver just opens and closes doors. Driver had been requesting manual operation, as the trains had been overshooting the stops - this was denied by the humans in “central control” - RISKS Forum, 26 Jan 1998.

· Korean Airlines Flight 801, Boeing 747, 29 survivors (out of 254 passengers and crew). Came in 1000 ft too low. Combination of “normal” factors: severe weather, ground proximity system not switched on (or automatically disabled when landing gear lowered), secondary radar system should have sounded an alarm - either didn’t or wasn’t noticed. - Computer Weekly, 14 Aug 1987.

· Airbus crashes into trees at an air show at Habsham, Germany. Software protected engines by preventing them from being accelerated as fast as the pilot required to avoid hitting some trees after a low fly pass.

The following example is not of a safety critical system, rather a banking system. However, this did contribute to a death.

· An elderly man bought presents for his grandchildren, and as a result became overdrawn for the first time in his life. The banks system automatically sent threatening letters and charged him for being overdrawn. The man committed suicide. Source: client from major UK bank.

Testing is necessary:
Testing is necessary because software is likely to have faults in it and it is better (cheaper, quicker and more expedient) to find and remove these faults before it is put into live operation. Failures that occur during live operation are much more expensive to deal with than failures that occur during testing prior to the release of the software. Of course other consequences of a system failing during live operation include the possibility of the software supplier being sued by the customers!

Testing is also necessary so we can learn about the reliability of the software (that is, how likely it is to fail within a specified time under specified conditions).

Testing is not ‘to prove that the software has no faults’ because it is impossible to prove that software has no faults. Neither is testing done ‘because testing is included in the project plan’. Testing should be included in the project plan, but this is not the reason for testing. (Why is this testing included in the plan - this is the reason for testing.)

Testing is essential in the development of any software system. Testing is needed in order to assess what the system actually does, and how well it does it, in its final environment. A system without testing is merely a paper exercise - it may work or it may not, but there is no way of knowing it without testing it.

Exhaustive testing:
Exhaustive testing is defined as exercising every possible combination of inputs and preconditions. This usually amounts to an impractical number of test cases. For most systems it is relatively easy to identify millions of possible test cases (not necessarily all desirable test cases). For example, a system that accepts an input value corresponding to an employee’s annual salary may be expected to handle figures in the range £0.00 to £999,999.00. Given that every different input value could be made a separate test case this means that we can identify 100,000,000 test cases (the number of ip’s in the range) just from this one input value. Of course, no one would reasonably expect to have to try out every one of these input values but that is what exhaustive testing would require. Exhaustive testing would also require each of these 100,000,000 input values to be tried in different combinations with other inputs and with different preconditions (such as data values in a database). For even simple systems the number of test cases required for exhaustive testing is likely to run into millions of millions of millions.

Exhaustive testing is, therefore, impractical (it is not considered impossible, as it may be achievable for a very small program).

How much testing is enough?
It is possible to do enough testing but determining the how much is enough is difficult. Simply doing what is planned is not sufficient since it leaves the question as to how much should be planned. What is enough testing can only be confirmed by assessing the results of testing. If lots of faults are found with a set of planned tests it is likely that more tests will be required to assure that the required level of software quality is achieved. On the other hand, if very few faults are found with the planned set of tests, then (providing the planned tests can be confirmed as being of good quality) no more tests will be required.

Saying that enough testing is done when the customers or end-users are happy is a bit late, even though it is a good measure of the success of testing. However, this may not be the best test stopping criteria to use if you have very demanding end-users who are never happy!

Why not stop testing when you have proved that the system works? It is not possible to prove that a system works without exhaustive testing (which is totally impractical for real systems).

Have you tested enough when you are confident that the system works correctly? This may be a reasonable test stopping criterion, but we need to understand how well justified that confidence is. It is easy to give yourself false confidence in the quality of a system if you do not do good testing.

Ultimately, the answer to “How much testing is enough?” is “It depends!” (this was first pointed out by Bill Hetzel in his book “The Complete Guide to Software Testing”). It depends on risk, the risk of missing faults, of incurring high failure costs, of losing creditability and market share. All of these suggest that more testing is better. However, it also depends on the risk of missing a market window and the risk of over-testing (doing ineffective testing) which suggest that less testing may be better.

We should use risk to determine where to place the emphasis when testing by prioritizing our test cases. Different criteria can be used to prioritize testing including complexity, criticality, visibility and reliability.

Testing measures software quality:
Testing and quality go hand in hand! Basically we don’t know how good the software is until we have run some tests. Once we have run some good tests we can state how many faults we have found (of each severity level) and may also be able to predict how many faults remain (of each severity level).

Other factors that influence testing:
Other factors that affect our decision on how much testing to perform include possible contractual obligations. For example, a contract between a customer and a software supplier for a bespoke system may require the supplier to achieve 100% statement coverage (coverage measures are discussed in session 3). Similarly, legal requirements may impose a particular degree of thoroughness in testing although it is more likely that any legal requirements will require detailed records to be kept (this could add to the administration costs of the testing).
In some industries (such as the pharmaceutical industry and safety critical industries such as railroad switching and air traffic control) there are standards defined that have the intent of ensuring rigorous testing.

Configuration Management

2008-07-03T04:25:00.000-07:00

Configuration management (CM): is a field of management that focuses on establishing and maintaining consistency of a product's performance and its functional and physical attributes with its requirements, design, and operational information throughout its life. For information assurance, CM can be defined as the management of security features and assurances through control of changes made to hardware, software, firmware, documentation, test, test fixtures, and test documentation throughout the life cycle of an information system.

ISO (International Standards Organization) definition of CM:
- Configuration management (CM) provides a method to identify, build, move, control and recover any baseline in any part of the life cycle, and ensures that is secure and free from external corruption.
- Configuration identification requires that all configuration items (CI) and their versions in test system are known.
- Configuration control is maintenance of CI’s in a library and maintenance of records on how CI’s change over time.
- Status accounting is the function of recording and tracking problem reports, change requests, etc.
- Configuration auditing is the function to check on the contents of libraries, etc. for standards compliance, for instance.
- CM can be very complicated in environments where mixed hardware and software platforms are being used, but sophisticated cross platform CM tools are increasingly available.

Traditionally configuration management has four elements:
1 - Configuration identification
2 - Configuration change control (or change management)
3 - Configuration status accounting
4 - Configuration verification and auditing
These terms and definitions change from standard to standard, but are essentially the same.

Configuration identification:
Configuration identification is the process of identifying the attributes that define every aspect of a configuration item. A configuration item is a product (hardware and/or software) that has an end-user purpose. These attributes are recorded in configuration documentation and baselined. Baselining an attribute forces formal configuration change control processes to be effected in the event that these attributes are changed.

Configuration change control:
Configuration change control is a set of processes and approval stages required to change a configuration item's attributes and to re-baseline them.

Configuration status accounting:
Configuration status accounting is the ability to record and report on the configuration baselines associated with each configuration item at any moment of time.

Configuration audits:
Configuration audits are broken into functional and physical configuration audits. They occur either at delivery or at the moment of effecting the change. A functional configuration audit ensures that functional and performance attributes of a configuration item are achieved, while a physical configuration audit ensures that a configuration item is installed in accordance with the requirements of its detailed design documentation.

Top level Configuration Management Activity Model:
(Click on Image to Enlarge)

Deliverables:
The Deliverables from the Test team would include the following:
1. Test Plan.
2. Test Case Documents.
3. Defect Reports.
4. Status Reports (Daily/weekly/Monthly).
5. Test Scripts (if any).
6. Metric Reports.
7. Product Sign off Document.

Defect Tracking Process

2008-07-03T01:06:00.000-07:00

The Defect Tracking process should answer the following questions:
1. When is the defect found?
2. Who raised the defect?
3. Is the defect reported properly?
4. Is the defect assigned to the appropriate developer?
5. When was the defect fixed?
6. Is the defect re-tested?
7. Is the defect closed?
The defect tracking process has to be handled carefully and managed efficiently.

The following figure illustrates the defect tracking process:
(Click Image to Enlarge)

Defect Classification:
This section defines a defect Severity Scale framework for determining defect criticality and the associated defect Priority Levels to be assigned to errors found software.

The defects can be classified as follows:
CRITICAL - There is a functionality block or the defect is a show stopper. The application is not able to proceed any further and testing process cannot proceed further till the critical defect is fixed.
MAJOR - The application is not working as desired. There are variations in the functionality.
MINOR - There is no failure reported due to the defect, but certainly needs to be rectified.
TRIVIAL - Defects will less importance or insignificant.
COSMETIC - Defects in the User Interface or Navigation.
SUGGESTION - Feature which can be added for betterment.

Priority Level of the Defect:
The priority level describes the time for resolution of the defect. The priority level would be classified as follows:

IMMEDIATE - Resolve the defect with immediate effect.
HIGH - Resolve the defect at the earliest, on priority at the second level.
MEDIUM - Resolve the defect.
LOW - Could be resolved at the later stages.

Bug Lifecycle

2008-07-03T00:32:00.000-07:00

A Bug Lifecycle can be described in the following 6 steps:

Step:1 Bug is identified in system and logged in bug tracking tool.
Step:2 Bug is assigned to a developer
Step:3 Developer resolves bug or clarifies with user
Step:4 Developer sets bug to "Completed" status and assigns back to original user
Step:5 Original user verifies that bug has been resolved and if so sets bug to "Closed" status. Only the original user who created the bug has access to "Close" the bug. Once the bug is closed it may never be re-opened or have future activity.
Step:6 If the bug was not resolved to the user's satisfaction they may assign it back to the developer with a description (by adding a new detail). If this occurs then the bug returns to step 2 above.

It is important to note that throughout the lifecycle of a bug, it should be assigned to someone. The system will allow for a bug to not be assigned but the usage of this feature should be minimal. By insuring that a bug is always assigned to a user or a developer, system administrators will maintain a high level of accountability for all bugs submitted to the system.

Simple Bug Lifecycle with stages:
(Click On Image to Enlarge)

Descriptive Bug Lifecycle:
(Click On Image to Enlarge)

Testing Terminology

2008-07-02T04:29:00.004-07:00

1.0 Testing

New standard -
There is a lot of terminology surrounding testing, but not until recently has there been an industry standard. A new standard (first published in August 1998) seeks to provide a standard set of terms: BS 7925-i Glossary of Testing Terms. Although a British Standard, it is being adopted by the International Standards Organization (ISO) and will hopefully become an ISO standard within two or three years. A selection of terms from this standard, plus some others used in this course are in the Glossary at the back of these notes.

Error, fault and failure:
Three terms that have specific meanings are error, fault and failure.

Error: a human action that produces an incorrect result.
Fault: a manifestation of an error in software.
Failure: a deviation of the software from its expected delivery or service.

An error is something that a human does, we all make mistakes and when we do whilst developing software it is known as an error. The result of an error being made is a fault. It is something that is wrong in the software (source code or documentation - specifications, manuals, etc.). Faults are also known as a defects or bugs but in this course we will use the term fault.

When a system or piece of software produces an incorrect result or does not perform the correct action this is known as a failure. Failures are caused by faults in the software. Note that software system can contain faults but still never fail (this can occur if the faults are in those parts of the system that are never used).

Reliability:

Another term that should be understood is reliability. A system is said to be reliable when it performs correctly for long periods of time. However, the same system used by two different people may appear reliable to one but not to the other. This is because the different people use the system in different ways.

Reliability: the probability that the software will not cause the failure of the system for a specified time under specified conditions.
The definition of reliability therefore includes the phrase ‘under specified conditions’. When reporting on the reliability of a system it is important to explain under what conditions the system will achieve the specified level of reliability. For example, a system may achieve a reliability of no more than one failure per month providing no more than 10 people use the system simultaneously

Acceptance criteria -
The criteria that a system or component must satisfy in order to be accepted by a user, customer, or other authorized entity.
Acceptance testing-
Formal tests conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept a system.
Application domain -
A bounded set of related systems (i.e., systems that address a particular type of problem). Development and maintenance in an application domain usually requires special skills and/or resources. Examples include payroll and personnel systems, command and control systems, compilers, and expert systems.
Audit -
An independent examination of a work product or set of work products to assess compliance with specifications, standards, contractual agreements, or other criteria.
Baseline -
A specification or product that has been formally reviewed and agreed upon, that thereafter serves as the basis for further development, and that can be changed only through formal change control procedures.
Black-box testing-
Testing based on functional requirements without knowledge of the internal program structures and data. Also called functional testing.
Boundary Value-
An input value or output value which is on the boundary between equivalence classes, or an incremental distance either side of the boundary.
Boundary Value Analysis-
A test case design technique for a component in which test cases are designed which include representatives of boundary values.
Branch testing-
A test method satisfying coverage criteria that require that for each decision point, each possible branch be executed at least once
Code Coverage-
An analysis method that determines which parts of the software have been executed (covered) by the test case suite and which parts have not been executed and therefore may require additional attention
Component-
A part of a software system smaller than the entire system but larger than an element.
Customer - The individual or organization that is responsible for accepting the product and authorizing payment to the developing organization.
Cyclomatic complexity-
The number of linearly independent paths through a program. The cyclomatic complexity of a program is equivalent to the number of decision statements plus 1.
Defect-
Nonconformance to requirements.
(Or)
A flaw in a system or system component that causes the system or component to fail to perform its required function. A defect, if encountered during execution, may cause a failure of the system.

Defect density-
Ratio of the number of defects to program length.
(Or)
The number of defects identified in a product divided by the size of the product component (expressed in standard measurement terms for that product).
Deviation -
A noticeable or marked departure from the appropriate norm, plan, standard, procedure, or variable being reviewed.
Dynamic Analysis-
The process of evaluating a system or component based on its behavior during execution.
Error-
The difference between a computed, observed, or measured value or condition and the true, specified, or theoretically correct value or condition.
End user - The individual or group who will use the system for its intended operational use when it is deployed in its environment.
Equivalence class partitioning-
Partitioning the input domain of a program into a finite number of classes [sets], to identify a minimal set of well selected test cases to represent these classes. There are two types of input equivalence classes, valid and invalid.
Integration Testing-
Exposes faults during the process of integration of software components, or software units and it is specifically aimed at exposing faults in their interactions. The integration approach could be either bottom-up (using drivers), top-down (using stubs) or a mixture of the two. Also known as interface testing.
Interface-
The informational boundary between two software systems, software system components, elements, or modules.
Path testing-
A test method satisfying coverage criteria that each logical path through the program be tested. Often paths through the program are grouped into a finite set of classes; one path from each class is tested.
Pseudo code-
A form of software design in which programming actions are described in a program-like structure; not necessarily executable but generally held to be humanly readable.
Quality -
(1) The degree to which a system, component, or process meets specified requirements.
(2) The degree to which a system, component, or process meets customer or user needs or expectations.
Regression Testing-
Re-testing after fixes or modifications of the software or its environment. Automated testing tools can be especially useful for this type of testing.
Required training -
Training designated by an organization to be required to perform a specific role.
Risk -
Possibility of suffering loss.
Risk management -
An approach to problem analysis which weighs risk in a situation by using risk probabilities to give a more accurate understanding of the risks involved. Risk management includes risk identification, analysis, prioritization, and control.
Risk management plan -
The collection of plans that describe the risk management activities to be performed on a project.
Role -
A unit of defined responsibilities that may be assumed by one or more individuals.
Software life cycle -
The period of time that begins when a software product is conceived and ends when the software is no longer available for use. The software life cycle typically includes a concept phase, requirements phase, design phase, implementation phase, test phase, installation and checkout phase, operation and maintenance phase, and, sometimes, retirement phase.
Statement testing-
Testing designed to execute each statement of a computer program. See test coverage.
Static Analysis-
Examination of the form and structure of a product without executing the product. It may be applied to requirements, design, or code.
Stress testing-
A test which exercises code up to, including and beyond all stated limits in order to exercise all aspects of the system (e.g., to include hardware, software, and communications). Its purpose is to insure that response times and storage capacities meet requirements.
System testing-
It primarily demonstrates that the software system does fulfill requirements specified in the requirement specification during exposure to the anticipated environmental conditions. All testing objectives relevant to specific requirements should be included during the software system testing. System testing includes testing of performance, security, configuration sensitivity, stress, startup and recovery from failure modes
Testing-
Testing is the execution of a system in a real or simulated environment with the intent of finding faults
Test case-
A set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.
Test design-
Documentation specifying the details of the test approach for a software feature or combination of software features and identifying the associated tests.
Test plan-
A document describing the scope, approach, resources, and schedule of intended test activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning.
Test procedure-
The formal or informal procedure that will be followed to execute the test case. This is usually a written document that will allow others to carry out the test with a minimum of training and confusion.
Unit-
The smallest piece of software that can be independently tested (i.e., compiled or assembled, loaded, and tested).
Unit Testing-
Unit testing is meant to expose faults on each software unit as soon as this is available regardless of its interaction with other units. The unit is exercised against its detailed design and by ensuring that defined logic coverage is performed. White-box oriented testing in combination with at least one black box method is used.
Validation-
The process of evaluation software at the end of the software development process to ensure compliance with software requirements.
Verification-
The Process of evaluating a system or component to determine whether the product at a given development phase satisfy the conditions imposed at the start of that phase.
White-box testing-
A testing method to test the internal behavior and structure of the program. The testing strategy permits one to examine the internal structure of the program. In using this strategy, the tester derives test data from an examination of the program's logic without neglecting the requirements in the specification. The goal of this test method is to achieve test coverage, that is examination of as much of the statements, branches, paths as possible.

2.0 CMM

Activity -
Any step taken or function performed, both mental and physical, toward achieving some objective. Activities include all the work the managers and technical staff do to perform the tasks of the project and organization.
CMM:
Framework representing a path of improvements recommended for software organizations that want to increase their software process capability.
(Or)
A description of the stages through which software organizations evolve as they define, implement, measure, control, and improve their software processes. This model provides a guide for selecting process improvement strategies by facilitating the determination of current process capabilities and the identification of the issues most critical to software quality and process improvement.
Commitment -
A pact that is freely assumed, visible, and expected to be kept by all parties.
Common features-
The key practices are divided among five Common Features sections namely, Commitment to perform, Ability to Perform, Activities Performed, Measurement and Analysis, and Verifying Implementation. The common features are attributes that indicate whether the implementation and institutionalization of a key process area is effective, repeatable, and lasting.
Commitment to perform -
The actions the organization must take to ensure that the process is established and will endure. Commitment to Perform typically involves establishing organizational policies and senior management sponsorship.
Ability to perform -
The preconditions that must exist in the project or organization to implement the software process competently. Ability to Perform typically involves resources, organizational structures, and training.
Activities performed -
A description of the roles and procedures necessary to implement a key process area. Activities Performed typically involve establishing plans and procedures, performing the work, tracking it, and taking corrective actions as necessary.
Measurement and analysis -
A description of the need to measure the process and analyze the measurements. Measurement and Analysis typically includes examples of the measurements that could be taken to determine the status and effectiveness of the Activities Performed.
Verifying implementation -
The steps to ensure that the activities are performed in compliance with the process that has been established. Verification typically encompasses reviews and audits by management and software quality assurance.
Dependency item -
A product, action, piece of information, etc., that must be provided by one individual or group to a second individual or group so that the second individual or group can perform a planned task.
Effective process -

A process that can be characterized as practiced, documented, enforced, trained, measured, and able to improve.

Goals-
Summarize the key practices of a key process area and can be used to determine whether an organization or project has effectively implemented the key process area. The goals signify the scope, boundaries, and intent of each key process area.
Institutionalization-
Building an infrastructure and a corporate culture that supports the methods, practices and procedures of the business so that they endure after those who originally defined them have gone.
Key practices-
Each key process area is described in terms of key practices that, when implemented, help to satisfy the goals of that key process area. The key practices describe the infrastructure and activities that contribute most to the effective implementation and institutionalization of the key process area.
Key process area-
Each maturity level is composed of key process areas. Each key process area identifies
a cluster of related activities that, when performed collectively, achieve a set of goals considered important for enhancing process capability. For convenience, the key process areas are organized by common features.
Maturity level-
Well-defined evolutionary plateau toward achieving a mature software process. Each maturity level indicates a level of process capability.

The five maturity levels in the SEI's Capability Maturity Model are:
LEVEL-1 : Initial - The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort.
LEVEL-2 : Repeatable - Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.
LEVEL -3 : Defined - The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization's standard software process for developing and maintaining software.
LEVEL -4 : Managed - Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.
LEVEL -5 : Optimizing - Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.

Measure - A unit of measurement (such as source lines of code or document pages of design).
Measurement - The dimension, capacity, quantity, or amount of something (e.g., 300 source lines of code or 7 document pages of design).
Method - A reasonably complete set of rules and criteria that establish a precise and repeatable way of performing a task and arriving at a desired result.
Methodology - A collection of methods, procedures, and standards that defines an integrated synthesis of engineering approaches to the development of a product.
Milestone - A scheduled event for which some individual is accountable and that is used to measure progress.
Peer review - A review of a software work product, following defined procedures, by peers of the producers of the product for the purpose of identifying defects and improvements.
Peer review leader - An individual specifically trained and qualified to plan, organize, and lead a peer review.
Periodic review/activity - A review or activity that occurs at specified regular time intervals. (See event-driven review/activity for contrast.)
Policy - A guiding principle, typically established by senior management, which is adopted by an organization or project to influence and determine decisions.
Procedure - A written description of a course of action to be taken to perform a given task. [IEEE-STD-610]
Process - A sequence of steps performed for a given purpose; for example, the software development process.
Software capability evaluations:
To identify contractors who are qualified to perform the software work or to monitor the state of the software process used on an existing software effort.
Software process:
A set of activities, methods, practices, and transformations that people use to develop and maintain software and the associated products.
Software process assessments:
To determine the state of an organization's current software process, to determine the highpriority software process-related issues facing an organization, and to obtain the organizational support for software process improvement.
Software process capability:
Describes the range of expected results that can be achieved by following a software process.
Software process maturity:
Is the extent to which a specific process is explicitly defined, managed, measured, controlled, and effective.
Software process performance:
Represents the actual results achieved by following a software process. Thus, software process performance focuses on the results achieved, while software process capability focuses on results expected.
Software product -
The complete set, or any of the individual items of the set, of computer programs, procedures, and associated documentation and data designated for delivery to a customer or end user.

Question Bank-1

2008-07-01T06:01:00.000-07:00

1 -We split testing into distinct stages primarily because:
a) Each test stage has a different purpose.
b) It is easier to manage testing in stages.
c) We can run different tests in different environments.
d) The more stages we have, the better the testing.

2- Which of the following is likely to benefit most from the use of test tools providing test capture and replay facilities?
a) Regression testing
b) Integration testing
c) System testing
d) User acceptance testing

3 - Which of the following statements is NOT correct?
a) A minimal test set that achieves 100% LCSAJ coverage will also achieve 100% branch coverage.
b) A minimal test set that achieves 100% path coverage will also achieve 100% statement coverage.
c) A minimal test set that achieves 100% path coverage will generally detect more faults than one that achieves 100% statement coverage.
d) A minimal test set that achieves 100% statement coverage will generally detect more faults than one that achieves 100% branch coverage.

4 -Which of the following requirements is testable?
a) The system shall be user friendly.
b) The safety-critical parts of the system shall contain 0 faults.
c) The response time shall be less than one second for the specified design load.
d) The system shall be built to be portable.

5 - Analyse the following highly simplified procedure:
Ask: “What type of ticket do you require, single or return?”
IF the customer wants ‘return’
Ask: “What rate, Standard or Cheap-day?”
IF the customer replies ‘Cheap-day’
Say: “That will be £11:20”
ELSESay: “That will be £19:50”
ENDIF
ELSESay: “That will be £9:75”
ENDIF
Now decide the minimum number of tests that are needed to ensure that allthe questions have been asked, all combinations have occurred and all replies given.
a) 3
b) 4
c) 5
d) 6

6- Error guessing:
a) supplements formal test design techniques.
b) can only be used in component, integration and system testing.
c) is only performed in user acceptance testing.
d) is not repeatable and should not be used.

7- Which of the following is NOT true of test coverage criteria?
a) Test coverage criteria can be measured in terms of items exercised by a test suite.
b) A measure of test coverage criteria is the percentage of user requirements covered.
c) A measure of test coverage criteria is the percentage of faults found.
d) Test coverage criteria are often used when specifying test completion criteria.

8 -In prioritising what to test, the most important objective is to:
a) find as many faults as possible.
b) test high risk areas.
c) obtain good test coverage.
d) test whatever is easiest to test.

9- Given the following sets of test management terms (v-z), and activity descriptions (1-5), which one of the following best pairs the two sets?
v – test control
w – test monitoring
x - test estimation
y - incident management
z - configuration control

1 - calculation of required test resources
2 - maintenance of record of test results
3 - re-allocation of resources when tests overrun
4 - report on deviation from test plan
5 - tracking of anomalous test results

a) v-3,w-2,x-1,y-5,z-4
b) v-2,w-5,x-1,y-4,z-3
c) v-3,w-4,x-1,y-5,z-2
d) v-2,w-1,x-4,y-3,z-5

10 - Which one of the following statements about system testing is NOT true?
a) System tests are often performed by independent teams.
b) Functional testing is used more than structural testing.
c) Faults found during system tests can be very expensive to fix.
d) End-users should be involved in system tests.

11- Which of the following is false?
a) Incidents should always be fixed.
b) An incident occurs when expected and actual results differ.
c) Incidents can be analysed to assist in test process improvement.
d) An incident can be raised against documentation.

12 -Enough testing has been performed when:
a) time runs out.
b) the required level of confidence has been achieved.
c) no more faults are found.
d) the users won’t find any serious faults.

13- Which of the following is NOT true of incidents?
a) Incident resolution is the responsibility of the author of the software under test.
b) Incidents may be raised against user requirements.
c) Incidents require investigation and/or correction.
d) Incidents are raised when expected and actual results differ.

14- Which of the following is not described in a unit test standard?
a) syntax testing
b) equivalence partitioning
c) stress testing
d) modified condition/decision coverage

15- Which of the following is false?
a) In a system two different failures may have different severities.
b) A system is necessarily more reliable after debugging for the removal of a fault.
c) A fault need not affect the reliability of a system.
d) Undetected errors may lead to faults and eventually to incorrect behaviour.

16 -Which one of the following statements, about capture-replay tools, is NOT correct?
a) They are used to support multi-user testing.
b) They are used to capture and animate user requirements.
c) They are the most frequently purchased types of CAST tool.
d) They capture aspects of user behaviour.

17 - How would you estimate the amount of re-testing likely to be required?
a) Metrics from previous similar projects
b) Discussions with the development team
c) Time allocated for regression testing
d) a & b

18- Which of the following is true of the V-model?
a) It states that modules are tested against user requirements.
b) It only models the testing phase.
c) It specifies the test techniques to be used.
d) It includes the verification of designs.

19 -The oracle assumption:
a) is that there is some existing system against which test output may be checked.
b) is that the tester can routinely identify the correct outcome of a test.
c) is that the tester knows everything about the software under test.
d) is that the tests are reviewed by experienced testers.

20 -Which of the following characterises the cost of faults?
a) They are cheapest to find in the early development phases and the most expensive to fix in the latest test phases.
b) They are easiest to find during system testing but the most expensive to fix then.
c) Faults are cheapest to find in the early development phases but the most expensive to fix then.
d) Although faults are most expensive to find during early development phases, they are cheapest to fix then.

21- Which of the following should NOT normally be an objective for a test?
a) To find faults in the software.
b) To assess whether the software is ready for release.
c) To demonstrate that the software doesn’t work.
d) To prove that the software is correct.

22- Which of the following is a form of functional testing?
a) Boundary value analysis
b) Usability testing
c) Performance testing
d) Security testing

23 -Which of the following would NOT normally form part of a test plan?
a) Features to be tested
b) Incident reports
c) Risks
d) Schedule

24- Which of these activities provides the biggest potential cost saving from the use of CAST?
a) Test management
b) Test design
c) Test execution
d) Test planning

25 -Which of the following is NOT a white box technique?
a) Statement testing
b) Path testing
c) Data flow testing
d) State transition testing

26- Data flow analysis studies:
a) possible communications bottlenecks in a program.
b) the rate of change of data values as a program executes.
c) the use of data on paths through the code.
d) the intrinsic complexity of the code.

27- In a system designed to work out the tax to be paid:An employee has £4000 of salary tax free. The next £1500 is taxed at 10%The next £28000 is taxed at 22%Any further amount is taxed at 40%To the nearest whole pound, which of these is a valid Boundary Value Analysis test case?
a) £1500
b) £32001
c) £33501
d) £28000

28 - An important benefit of code inspections is that they:
a) enable the code to be tested before the execution environment is ready.
b) can be performed by the person who wrote the code.
c) can be performed by inexperienced staff.
d) are cheap to perform.

29 -Which of the following is the best source of Expected Outcomes for User Acceptance Test scripts?
a) Actual results
b) Program specification
c) User requirements
d) System specification

30- What is the main difference between a walkthrough and an inspection?
a) An inspection is lead by the author, whilst a walkthrough is lead by a trained moderator.
b) An inspection has a trained leader, whilst a walkthrough has no leader.
c) Authors are not present during inspections, whilst they are during walkthroughs.
d) A walkthrough is lead by the author, whilst an inspection is lead by a trained moderator.

31 -Which one of the following describes the major benefit of verification early in the life cycle?
a) It allows the identification of changes in user requirements.
b) It facilitates timely set up of the test environment.
c) It reduces defect multiplication.
d) It allows testers to become involved early in the project.

32- Integration testing in the small:
a) tests the individual components that have been developed.
b) tests interactions between modules or subsystems.
c) only uses components that form part of the live system.
d) tests interfaces to other systems.

33 -Static analysis is best described as:
a) the analysis of batch programs.
b) the reviewing of test plans.
c) the analysis of program code.
d) the use of black box testing.

34 -Alpha testing is:
a) post-release testing by end user representatives at the developer’s site.
b) the first testing that is performed.
c) pre-release testing by end user representatives at the developer’s site.
d) pre-release testing by end user representatives at their sites.

35 -A failure is:
a) found in the software; the result of an error.
b) departure from specified behaviour.
c) an incorrect step, process or data definition in a computer program.
d) a human action that produces an incorrect result.

36- In a system designed to work out the tax to be paid:An employee has £4000 of salary tax free. The next £1500 is taxed at 10%The next £28000 is taxed at 22%Any further amount is taxed at 40%Which of these groups of numbers would fall into the same equivalence class?
a) £4800; £14000; £28000
b) £5200; £5500; £28000
c) £28001; £32000; £35000
d) £5800; £28000; £32000

37- The most important thing about early test design is that it:
a) makes test preparation easier.
b) means inspections are not required.
c) can prevent fault multiplication.
d) will find all faults.

38- Which of the following statements about reviews is true?
a) Reviews cannot be performed on user requirements specifications.
b) Reviews are the least effective way of testing code.
c) Reviews are unlikely to find faults in test plans.
d) Reviews should be performed on specifications, code, and test plans.

39- Test cases are designed during:
a) test recording.
b) test planning.
c) test configuration.
d) test specification.

40- A configuration management system would NOT normally provide:
a) linkage of customer requirements to version numbers.
b) facilities to compare test results with expected results.
c) the precise differences in versions of software component source code.
d) restricted access to the source code library.

ANSWERS - Question Bank -1