Software Testing Fundamentals

Various Levels of SoftwareTesting

noreply@blogger.com (Ritesh Kumar) — Fri, 11 Mar 2011 11:44:00 +0000

Testing can be done on the following levels:

Component or Unit Testing: Unit testing tests the minimal software component, or module. Each unit (basic component) of the software is tested to verify that the detailed design for the unit has been correctly implemented. In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the constructors and destructors.
Integration Testing: Integration testing exposes defects in the interfaces and interaction between integrated components (modules). Progressively larger groups of tested software components corresponding to elements of the architectural design are integrated and tested until the software works as a system
System Testing: System testing tests a completely integrated system to verify that it meets its requirements
Acceptance Testing: Finally, acceptance testing can be conducted by the end-user, customer, or client to validate whether or not to accept the product. Acceptance testing may be performed as part of the hand-off process between any two phases of development

Difference between Alpha Testing & Beta Testing

noreply@blogger.com (Ritesh Kumar) — Tue, 01 Mar 2011 18:39:00 +0000

These are the testing done by the users. Once the developers have finished their own testing with the help of other professional developers, the software is ready for release to the public or to the intended users.
The stages usually followed are:

Alpha Testing – Developers usually choose the users who will try out the program. These users can be any potential users/customers or an independent test team who carry out the testing at developer's site. It is often tested on known users or even technology bloggers who can honestly give their opinion of the software. It is also the time where developers try to create scenarios for their software such as:

Recovery – Developers will try recovering the software or the program in case it crashes.
Security – Like all software, developers have to test the program for possible hacks and virus infections.
Performance – Use it extensively by getting more than one users to exhaust all possible options and see if the software will hold up to expectations.

Beta Testing – This type of testing is more open to the public and is done outside the developer's site. Developers or software companies can accept users sometimes by invitation or openly accepting users. Beta testing is tested in a real environment where there is a bigger possibility of hacks, lack of performance or bugs. The main reason the software is given to the public for testing is to open it for possible hacks and feedback about the software. If something wrong is found, developers can just go back to the software easily. Versions of software that is given for beta testing are known as Beta Versions.

Types of Testing: Testing by Developers

noreply@blogger.com (Ritesh Kumar) — Tue, 01 Mar 2011 18:11:00 +0000

Developers will have to test the software themselves extensively. Even though they are also the one who have developed the software, they still have to run it extensively. These are the usual steps developers follow in testing their software:

Checking of Syntax – The software is checked as it is. Developers just run software and look for possible errors. A very basic checking to know if there are major functions and errors that will occur.
Code inspection – After the execution, developers review their codes to look for possible bugs or errors in coding technique. Even though it has worked in initial checking, the language has to be checked extensively to predict possible problems in some executions.
Desk Checking – In this part of the testing, developers check the code and the software at the same time. Each line of command is tested if the desired output is met. This is done manually as each line is expected to work and one way of making sure it will work is to manually check each line so that developers will know what output the line has created.
Walkthrough – After checking the software themselves, its time to present the software to the experts. Not the upper management but possibly experts on the coding language or other developers of this type of software. The developer will present their work to the pool of professionals who will scrutinize their work. It is also important for the developer to present not only the coding technique but also program itself. Using simple case studies the software will somehow provide answers faster or with a better result.
Module Testing – After the walk through, developers will again check the software in terms of individual modules. Each module will be tested extensively for possible errors.This is also known as Unit Testing.
Integration Testing – Once the module has been tested, it is time to test them as being integrated to other modules. In this type of testing, developer checks that interface between the two module is working properly.
System Testing – After the integration, the whole program will again be tested. Case studies will again be applied with integrated software.

Gray Box Technique

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 09:11:00 +0000

Gray box testing is a software testing technique that uses a combination of black box testing and white box testing. Gray box testing is not black box testing, because the tester does know some of the internal workings of the software under test. In gray box testing, the tester applies a limited number of test cases to the internal workings of the software under test. In the remaining part of the gray box testing, one takes a black box approach in applying inputs to the software under test and observing the outputs.

Gray box testing is a powerful idea. The concept is simple; if one knows something about how the product works on the inside, one can test it better, even from the outside. Gray box testing is not to be confused with white box testing; i.e. attesting approach that attempts to cover the internals of the product in detail. Gray box testing is a test strategy based partly on internals. The testing approach is known as gray box testing, when one does have some knowledge, but not the full knowledge of the internals of the product one is testing.

In gray box testing, just as in black box testing, you test from the outside of a product, just as you do with black box, but you make better-informed testing choices because you're better informed; because you know how the underlying software components operate and interact.

Conclusion about White Box Testing

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 09:09:00 +0000

White box testing for security is useful and effective. It should follow a risk-based approach to balance the testing effort with consequences of software failure. Architectural and design-level risk analysis provides the right context to plan and perform white box testing. White box testing can be used with black box testing to improve overall test effectiveness. It uncovers programming and implementation errors.

This paper introduces a risk-based approach and tools and techniques applicable to white box testing for security. Concepts and knowledge from two areas, traditional white box testing and security-based testing, were brought together. Other content areas on this web portal discuss different aspects of software security in detail. To gain more in-depth understanding of white box testing, readers are urged to supplement the knowledge gained here with other related areas available on this web site. The links to related topics are given in the Links section.

Case Study for White Box Testing

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 09:07:00 +0000

A large merchant organization involved in online business was in the process of developing an online e-commerce web site. In an effort to allow customers to electronically and efficiently transfer funds from customer checking accounts to merchant accounts, the merchant organization had outsourced its payment processing to a third-party Internet-enabled financial transaction payment firm. The third-party payments software provided customized interfaces to facilitate payment processing between the customers and the merchant organization.

A high-level security risk analysis was conducted on the system. Risk assessment identified transactions processing between the payments interface and the application as one of the risks. The impact of fraudulent transactions is serious for both the customers and for the merchant organization. The customers could suffer significant financial loss and hardships resulting from unauthorized transactions that could deplete account balances. The credibility and the reputation of the merchant organization could be severely damaged as a result of fraudulent transactions should they become publicized.

A thorough white box testing was conducted on the modules using the payments interface. First, all the component interfaces were identified and illustrated as interface diagrams; second, trust relationship boundaries were drawn on the component interactions; and third, data flows between the components were drawn. Abuse cases were developed based on this information. One of the abuse cases pointed to exercising the payments processing functionality as an anonymous user. The trust relationship mapping and data flow showed a path where inputs from the users were not validated or authenticated. A test case was developed to submit an account transfer from an external account to the merchant account anonymously and the account transfer was completed successfully, which is a critical software failure: unauthorized transactions via unauthenticated channel were allowed.

Risk assessment also identified a weak authentication component in the payment customer service component of the system. As in the case above, trust relationship boundaries and data-flow analysis were conducted on this component. After analysis and testing, it was shown that an attacker could gain direct access to the merchant administrative accounts. With this access, an attacker could redirect transactions from a merchant account to another, non-merchant account.

Using the two exploits described above, white box testers showed that an attacker could funnel payments from an unwitting customer to the merchant account and then redirect the transaction from the merchant account to a non-merchant account. The two bugs put together form a serious breach of security with significant business impact.

This example shows that risk analysis yields business-relevant results and points to specific vulnerable areas of the software. Subsequent to risk analysis, performing white box testing in concentrated areas aids in quickly uncovering design assumptions and implementation errors. This example also shows that finding an exploit does not mean that the job is done. Collecting information about various bugs in the software and analyzing them together shows how multiple exploits can be stringed together to form a full attack.

Skills and Training required for White box Testing

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 09:04:00 +0000

White box testing requires knowledge of software security design and coding practices, an understanding of an attacker’s mindset, knowledge of known attack patterns, vulnerabilities and threats, and the use of different testing tools and techniques. White box testing brings together the skills of a security developer, an attacker, and a tester.

Books like Writing Secure Code, 19 Deadly Sins, and Building Secure Software help educate software professionals on how to write secure software, and books like How to Break Software Security and Exploiting Software help educate professionals on how to think like an attacker. There are several good books on software testing, including the two classics Software Testing Techniques and Testing Computer Software. There are several valuable information sites that detail known vulnerabilities, attack patterns, security tools, etc. White box testing is knowledge intensive and relies on expertise and experience.

Alternatives to White Box Testing

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 09:02:00 +0000

There are alternatives to white box testing:

White box testing can complement black box testing to increase overall test effectiveness. Based on risk assessment, certain areas of the software may require more scrutiny than others. White box testing could be performed for specific high-risk areas, and black box testing could be performed for the whole system. By complementing the two testing methods, more tests can be developed, focusing on both implementation issues and usage issues.
Gray box testing can be used to combine both white box and black box testing methods in a powerful way. In a typical case, white box analysis is used to find vulnerable areas, and black box testing is then used to develop working attacks against these areas. The white box analysis increases productivity in finding vulnerable areas, while the black box testing method of driving data inputs decreases the cost of test setup and test execution.

All testing methods can reveal possible software risks and potential exploits. White box testing directly identifies more bugs in the software. White box testing is time consuming and expensive and requires specialized skills. As with any testing method, white box testing has benefits, associated costs, and alternatives. An effective testing approach balances efficiency and effectiveness in order to identify the greatest number of critical defects for the least cost.

Cost Drivers for White Box Testing

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:59:00 +0000

The main cost drivers for white box testing are the following:

specialized skill requirements: White box testing is knowledge intensive. White box testers should not only know how to analyze code for security issues but also understand different tools and techniques to test the software. Security testing is not just validating designed functionality but also proving that the defensive mechanisms work correctly. This requires invaluable experience and expertise. Testers who can perform such tasks are expensive and hard to get.
support software development and tools: White box testing requires development of support software and tools to perform testing. Both the support software and the tools are largely based on the context of the software under test and the type of test technique employed. The type of tools used includes program understanding tools, coverage tools, fault injection tools, and source code analyzers.
analysis and testing time: White box testing is time consuming, especially when applied to the whole system. Analyzing design and source code in detail for security testing is time consuming, but is an essential part of white box testing. Tools (source code analyzers, debuggers, etc.) and program understanding techniques (flow graphs, data-flow graphs, etc.) help in speeding up analysis. White box testing directly identifies implementation bugs, but whether the bugs can be exploited requires further analysis work. The consequences of failure help determine the amount of testing time and effort dedicated to certain areas.

Benefits of White Box Testing

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:56:00 +0000

There are many benefits to white box testing, including the following:

Analyzing source code and developing tests based on the implementation details enables testers to find programming errors quickly. For example, a white box tester looking at the implementation can quickly uncover a way, say, through an error handling mechanism, to expose secret data processed by a component. Finding such vulnerabilities through black box testing require comparatively more effort than found through white box testing. This increases the productivity of testing effort.
Executing some (hard to set up) black box tests as white box tests reduces complexity in test setup and execution. For example, to drive a specific input into a component, buried inside the software, may require elaborate setup for black box testing but may be done more directly through white box testing by isolating the component and testing it on its own. This reduces the overall cost (in terms of time and effort) required to perform such tests.
Validating design decisions and assumptions quickly through white box testing increases effectiveness. The design specification may outline a secure design, but the implementation may not exactly capture the design intent. For example, a design might outline the use of protected channels for data transfer between two components, but the implementation may be using an unprotected method for temporary storage before the data transfer. This increases the productivity of testing effort.
Finding”unintended” features can be quicker during white box testing. Security testing is not just about finding vulnerabilities in the intended functionality of the software but also about examining unintended functionality introduced during implementation. Having access to the source code improves understanding and uncovering the additional unintended behavior of the software. For example, a component may have additional functions exposed to support interactions with some other component, but the same functions may be used to expose protected data from a third component. Depending on the nature of the”unintended” functionality, it may require a lot more effort to uncover such problems through black box testing alone.

Business Case

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:48:00 +0000

The goal of security testing is to ensure the robustness of the software under test, even in the presence of a malicious attack. The designers and the specification might outline a secure design, the developers might be diligent and write secure code, but it’s the testing process that determines whether the software is secure in the real world. Testing is an essential form of assurance. Testing is laborious, time consuming, and expensive, so the choice of testing (black box, or white box, or a combination) should be based on the risks to the system. Risk analysis provides the right context and information to make tradeoffs between time and effort to achieve test effectiveness.

White box testing is typically very effective in validating design decisions and assumptions and finding programming errors and implementation errors in software. For example, an application may use cryptography to secure data from specific threats, but an implementation error such as a poor key management technique can still leave the application vulnerable to security violations. White box testing can uncover such implementation errors.

Results to Expect

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:45:00 +0000

Any security testing method aims to ensure that the software under test meets the security goals of the system and is robust and resistant to malicious attacks. Security testing involves taking two diverse approaches: one, testing security mechanisms to ensure that their functionality is properly implemented; and two, performing risk-based security testing motivated by understanding and simulating the attacker’s approach. White box security testing follows both these approaches and uncovers programming and implementation errors. The types of errors uncovered during white box testing are several and are very context sensitive to the software under test. Some examples of errors uncovered include

data inputs compromising security
sensitive data being exposed to unauthorized users
improper control flows compromising security
incorrect implementations of security functionality
unintended software behavior that has security implications
design flaws not apparent from the design specification
boundary limitations not apparent at the interface level

White box testing greatly enhances overall test effectiveness and test coverage. It can greatly improve productivity in uncovering bugs that are hard to find with black box testing or other testing methods alone.

WBT Tool: Program Understanding Tools

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:42:00 +0000

In general, white box testers should have access to the same tools, documentation, and environment as the developers and functional testers on the project do. In addition, tools that aid in program understanding, such as software visualization, code navigation, debugging, and disassembly tools, greatly enhance productivity during testing.

Coverage Analysis:

Code coverage tools measure how thoroughly tests exercise programs. There are many different coverage measures, including statement coverage, branch coverage, and multiple-condition coverage. The coverage tool would modify the code to record the statements executed and which expressions evaluate which way (the true case or the false case of the expression). Modification is done either to the source code or to the executable the compiler generates. There are several commercial and freeware coverage tools available. Coverage tool selection should be based on the type of coverage measure selected, the language of the source code, the size of the program, and the ease of integration into the build process.

Profiling:

Profiling allows testers to learn where the software under test is spending most of its time and the sequence of function calls as well. This information can show which pieces of the software are slower than expected and which functions are called more often than expected. From a security testing perspective, the knowledge of performance bottlenecks help uncover vulnerable areas that are not apparent during static analysis. The call graph produced by the profiling tool is helpful in program understanding. Certain profiling tools also detect memory leaks and memory access errors (potential sources of security violations). In general, the functional testing team or the development team should have access to the profiling tool, and the security testers should use the same tool to understand the dynamic behavior of the software under test.

WBT Tool: Source Code Analysis

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:38:00 +0000

Source code analysis is the process of checking source code for coding problems based on a fixed set of patterns or rules that might indicate possible security vulnerabilities. Static analysis tools scan the source code and automatically detect errors that typically pass through compilers and become latent problems. The strength of static analysis depends on the fixed set of patterns or rules used by the tool; static analysis does not find all security issues. The output of the static analysis tool still requires human evaluation. For white box testing, the results of the source code analysis provide a useful insight into the security posture of the application and aid in test case development. White box testing should verify that the potential vulnerabilities uncovered by the static tool do not lead to security violations. Some static analysis tools provide data-flow and control-flow analysis support, which are useful during test case development.

A detailed discussion about the analysis or the tools is outside the scope of this content area. This section addresses how source code analysis tools aid in white box testing. For a more detailed discussion on the analysis and the tools, refer to the Code Analysis Tools content area.

Error handling

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:30:00 +0000

The most neglected code paths during the testing process are error handling routines. Error handling in this paper includes exception handling, error recovery, and fault tolerance routines. Functionality tests are normally geared towards validating requirements, which generally do not describe negative (or error) scenarios. Even when negative functional tests are created, they don’t test for non-normative behavior or extreme error conditions, which can have security implications. For example, functional stress testing is not performed with an objective to break the system to expose security vulnerability. Validating the error handling behavior of the system is critical during security testing, especially subjecting the system to unusual and unexpected error conditions. Unusual errors are those that have a low probability of occurrence during normal usage. Unexpected errors are those that are not explicitly specified in the design specification, and the developers did not think of handling the error. For example, a system call may throw an ”unable to load library” error, which may not be explicitly listed in the design documentation as an error to be handled. All aspects of error handling should be verified and validated, including error propagation, error observability, and error recovery. Error propagation is how the errors are propagated through the call chain. Error observability is how the error is identified and what parameters are passed as error messages. Error recovery is getting back to a state conforming to specifications. For example, return codes for errors may not be checked, leading to uninitialized variables and garbage data in buffers; if the memory is manipulated before causing a failure, the uninitialized memory may contain attacker-supplied data. Another common mistake to look for is when sensitive information is included as part of the error messages.

Configuration

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:29:00 +0000

In many cases, software comes with various parameters set by default, possibly with no regard for security. Often, functional testing is performed only with the default settings, thus leaving sections of code related to non-default settings untested. Two main concerns with configuration parameters with respect to security are storing sensitive data in configuration files and configuration parameters changing the flow of execution paths. For example, user privileges, user roles, or user passwords are stored in the configuration files, which could be manipulated to elevate privilege, change roles, or access the system as a valid user. Configuration settings that change the path of execution could exercise vulnerable code sections that were not developed with security in mind. The change of flow also applies to cases where the settings are changed from one security level to another, where the code sections are developed with security in mind. For example, changing an endpoint from requiring authentication to not requiring authentication means the endpoint can be accessed by everyone. When a system has multiple configurable options, testing all combinations of configuration can be time consuming; however, with access to source code, a risk-based approach can help in selecting combinations that have higher probability in exposing security violations. In addition, coverage analysis should aid in determining gaps in test coverage of code paths.

Component Interfaces

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:28:00 +0000

Applications usually communicate with other software systems. Within an application, components interface with each other to provide services and exchange data. Common causes of failure at interfaces are misunderstanding of data usage, data lengths, data validation, assumptions, trust relationships, etc. Understanding the interfaces exposed by components is essential in exposing security bugs hidden in the interactions between components. The need for such understanding and testing becomes paramount when third-party software is used or when the source code is not available for a particular component. Another important benefit of understanding component interfaces is validation of principles of compartmentalization. The basic idea behind compartmentalization is to minimize the amount of damage that can be done to a system by breaking up the system into as few units as possible while still isolating code that has security privileges. Test cases can be developed to validate compartmentalization and to explore failure behavior of components in the event of security violations and how the failure affects other components.

Environment

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:25:00 +0000

Software can only be considered secure if it behaves securely under all operating environments. The environment includes other systems, users, hardware, resources, networks, etc. A common cause of software field failure is miscommunication between the software and its environment. Understanding the environment in which the software operates, and the interactions between the software and its environment, helps in uncovering vulnerable areas of the system. Understanding dependency on external resources (memory, network bandwidth, databases, etc.) helps in exploring the behavior of the software under different stress conditions. Another common source of input to programs is environment variables. If the environment variables can be manipulated, then they can have security implications. In general, analyzing entities outside the direct control of the system provides good insights in developing tests to ensure the robustness of the software under test, given the dependencies.

Data

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:24:00 +0000

All input data should be untrusted until proven otherwise, and all data must be validated as it crosses the boundary between trusted and untrusted environments [Howard 02]. Data sensitivity/criticality plays a big role in data-based testing; however, this does not imply that other data can be ignored—non-sensitive data could allow a hacker to control a system. When creating tests, it is important to test and observe the validity of data at different points in the software. Tests based on data and data flow should explore incorrectly formed data and stressing the size of the data. The section “Attacking with Data Mutation” in [Howard 02] describes different properties of data and how to mutate data based on given properties. To understand different attack patterns relevant to program input, refer to chapter six, “Crafting (Malicious) Input,”. Tests should validate data from all channels, including web inputs, databases, and networks; file systems, and environment variables. Risk analysis should guide the selection of tests and the data set to be exercised.

Classes of Tests

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:22:00 +0000

Creating security tests other than ones that directly map to security specifications is challenging, especially tests that intend to exercise the non-normative behavior of the system. When creating such tests, it is helpful to view the software under test from multiple angles, including the data the system is handling, the environment the system will be operating in, the users of the software (including software components), the options available to configure the system, and the error handling behavior of the system. There is an obvious interaction and overlap between the different views; however, treating each one with specific focus provides a unique perspective that is very helpful in developing effective tests.

Code Coverage Analysis

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:13:00 +0000

Code coverage is an important type of test effectiveness measurement. Code coverage is a way of determining which code statements or paths have been exercised during testing. With respect to testing, coverage analysis helps in identifying areas of code not exercised by a set of test cases. Alternatively, coverage analysis can also help in identifying redundant test cases that do not increase coverage. During ad hoc testing (testing performed without adhering to any specific test approach or process), coverage analysis can greatly reduce the time to determine the code paths exercised and thus improve understanding of code behavior. There are various measures for coverage, such as path coverage, path testing, statement coverage, multiple condition coverage, and function coverage. When planning to use coverage analysis, establish the coverage measure and the minimum percentage of coverage required. Many tools are available for code coverage analysis. It is important to note that coverage analysis should be used to measure test coverage and should not be used to create tests. After performing coverage analysis, if certain code paths or statements were found to be not covered by the tests, the questions to ask are whether the code path should be covered and why the tests missed those paths. A risk-based approach should be employed to decide whether additional tests are required. Covering all the code paths or statements does not guarantee that the software does not have faults; however, the missed code paths or statements should definitely be inspected. One obvious risk is that unexercised code will include Trojan horse functionality, whereby seemingly innocuous code can carry out an attack. Less obvious (but more pervasive) is the risk that unexercised code has serious bugs that can be leveraged into a successful attack.

Trust Boundaries Mapping

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:12:00 +0000

Defining zones of varying trust in an application helps identify vulnerable areas of communication and possible attack paths for security violations. Certain components of a system have trust relationships (sometimes implicit, sometime explicit) with other parts of the system. Some of these trust relationships offer ”trust elevation” possibilities—that is, these components can escalate trust privileges of a user when data or control flow cross internal boundaries from a region of less trust to a region of more. For systems that have n-tier architecture or that rely on several third-party components, the potential for missing trust validation checks is high, so drawing trust boundaries becomes critical for such systems. Drawing clear boundaries of trust on component interactions and identifying data validation points (or chokepoints) helps in validating those chokepoints and testing some of the design assumptions behind trust relationships. Combining trust zone mapping with data-flow analysis helps identify data that move from one trust zone to another and whether data checkpoints are sufficient to prevent trust elevation possibilities. This insight can be used to create effective test cases.

Abuse Cases

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:11:00 +0000

Abuse cases help security testers view the software under test in the same light as attackers do. Abuse cases capture the non-normative behavior of the system. While in [McGraw 04c] abuse cases are described more as a design analysis technique than as a white box testing technique, the same technique can be used to develop innovative and effective test cases mirroring the way attackers would view the system. With access to the source code, a tester is in a better position to quickly see where the weak spots are compared to an outside attacker. The abuse case can also be applied to interactions between components within the system to capture abnormal behavior, should a component misbehave. The technique can also be used to validate design decisions and assumptions. The simplest, most practical method for creating abuse cases is usually through a process of informed brainstorming, involving security, reliability, and subject matter expertise. Known attack patterns form a rich source for developing abuse cases.

Testing Techniques: Code-Based Fault Injection

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:10:00 +0000

The fault injection technique perturbs program states by injecting software source code to force changes into the state of the program as it executes. Instrumentation is the process of non-intrusively inserting code into the software that is being analyzed and then compiling and executing the modified (or instrumented) software. Assertions are added to the code to raise a flag when a violation condition is encountered. This form of testing measures how software behaves when it is forced into anomalous circumstances. Basically this technique forces non-normative behavior of the software, and the resulting understanding can help determine whether a program has vulnerabilities that can lead to security violations. This technique can be used to force error conditions to exercise the error handling code, change execution paths, input unexpected (or abnormal) data, change return values, etc. One of the drawbacks of code based methods listed in the book is the lack of access to source code. However, in this content area, the assumptions are that source code is available and that the testers have the knowledge and expertise to understand the code for security implications.

Testing Techniques: Data-Flow Analysis

noreply@blogger.com (Ritesh Kumar) — Sun, 30 Jan 2011 08:03:00 +0000

Data-flow analysis can be used to increase program understanding and to develop test cases based on data flow within the program. The data-flow testing technique is based on investigating the ways values are associated with variables and the ways that these associations affect the execution of the program. Data-flow analysis focuses on occurrences of variables, following paths from the definition (or initialization) of a variable to its uses. The variable values may be used for computing values for defining other variables or used as predicate variables to decide whether a predicate is true for traversing a specific execution path. A data-flow analysis for an entire program involving all variables and traversing all usage paths requires immense computational resources; however, this technique can be applied for select variables.

The simplest approach is to validate the usage of select sets of variables by executing a path that starts with definition and ends at uses of the definition. The path and the usage of the data can help in identifying suspicious code blocks and in developing test cases to validate the runtime behavior of the software. For example, for a chosen data definition-to-use path, with well-crafted test data, testing can uncover time-of-check-to-time-of-use (TOCTTOU) flaws.