• Verify equivalence classes of expected results
• Verify equivalence class boundaries
• Pair boundary values and edges to verify corner cases

When test factors have values that depend on other test factors, the values are given by simple, user-defined functions. Each function’s arguments are the test factors that determine the dependent values. The functions are called during test case generation. And when test factors are both dependent and determinant, their composite relations are resolved automatically.

Embedded functions can be reused for different types of designs, and they can enhance consistency and accuracy as test factors and values change. Specifying equivalence class and boundary requirements as simple functions, in a language familiar to software engineers, can automate much of the analysis for test designs.

For more details on how embedded functions work, see Test design automation: equivalence classes, boundaries, edges and corner cases. Examples with specified test cases are included.

About Testcover.com
Testcover.com offers an automated, high-performance test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is accessible via the cloud–a software as a service (SaaS) independent test design solution. For more information visit Testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.

“While the two approaches can be adopted as needed, there are significant differences,” Sherwood said. “The rifle approach directs test cases to each type of result, for efficient testing with fewer cases. But the expected results need to be known. When the system is not well specified, the shotgun approach may be required.” Slides presented at the workshop are available at the Testcover.com background page.

About Testcover.com
Testcover.com offers a pairwise, automated, high-performance test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is accessible via the cloud–a software as a service (SaaS) independent test design solution–and its Web Services Description Language (WSDL) interface enables ready integration with existing test tools. For more information visit Testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.

“Our customer base is increasingly global, including multinational corporations and researchers at major universities,” said George Sherwood, CEO, Testcover.com. “We are gratified to be working with Business Innovations to continue expanding that global footprint, supporting Gary Rueda Sandoval and his team in delivering the optimal test design experience for a wide range of users.” The group administration interface described in Testcover.com’s June 2011 release was precisely intended to support Testcover.com resellers such as Business Innovations. Using this interface, resellers can manage subscriptions for their customers with the same capabilities as self-management for Testcover.com’s MNC customers.

About Testcover.com
Testcover.com offers a pairwise, automated, high-performance test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is accessible via the cloud–a software as a service (SaaS) independent test design solution–and its Web Services Description Language (WSDL) interface enables ready integration with existing test tools. Testcover.com was featured at Management World 2011 based on its selection to Stratecast’s 10 To Watch for 2011 and is a Premier Sponsor of VerifyATI. For more information and links to the company’s social media sites visit Testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.

Testcover.com CEO George Sherwood said, “We see this capability as a significant enhancement to the convenience and efficiency of our service. Now customers can opt in to self-management and get the highest levels of visibility and control.” Sherwood explained a popular feature of self-management: “Customers with multiple software engineers use our group subscriptions to simplify billing and reduce costs. They purchase a number of subscription activation codes for individual engineers in the group. These customers often have development groups that change, with members joining or leaving independent of any overall subscription period, and customers expect a cloud service to handle their changing needs simply and efficiently.” Testcover.com’s new self-management provides these customers the means to assign an unused activation code to an engineer joining the group, and start the subscription at that time. If an engineer leaves, the unused subscription time can be retained in a new activation code for the replacement.

Use of Testcover.com’s new group administration interface is optional, and there is no additional cost for it. The interface allows a designated user to access summary information about the group and to perform several management tasks:

• List available subscription activation codes – show which activation codes have not been used yet.
• Register a new user – select an activation code to register a new user in forms that are prepopulated with group information.
• List active users – show current users and their respective renewal dates.
• Reset password for an active user – choose a new password for an active user and email it to that user.
• Renew subscriptions for active users – extend service for selected users.
• Suspend subscriptions for active users – end service for selected users and transfer remaining subscription time to new activation codes.
• Purchase new subscriptions – generate subscription activation codes for new users.

About Testcover.com
Testcover.com offers a pairwise test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is accessible via the cloud–a software as a service (SaaS) independent test design solution–and its Web Services Description Language (WSDL) interface enables ready integration with existing test tools. Testcover.com was featured at Management World 2011 based on its selection to Stratecast’s 10 To Watch for 2011. For more information and links to the company’s social media sites visit Testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.

Colts Neck, NJ, USA, 11 June 2011 – Testcover.com has announced a performance improvement of up to 25% in its industry-leading cloud-based test case generator service. This improvement is available at no additional cost to all users. Current response time examples are given on the Testcover.com performance statistics page.

Proprietary research conducted by Testcover.com last year found significant response time improvements from an optimization of its search algorithm. Response time was reduced in small search spaces as well as large ones, with greater reductions in the larger search spaces. Testcover.com CEO George Sherwood said, “Users running typical jobs may or may not notice the improvement because our previous response times with the service were so quick, but we have now achieved response time reductions of 25% for large requests.”

The performance of a pairwise test case generator depends on two essential components: the number of test cases produced and the response time for generating them. Practical software engineering tools require an optimal balance between the two. Testcover.com runs an ongoing research program to reduce the number of test cases generated by the service and employs state-of-the art engineering techniques for prompt response times.

Sherwood also noted, “This enhancement reinforces our commitment to continual performance improvement–reducing the number of test cases as well as the response time to generate them.”

About Testcover.com
Testcover.com offers a pairwise test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is accessible via the cloud–a software as a service (SaaS) independent test design solution–and its Web Services Description Language (WSDL) interface enables ready integration with existing test tools. Testcover.com was featured at Management World 2011 based on its selection to Stratecast’s 10 To Watch for 2011. For more information and links to the company’s social media sites visit Testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.

Colts Neck, NJ, USA and Dublin, Ireland | 19 May 2011 – Testcover.com will be in the Innovation Spotlight at the TM Forum’s Management World 2011 in Dublin, Ireland, 23-26 May. Testcover.com was chosen to appear based on its selection by research & consulting firm Stratecast, a division of Frost & Sullivan, to its “OSS/BSS Rat Pack: 10 to Watch for 2011” report published in December 2010. Stratecast publishes this report annually, selecting 10 companies to watch in the coming year that, per Stratecast, “share characteristics with the original Rat Pack team of Hollywood actors from the mid-1950s including Frank Sinatra, Dean Martin, Sammy Davis Jr., Peter Lawford and Joey Bishop. Like its namesake, the companies included in this report are solid performers who work well, both individually and as an ensemble through partnerships, to satisfy the needs of communications service providers (CSPs) worldwide.”

In his remarks at the Innovation Spotlight, Testcover.com CEO George Sherwood is expected to highlight the versatility, performance and Cloud features of Testcover.com’s automated test design service. In a recent interview, Sherwood said: “Testcover.com is meeting the growing need for test design of complex systems, with features and performance for practicing software engineers building real systems. We provide Testcover.com as a Cloud service for easy, up-to-date use, everywhere.”

One example Sherwood is expected to cite at the event illustrates how Testcover.com’s pairwise (or all-pairs) advantage translates into real-world benefit. Faced with more than 205 trillion possible test cases for a feature with 30 inputs, a company was able to cover all 3,915 pairs of test values with only 18 test cases. Pairwise test designs from Testcover.com use far fewer test cases than random selection for comparable coverage.

Sherwood worked 25 years at Bell Labs and AT&T Labs on a variety of hardware, software and service development projects. He is a former district manager in the IP Services Delivery Division at AT&T Labs, responsible for the development of Broadband ISP services for leading communications service providers. Sherwood is the inventor of the Constrained Array Test System (CATS), the first pairwise testing tool to use a so-called “greedy search” algorithm. He has championed innovative test case reduction, faster performance and improved usability in test design tools.

About Stratecast
As people around the globe communicate more widely, frequently and enthusiastically than ever before, the potential for fusion of devices and services is unlimited. Making it all possible—from leading-edge mobile applications to a simple wireless or landline phone call—are operations/business support systems (OSS/BSS), which work behind the scenes to deliver, manage, and monetize the people, processes, and infrastructure that provide these services. Stratecast’s OSS/BSS Global Competitive Strategies program examines business and operational strategies, monitoring and analyzing software and services for this rapidly-changing market.

About TM Forum
With more than 700 corporate members in 195 countries, TM Forum is the world’s leading industry association focused on enabling best-in-class IT for service providers in the communications, media and cloud service markets. The Forum provides business-critical industry standards and expertise to enable the creation, delivery and monetization of digital services.

About Testcover.com
Testcover.com offers a pairwise test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is accessible via the cloud–a software as a service (SaaS) independent test design solution–and its Web Services Description Language (WSDL) interface enables ready integration with existing test tools. Founder and CEO George Sherwood worked 25 years at Bell Labs and AT&T Labs and is on the board of the Princeton, NJ, USA Joint Professional Chapter of the ACM and IEEE Computer Society. For more information and links to the company’s social media sites visit Testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.

According to content publicly available at http://stratecast.com, Stratecast names 10 companies each year that embrace what it calls “the Rat Pack mentality,” sharing a number of characteristics with the original Rat Pack team of entertainers from the mid-1950s including Frank Sinatra, Dean Martin, Sammy Davis Jr., Peter Lawford and Joey Bishop. Stratecast stated that the suppliers described in the report are  “small, agile, hungry, well-managed companies that are emerging as real competitors to the established elite” and present the market with a host of innovative new concepts and capabilities.

Colts Neck, NJ, USA | 22 November 2010 – Quality Testing, the world’s leading and fastest-growing social network and resource center for the software testing/QA community, interviewed Testcover.com CEO George Sherwood on the importance of pairwise testing generators. Quality Testing has published the content of the interview throughout its site including an overview on the Quality Testing home page, an appearance on the Quality Testing Featured Interviews page and a unique site page dedicated to the full interview. Quality Testing also published the interview on PRLog.org.

About Quality Testing
Quality Testing (QT) is the world’s leading and fastest-growing social network and resource center for the software testing/QA community where quality and testing professionals can meet and share best practices about software testing. QT is compatible with all popular web browsers and also offers an iPhone interface so the growing global testing community can access QT on the go. For more information visit the Quality Testing website.

About Testcover.com
Testcover.com offers a pairwise test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is accessible via the cloud–a software as a service (SaaS) independent test design solution–and its Web Services Description Language (WSDL) interface enables ready integration with existing test tools. For more information visit Testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.

Introduction

The performance of a pairwise test case generator depends on two essential components: the number of test cases produced and the response time for generating them. Practical software engineering tools require an optimal balance between the two. A quick response with many test cases may present the test team with unnecessary work. On the other hand, a fast-paced development organization needs tools that can keep up. For this reason, Testcover.com’s research program involves ongoing improvements to both components, and the examples on our performance page report both.

Testcover.com’s research results are of interest to software engineering practitioners as well as researchers. Although some work is proprietary, we have shared many results in peer-reviewed publications and in conference presentations. This summary reports on Testcover.com’s research activities over the last five years and the impact of this work on service performance–number of test cases and response time.

Proprietary research

WSDL interface. In early 2010 Testcover.com introduced its WSDL interface to enable integration with other test tools. Although performance improvement was not the main objective, this WSDL implementation offers even faster response times compared with the HTML interface. When coupled with a test automation tool, the service has the potential to save even more time in the test design and execution process.

Test case search improvement. Unpublished research conducted this year has found significant response time improvements from an optimization of our search algorithm. Response time is reduced in small search spaces as well as large ones, with bigger reductions in the larger search spaces. Current plans call for introduction of the improvement in 2011.

Test case reduction. Testcover.com runs an ongoing research program to reduce the number of test cases generated by the service. For example, at STAREAST 2005 we demonstrated our results generating test cases for an HTML form. Test values were selected for 21 test factors, and 63 test cases were generated. Since this example was first shown, its results have been improved to 59 test cases (a reduction of more than 6%). The service has had thousands of such improvements in the last five years.

Academic papers

Optimal and near-optimal mixed covering arrays by column expansion. GB Sherwood. Discrete Mathematics, 308(24):6022-6035, 2008. This paper described new constructions for strength two mixed arrays having minimal numbers of test cases. It also showed how to maximize the numbers of factors in the arrays. The best constructions for various array parameters were reported in several tables.

Products of mixed covering arrays of strength two. CJ Colbourn, et al. Journal of Combinatorial Designs, 14(2):124-138, 2006. Strength two mixed covering arrays are the templates for pairwise test cases. This paper brought six authors together to report on several different construction methods for these arrays. It also discussed how the methods can be combined effectively.

Covering arrays of higher strength from permutation vectors. GB Sherwood, et al. Journal of Combinatorial Designs, 14(3):202-213, 2006. Higher-strength covering arrays are templates for testing values beyond pairwise: Three-at-a-time, four-at-a-time, etc. This paper introduced a new technique for constructing arrays having fewer test cases than previously known arrays. Subsequently, academic researchers have extended the technique to find additional higher strength arrays.

Public presentations

Pairwise Testing Comes of Age. This talk at the STAREAST 2008 reviewed the development of pairwise testing and provided examples to illustrate its use. The talk also discussed research topics and technical advances related to performance improvement.

A Column Expansion Construction for Optimal and Near-Optimal Mixed Covering Arrays. This presentation in May 2006 described research in progress for the Workshop on Covering Arrays at Carleton University in Ottawa, Canada. This work in its completed form was published in Discrete Mathematics (above).

Improving Test Design with a Novel Test Case Generator. This talk at STAREAST 2005 introduced Testcover.com’s new technology and its continuous improvement through ongoing research. The HTML form example (above) was first demonstrated at this conference.

About Testcover.com
Testcover.com offers a pairwise test case generator that helps testers get the results they need to develop optimal products and services in a fraction of the time, saving time and money on every project. Testcover.com is a software as a service (SaaS) independent test design solution and its Web Services Description Language (WSDL) interface enables ready integration with existing test tools. For more information visit http://testcover.com, dial +1 732 299 4852 or email mgmt@testcover.com.