Automation Testing Using CodedUI - Visual Studio 2010

Hot to convert Coded UI .trx report to .html format

2013-09-27T01:58:00.000-07:00

Hi Guys

In this post I will show you how to conert .trx report file to .html format. for this you need to download utility from below URL:

http://trx2html.codeplex.com/

TRX2HTML is used to convert .trx to .html format.

Below are the steps:

trx2html "TestResult.trx"

by executing statement in the command prompt you can convert .trx to .html format.

Regards
Team

If above post helps you please share it and post your comment.. :)

How to create set by step report of a test case while running a validation test case ?

2013-09-27T00:27:00.001-07:00

Hi Guys

Below sample shows how you can create a detailed report while running a validation test case thru coded-UI. Lets take a an example: You have to validate 4-5 elements on a page and what will happen if your 2nd element doesn't exist have some visibility issue.

TEST CASE GET FAILED AND IN THIS CASE WHAT WILL HAPPEN TO THE VALIDATION CHECK FOR OTHER ELEMENT WHICH ARE LEFT BECAUSE BECAUSE 2ND ELEMENT DOESN'T EXIST.

HOW ABOUT THIS: If your 2nd doesn't exist but your script will throw an error for that and also continue on validating other element too.

By using below you can successfully do that:

public void TC_Bing_VerifyingTheHomepageBing(string testCaseID, string inputFile)

Code:

{
#region Test Steps

try
{
//Declaring array to store exceptions and string should number of elements you are validating your test case
string[] exceptionMessage = new string[50];
int count = 0;

//fetch test data from csv file
string[] testdata = common.ReadCSV(testCaseID, separator, inputFile);

if (uIBingHomePage.Title == testdata[1])
{
if (!uIWebHyperlink.Exists)
{
exceptionMessage[count] = "Web link is not available";
count++;
}

if (!uIImagesHyperlink.Exists)
{
exceptionMessage[count] = "Image link is not available";
count++;
}

if (!uINewsHyperlink.Exists)
{
exceptionMessage[count] = "News link is not available";
count++;
}

if (!uIMoreHyperlink.Exists)
{
exceptionMessage[count] = "More link is not available";
count++;
}

}
else
{
Assert.Fail("HomePage of Bing is not opened as expected");
}

//Collecting all the assertion message and failing the test case
if (exceptionMessage != null && exceptionMessage[0] != null)
{
string errorMessage = string.Empty;
foreach (string x in exceptionMessage)
errorMessage += x + AppConstants.comma;
errorMessage.Substring(0, errorMessage.Length - 1);
errorMessage = errorMessage.Replace(",,", "");
errorMessage = errorMessage.Replace(",", " , ");
Assert.Fail(errorMessage);
}
}

catch (Exception ex)
{
throw ex;
}

#endregion
}
#endregion

Thanks
Team

If this post help you. Please share it and give your comments on it :)

Cross Browser Automation Testing Using Coded UI- VSTS

2013-09-25T22:52:00.001-07:00

Hi Guys

Soon I will explain the how to do cross browser automation testing using Coded-UI.....

Regards
Team

Here We Go....

General Information: Coded-UI Support Chrome and Firefox for cross browser testing, it doesn't support safari.

Pre-Requisites: You need to install "

"Selenium components for Coded UI Cross Browser Testing "

by following below steps: In the Visual Studio Goto Tools --> Extensions and Updates --> Search for selenium. Download and install it.

http://visualstudiogallery.msdn.microsoft.com/11cfc881-f8c9-4f96-b303-a2780156628d?SRC=VSIDE

Now by adding below line in the [TestMethod] you can run your test case in choice of your browser:

BrowserWindow.CurrentBrowser = "Chrome";

for firefox just replace chrome with firefox and it will launch the specified browser on executing the test case.

Please like the post if it helps you.

Thanks

Team

Software Testing with Visual Studio 2012 Certification

2013-06-21T02:48:00.002-07:00

On following URL:

http://www.microsoft.com/learning/en/us/exam.aspx?id=70-497#fbid=0WAhIdUoJdV

How to read remote system / machine registries using VS2010 CodedUI

2012-08-10T00:56:00.000-07:00

Hi All

Currently I am working on how to fetch required values or information from regedit to make sure the deployment done have correct server address or for others reason too.

Soon I will share my finding with the sample project..........

Let me know if any queries related to that.

Regards
Arun Kapoor

Upgraded Batch file for download project / code from Team Foundation Server (TFS) using command line/ command prompt

2012-08-10T00:53:00.000-07:00

In my last post I shared how we can download code/project from TFS:

http://www.blogger.com/blogger.g?blogID=8188117718822686915#editor/target=post;postID=583709375510837785;onPublishedMenu=overview;onClosedMenu=overview;postNum=12;src=postname

Please find upgraded version of the same batch file with some extra features:

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@echo off
:: variable declarations
::*********************************************************
set userName=
set password=
::GetLatest:Only If User Need Latest Code;
::CreateTFS:Only if User want to create new mapping with TFS and take latest code;
::NO: User Don't want to create New TFS not need latest code
set isLatestCodeRequired=CreateTFS
set workSpacename=
set tfsURL=
set tfsAddress=
::*********************************************************
::**************************************************
:: Create work space and Download code from TFS
::**************************************************
:: check for existence of [Program Files(x86)]
::if %isTFSMappingRequired% == Y (GOTO CreateNewTFSAndGetLatest)

if %isLatestCodeRequired% == GetLatest (GOTO DownloadTFSCode)
if %isLatestCodeRequired% == CreateTFS (GOTO CreateNewTFSAndGetLatest)
if %isLatestCodeRequired% == NO (GOTO NotDownloadTFSCode)
:DownloadTFSCode
set EnvpathVS2010=%VS100COMNTOOLS%
::Creating path of Visual Studio Command Prompt
set PathToAppend=\VC\vcvarsall.bat" x86
set EnvpathVS2010=%EnvpathVS2010:\Common7\Tools\=%
set RootDirectory=%windir%
set RootDirectory=%windir:\Windows=%
set FinalPath="%EnvpathVS2010%%PathToAppend%
call %FinalPath%
cd\
%RootDirectory%
::Create Directory Application_App_Checkout
md %workSpacename%
::Into Directory Application_App_Checkout
cd %workSpacename%
::tf workspace /delete %workSpacename% /noprompt
::tf workspace /new %workSpacename% /noprompt /collection:%tfsURL% /login:%userName%,%password%
tf.exe get %tfsAddress% /recursive /all /overwrite /force /login:%userName%,%password%
echo %isLatestCodeRequired%
call C:\Application_AppCheckout_SourceCode_Download\AutomatedTests\ApplicationCheckOut\Development\Application_AppCheckoutRelease.bat
GOTO End
:NotDownloadTFSCode
echo "User does not wish to download latest code from TFS."
pause
echo %isLatestCodeRequired%

GOTO End
:CreateNewTFSAndGetLatest
set EnvpathVS2010=%VS100COMNTOOLS%
::Creating path of Visual Studio Command Prompt
set PathToAppend=\VC\vcvarsall.bat" x86
set EnvpathVS2010=%EnvpathVS2010:\Common7\Tools\=%
set RootDirectory=%windir%
set RootDirectory=%windir:\Windows=%
set FinalPath="%EnvpathVS2010%%PathToAppend%
call %FinalPath%
cd\
%RootDirectory%
::Create Directory Application_App_Checkout
md %workSpacename%
::Into Directory Application_App_Checkout
cd %workSpacename%
tf workspace /delete %workSpacename% /noprompt
tf workspace /new %workSpacename% /noprompt /collection:%tfsURL% /login:%userName%,%password%
tf.exe get %tfsAddress% /recursive /login:%userName%,%password%
echo %isLatestCodeRequired%

GOTO End
End Local

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Automation Testing In The Cloud Computing

2012-07-20T02:55:00.000-07:00

In some form or another, more aspects of application testing will become increasingly automated. While there is a general bias toward manual testing, IT services companies such as Infosys increasingly are focused on automating as much of the application-testing process as possible.

According to Aparna Sharma, head of client services for independent validation and testing services for Infosys, it’s really just a matter of scale. Not only is the sheer volume of code being generated increasing, existing code is being updated much more often. Much of this has to do with iterative application-development methods that greatly speed the application-development process.

The biggest challenge, however, is getting the best application developers to focus on automating the testing. The best developers naturally want to do anything but test. The folks that generally do the testing, however, don’t have enough insight into the application.

Infosys is working on automated testing routines for a variety of applications in various vertical industries that will be available as a cloud computing service. Even though automating application testing might not be all that high a priority within the enterprise, as IT services firms go, the rest of the enterprise is sure to follow. And there are going to be a lot more IT services firms offering automated testing services in the cloud than just Infosys.

In the end, it will come down to economics. Manual application testing is too slow and costly. We’ll never automate the entire application-testing process, but it's pretty clear that much of it can be done just as well by a machine in the cloud as a programmer on premise. But few machines can actually create the program in the first place.

More from our Network

Assuring Quality in Third-party Code Integrations CTO Edge
The Middleware Muddle CTO Edge
Application Deployment in the New Virtual World CTO Edge
Giving Developers Keys to the Virtual Kingdom CTO Edge
Getting Ready for the Real-time Enterprise CTO Edge

Unit Test Automation

2012-07-20T02:45:00.000-07:00

NUnit is a unit-testing framework for all .Net languages. Initially ported from JUnit, the current production release, version 2.6, is the seventh major release of this xUnit based unit testing tool for Microsoft .NET. It is written entirely in C# and has been completely redesigned to take advantage of many .NET language features, for example custom attributes and other reflection related capabilities.
NUnit does not create any test scripts by itself. You have to write test scripts by yourself, but NUnit allows you to use its tools and classes to make the unit testing easier. The points to be remembered about NUnit are listed below:

NUnit is not an automated GUI testing tool.
It is not a scripting language, all test are written in .NET supported language, e.g., C#, VC, VB.NET, J#, etc.

Nunit works by providing a class framework and a test runner application. The class framework allows to write the test cases based on the application and the test is run using the test runner application.
The latest version of Nunit framework and the Help related to it can be downloaded from http://www.nunit.org/index.php?p=download
So far we took a look at unit testing, and NUnit, seeing how easy it was to create automated tests through the help of NUnit. Unit testing with NUnit is useful for testing class libraries that makeup an ASP.NET application, but is not designed to test the UI portions of an ASP.NET application.

Test Automation Guidance using VS 2010 Coded UI

2012-07-20T02:10:00.003-07:00

The objective of this artifact is to provide a L200 test automation guidance leveraging VS 2010 Coded UI features to foster a continuous, consistent and standard automation approach. The document can also be used as a quick jump start guide for beginners.
However the bigger objective is Build, Deploy and Test leveraging Visual Studio Lab Management features, which will allow Automation Testers to reap the benefits of the following
· Integrating with the build process for automatic build deployment
· Execution on VMs
· Capabilities of getting the test automation results reported as part of Test Manager (MTM) for better reporting.

Pre-requisites:
Ensure you have ‘CodedUISampleFramework’ sample Coded UI project
For VS 2010 installation & Configuration, go to Appendix: VS 2010 Installation Reference section
If you are new to Visual Studio Coded UI, go to Appendix: Getting started with VS2010
Ensure your application is supported by Coded UI, go to MSDN link

Jump to Case Study, if your objective is to get started with Coded UI using our recommended framework and practices.

http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-01-42-73-CodedUI/2086.Automation-Guidance-_2800_VS-2010-Coded-UI_2900_-ExternalCommunity.pdf

http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-01-42-73-CodedUI/0167.CodedUISampleFramework.rar

Author:
Raj Kamal (rajkamal@microsoft.com)

Automation Testing Using CodedUI - Visual Studio 2010: Discussion about design of a custom testing framew...

2012-07-20T00:43:00.001-07:00

Automation Testing Using CodedUI - Visual Studio 2010: Discussion about design of a custom testing framew...: Hello All, I want to discuss and get feedback about my ideas for creating a custom testing framework for our applications that leverages ...

Discussion about design of a custom testing framework that leverages Coded UI Testing in Visual Studio 2010

2012-07-20T00:14:00.000-07:00

Hello All,

I want to discuss and get feedback about my ideas for creating a custom testing framework for our applications that leverages the Coded UI Test in Visual Studio 2010. I need to evaluate my current design.

Definition: A UI testing platformis a system that provides interaction with the UI in an application under test. Coded UI Testing and Quicktest Pro are examples.

Definition: A custom testing frameworkis a layer or several layers of basis functionality that provides levels of indirection between the testing platform and actual tests that will be executed. The tests call functionality from the custom testing framework, which manipulates the UI using the UI testing platform or provides custom functionality of some sort. These levels of indirection are meant to enhance maintainability and robustness of Coded UI Tests.

In investigating Coded UI Testing I have found that recording of individual tests was often leading to very similar steps and UIMaps in each test. I have already gone throughhttp://msdn.microsoft.com/en-us/library/ff398056%28v=VS.100%29.aspx and have implemented multiple UIMaps. There is still a problem recording actions on a per-test basis though since for every test I am generating identical functionality. An example of this would be the logical group of actions called “open a project” which is a composite of many sub steps and is used in almost every test. It is therefore desirable to have a single method recorded or built by hand and stored in some library of functions that can be invoked when required. I would call these methods that implement logical groups “test components”.
Definition: A test componentis a test method existing in some layer of the custom testing framework that uses one or more UIMaps to manipulate UI elements to accomplish some specific task such as clicking an individual button, or to accomplish a logical group of actions such as opening a project. Test components can be identified by their common patterns in many tests.

There are two types of test component. An “atomic test component” might perform some number of actions against the UI, but will never reference other components. That is, an atomic test component only uses the UIMap to manipulate UI elements. The second type of test component is a composite test component that is built from atomic components and composite components. Each test component will require access to one or more UIMap(s) to function. Since each test component will be implemented as a “test method”, the exception handling mechanism that provides results for tests can still be used and will filter up through the hierarchy of components in the custom testing framework. Since a test component is either atomic or is a collection of test components, and any test component is handled in the same way whether it is atomic or a collection, this is somewhat analogous to the composite design pattern except not in the context of object construction. A good example of a composite component is the “open a project” test component. This method would take the name of the project to open as a parameter and use atomic and potentially other composite components to execute the logical group of operations that open a project. The UIMaps used by such components are a little bit of a concern.

The UIMap used by the “open a project” method also exists in the library and as such is not duplicated in multiple tests. Thus at the beginning of most tests, the test author will have to insert a call to the “open a project” method. Unfortunately as far as I can see at this point, this seems to break the test recording model since the Coded UI Test builder doesn’t provide a way to indicate a call to a method in the library instead of recording a new UIMap and method.

Ultimately the goal is to provide a comprehensive set of test components implementing logical groups of actions such that a test creator can avoid duplicating functionality and quickly assemble a test by calling the appropriate components. The testers here have limited development experience, but are in the process of learning the basics. Hopefully in the long term, anyone from the most experienced developer to the programming challenged will be able to create automated tests that are maintainable and robust.

With this model in mind, the recording functionality in VS2010 will be used only for creating the library of logical groups of actions (test components) and for per-test functionality that has not been included in the library at the time of test creation. This is ok, but it is note ideal and it would be nice to be able to extend the Coded UI Test Builder to add calls to the component library when required. A more serious problem lies in asserting or checkpointing values using the test creation.

While there may be some common assertions or checkpoints included in the library of components, for all tests there exist test specific expected values. These test specific expected values must be stored in each test or in some structure defined to hold them; perhaps a CSV file. Since the UIMaps for everything lie in the application specific library of methods, using the assertion creator of VS2010 will not reference the library UIMaps and will thus try to create a per test UIMap that may duplicate some of the library UIMap for that portion of the application under test. This may even be desirable in some cases, but it still creates a potential maintainability problem. In most cases it would be better if the assertion recorder and generator could generate expected values in a UIMap that subclasses (inheritance) one of the library UIMaps. Another useful way to do things would be to allow the assertion generator to add expected values to some file such as a CSV or Excel spreadsheet. The UIMap subclass situation would be ideal since a test can call its recorded assertion methods that exist local to the test, and these methods would be able to use the preexisting UIMap to verify expected values. As far as I know, using the Coded UI Test Builder in this way is not possible at this time and as such, assertion methods need to be provided via the library of test functionality. Somehow the test specific expected values need to be accounted for, but they cannot be recorded without duplicating UIMaps; undermining the usefulness of the recording tool.

I must make a design decision and balance the tradeoffs of maintainability and ease of use in whatever system I implement. Making a component-wise library of functionality will provide maintainability and scalability, but will require much more development time initially and could potentially be more difficult to use for non-developers. That said, I believe maintainability and robustness of the system is the highest priority for our automated testing system due to personnel constraints. I do not want to get bogged down maintaining old tests; I would rather build new tests and have to perform simple fixes to components used in older tests.

I have defined three levels of testing functionality as follows, each of which is a “test project” inside a “testing solution”. Each test project is compiled to a .dll and is referenced in the next higher level of functionality. These levels are as follows,

. The top level (level 1) is the common functionality that will be used in testing all of our software products. Examples of this are something like bitmap checkpointing, which any individual test might need to use.

2. The middle level (level 2) is the application dependent test functionality (test component) provider. This level is where all UIMaps exist and where the definitions of recorded or hand built atomic components and composite components exist.

3. The bottom level (level 3) contains the actual tests that will be executed. These tests will call test components from level 2 to perform some common actions such as “opening a project” and will also allow use of recording for per-test steps. These per-test recordings will allow the components from level 2 to be glued together to accomplish tasks for which there has been no test component defined in level 2. If it is found that all or part of these per-test recordings are being duplicated across different tests, then the common pattern can be extracted and will be a strong candidate for promotion to level 2.

I would like to know what anyone thinks about my design, and if anyone else has tried something similar. I am concerned about this design cutting out much of the recording of actual tests, but at this point I feel that once a large number of tests are created, this framework will make the tests more maintainable. Please give me your feedback and any ideas you have. Any input is welcome! I am trying to learn how to maximize the effectiveness of my testing project, and how to leverage VS2010 properly.

Performance Test with Capacity Tool, load balancing issues

2012-07-19T00:19:00.000-07:00

Excellent question! The answer is that with Configuration Services, the load balancing/failover detection happens from the Config-Services generated client itself (hence point to point, and this provides some big advantages especially for intranet scenarios; albeit some disadvantages as well). Hence, to get load balancing ala the StockTrader web app to the middle tier stock trader business services (or from business service tier to Order Processor if running in that remote service config); this is predicated on the client app implementing Configuration Services. The Capacity Planner tool does not---so it will *not* automatically load balance requests against backend service tiers (Java or .NET nodes). I have though about re-implementing the Capacity Planner tool using Configuration Services (actually, not hard to do), such that its agents can auto-detect nodes of middle tier being tested, or user can use ConfigWeb to setup multiple backend servers to load balance against. This would also help in testing of failover characteristics of Config Services...but for now the Capacity Planner tool just works against a single node at a time.

There are of course other load balancing options; for example hardware or using Windows Network Load Balancing software built into Windows Server--that virtualize an IP address against mulitple hosts...if using this, then Capacity Planner can be pointed at the virtualized address, and load balance accordingly; but not a test of COnfig-service driven load balancing. I will try to add into Capacity Planner an implementation of Config Services; this might be handy for setting config options, etc as well as doing load balancing and failover ala StockTrader and other samples for Config Service that are part of the download.

(there are also a couple of bugs to address in the tool, I hope to do, plus I want to add binding/config options for testing with both transport and message-level security--to both .NET and Java implementations of the services).

Also to note...I realize the Capacity Planner tool is very limited in what it can and cannot do; but the intent to to provide something *very* turnkey for the two workloads, as find many customers simply do not have the budget/time for buying and learning some of the more sophisticated load test tools out there. And without being able to drive load:

1) Cannot verify any results I publish

2) You cannot tune a system to any degree

3) Cannot judge relative performance of various stacks; architecture tradeoffs within a given stack; different deployment options regarding location of front end web server, etc.

So for now it does what it does. One goal is that the two workloads chosen (much like a canned TPC or SPEC benchmark) are built using what I consider "standard building blocks" for the platforms they support; while simple, they do represent to some good degree the "raw" performance/capacity of software/hardware deployed for more sophisticated apps that will generally use the same core underlying building blocks. WSTest is about measuring just Web Service especially network stack/serialization/de-serialization performance (including .NET or any Java stack; and also .NET TCP-Binary encoding vs. HTTP-XML encoding--and important options WCF adds while simulatenously hosting HTTP-XML endpoints for interop). StockTrader is about introducing typical business logic behind the web service, involving database access (to SQL or Oracle); DB transactions including potentially distributed two-phase transactions against JTA or .NET Distributed Transaction Coordinator; etc. So both workloads are useful, even given current constraints of this pre-packaged tool.

If only I had time to add some level of scripting or auto-generation of test scripts based on WSDL/Meta-DataExchange of service operations...that would be really cool but all of a sudden I would be spending time building something like a poor-man's version of LoadRunner or even VS Load Test Tools; when these tools exist already (although free and turnkey has some advantages as well).

I really want to encourage people to do more load testing...I think not matter the platform, load testing early and often saves tons of money in the long wrong----finding out an option/feature/architecture decision does not scale early in the dev process could mean saving millions of dollars finding out after a complete system has been developed. Hence the crude but free Capacity Planning tool. And also I really want people to *believe* the results I publish; I do not just do this as an MS guy but also as a curious guy who wants to find the "truth" (which is out there, to parden an X-Files reference).

I do have plans for Capacity Planner, I will add to plans:

1) Implement Config Services to enable turnkey testing of Config-Service hosted apps with Config Services load balancing, node notifications etc as you note is lacking.....

2) Expand to include testing of WS-Security; both message level encryption performance and transport security; as well as mixed Transport with Message Credentials for a variety of options StockTrader already supports (does anyone really yet *know* the relative perf of IBM's stack (or other Java stacks) implementing message level security vs. .NET WCF; note there is at least one MSDN whitepaper on the perf costs of WCF message level security relative to transport security or "wide-open" SOAP on MSDN. But relative to other stacks? I for one want to know!

3) Might be interesting as well to add a Federated security scenario, at least with WCF, for finding about this given interest on this forum around STS.....

got cutoff...

...performance. One big debate that I think will always rage is ORM-style apps/frameworks vs. more data-tier defined data access layers (more optimized, more flexible sometimes, less flexible other times). StockTrader sticks to the bread-and-butter: ADO.NET, largely using DataReaders; System.Transactions for transactions that promote automatically from single phase commit to two-phase commit/mid-tier tx logging--now on both Oracle and SQL Server backends); and use of a model class approach for passing data between tiers. Just like the WebSphere "Direct" mode of operations, that uses just a Stateless Session Bean to front-end direct access to DB from JDBC, vs. CMP and EJBs.

But interesting to test relative perf of both....my .NET style is akin to using JDBC/servlets in the Java world, and staying away from the ORM runtime stacks (although EJB 3.0 is better, that is a complete other architectural discussion (rant) I can go on, but should not right now)...I will note however that some of the more popular ORM open source technologies such as Hibernate are now also being implemented on .NET...sorry if I offended anyone...I am by no means the definitive expert....but have my opinions based on some level of experience; especially in the perf realm.

What is a Good Test Case?

2012-07-19T00:15:00.000-07:00

Great question! There are probably two 'paths' to answering that - the first path examines why you're testing at all, and the second looks at how you're actually writing your cases. In other words: strategy and process (ugh, I know..).

Some organizations view and implement testing as all about QA--validating that an application fulfills certain requirements (IE a tax calculation is completed and returns the expected result, or a server can serve up X pages per minute). I've worked at a company like that--they were implementing packaged software and they only cared that it accomplished what they bought it for. (Ask me, or head to my blog at http://thoughtsonqa.blogspot.com to read what I think about that approach.) Other organizations have to be (or choose to be) much more focused on overall quality. Not just 'will it fit my bill' but is it robust. SO there's a subtle difference, but the point is that a good test case is a solid step toward accomplishing the objectives. For instance, if the project is an internal line of business application for time entry, a test case which validates that two users can submit time concurrently and the data will retain integrity is a good test case. I think a case written which validates layout pixel by pixel would be a waste of time, money, and energy (would it get fixed, anyhow?).

Another point of quality for test cases is how it's written. I generally require my teams to write cases which contain the following (and I'm fine with letting them write 'titles only' and returning to flesh out later; as a matter of fact, one-time projects I generally shy away from much more than that).
Has a title which does NOT include 'path info' (ie, Setup:Windows:Installer recognizes missing Windows Installer 3.x and auto-installs). Keep the title short, sweet, and to the point.
Purpose: think of this as a mission statement. The first line of the description field explains the goal of the test case, if it's different from the title or needs to be expanded.
Justification: this is also generally included in the title or purpose, but I want each of my testers to explain why we would be spending $100, $500, or more to run this test case. Why does it matter? If they can't justify it, should they prioritize it?
Clear, concise steps "Click here, click there, enter this"
One (or more - another topic for a blog someday) clear, recognizeable validation points. IE, "VALIDATION: Windows begins installing the Windows Installer v 3.1" It pretty much has to be binary; leave it to management to decide what's a grey area (ie, if a site is supposed to handle 1,000 sessions per hour, it's binary - the site handles that, or not. Management decides whether or not 750 sessions per hour is acceptable)
Prioritization: be serious... Prioritize cases appropriately. If this case failed, would this be a recall-class issue, would we add code to an update for this case, would we fix it in the next version, or would we never fix it. Yes this is a bit of a judgment call but it's a valid way of looking a the case. Another approach is to consider the priority of a bug in terms of data loss, lack of functionality, inconvenience, or 'just a dumb bug' way.
Finally, I've flip-flopped worse than John Kerry on the idea of atomic cases. Should we write a bazillion cases covering one instance of everything, or should we write one super-case. I've come up with a description which I generally have to coach my teams on during implementation. Basically, write a case which will result in one bug. So for instance, I would generally have a login success case, a case for failed log in due to invalid password, a case for failed log in due to non-existant user name, a case for expired user name or password, etc. It takes some understanding of the code, or at least an assumption about the implementation. Again, use your judgment.

I read the response that a good case is one that has a high probability of finding a bug. Well... I see what the author is getting at, but I disagree with the statement if read at face-value. That implies a tester would 'filter' her case writing, probing more into a developer's areas of weakness. That's not helpful. Hopefully your cases will cover the project enough that all the important bugs will be exposed, but there's no guarantee. I think the middle ground is appropriate here - a good case 1) validates required functionality (proves the app does what it should) and 2) probes areas where, if a bug is found, the bug would be fixed (in a minimal QA environment) or (in a deeper quality environment) advances produce quality significantly.

Hope that helps!

http://www.yourtestmanager.com
http://thoughtsonqa.blogspot.com

Issue In Getting An Absolute Path - Visual Studio 2010 CodedUI

2012-07-19T00:07:00.001-07:00

Question:
When I am build my project corresponding DLLs file are getting at below location:
TestResults/Defaultfolder/Out

and also in the project bin location which is mentioned in the properties of the project.

When I am trying to find out the absolute path it is taking the path of dlls saved in the Out folder instead of bin folder. Have I made any change in the settings due to which this thing is happening?

Answer:
Check if you have enabled Deployment Items in your testsettings. If yes then the base directory is inside the TestResults folder and if it is disabled then it will be in the bin folder.

How To Use Regular Expression In Element Propeties Values using Visual Studio 2010 Coded UI

2012-07-18T04:41:00.001-07:00

Question: I have a object in my application whose id values is get dynamic:

this. muTextButton.SearchProperties[HtmlEdit.PropertyNames.Id] = "abc_yu2112121_ty56_hjhjh";

In the string "abc_yu2112121_ty56_hjhjh", following text is dynamic "yu2112121_ty56"

How can i handle this through regular expression or let me know if there is some other to handle this issue.

Answer:

Try This:

this.muTextButton.SearchProperties.Add("Id","abc_", PropertyExpressionOperator.Contains);

In this moment, there is no way to use regular expressions in the search properties.

How to maintain the values of Variables between multiple Test Methods in CodedUI when SmartMatchOptions is OFF

2012-07-18T03:06:00.001-07:00

I have defined a global variable say public int a=0;
There are two testmethods: login() and Search()
and following payback setting is set in TestInitialize
Playback.PlaybackSettings.SmartMatchOptions = SmartMatchOptions.None;
Now, in the test method login() I have written a line : a++ which sets the value of a as 1,
Now, when the Search() method is executed the value of a = 0 rather than 1.
So, can you please help how to maintain the session of the variables between Test Methods.

Script is able to find an element even when its ID is changed in the Search Criteria of that element in "UIMap.Designer.cs" file - CodedUI

2012-07-18T03:02:00.000-07:00

The automation script generated using Recording feature of CodedUI is able to find an element For Eg: a Textbox even when the ID of the textbox is changed in the Search Criteria of that element under UIMap.Designer.cs" file.
For example :

1) Suppose we have entered "ABC" in the Username textbox, the code generated for searching that textbox will be say:-
public HtmlEdit UITxtUserEdit
{
get
{
if ((this.mUITxtUserEdit == null))
{
this.mUITxtUserEdit = new HtmlEdit(this);
#region Search Criteria
this.mUITxtUserEdit.SearchProperties[HtmlEdit.PropertyNames.Id] ="txtUser";
#endregion
}
return this.mUITxtUserEdit;
}
}

2) Now if in the above code we change the Search Property value to "User" like this :-
public HtmlEdit UITxtUserEdit
{
get
{
if ((this.mUITxtUserEdit == null))
{
this.mUITxtUserEdit = new HtmlEdit(this);
#region Search Criteria
this.mUITxtUserEdit.SearchProperties[HtmlEdit.PropertyNames.Id] ="User";
#endregion
}
return this.mUITxtUserEdit;
}
}
then according to me the testcase should fail as we have changed the ID to be searched is different from the actual ID.
But the Script is not able to respond to this change and is able to find the Textbox.

How to Download Code from TFS (Team Foundation Server) using .bat file?

2012-07-18T02:17:00.001-07:00

Hello Guys

Below is the code to download code from TFS (Team Foundation Server) using command prompt or .bat file:

@echo off
:: variable declarations
::*********************************************************
set userName=Domain\UserName
set password=Password
set isCodeDownloadRequired=Y or N
set workSpacename=FOlder Name on your local drive
set tfsURL=URL of TFS
set tfsAddress=$/TFS Address
::*********************************************************
::**************************************************
:: Create work space and Download code from TFS
::**************************************************
:: check for existence of [Program Files(x86)]
if %isCodeDownloadRequired% == Y (GOTO DownloadTFSCode) ELSE (GOTO NotDownloadTFSCode)
:DownloadTFSCode
set EnvpathVS2010=%VS100COMNTOOLS%
::Creating path of Visual Studio Command Prompt
set PathToAppend=\VC\vcvarsall.bat" x86
set EnvpathVS2010=%EnvpathVS2010:\Common7\Tools\=%
set RootDirectory=%windir%
set RootDirectory=%windir:\Windows=%
set FinalPath="%EnvpathVS2010%%PathToAppend%
call %FinalPath%
cd\
%RootDirectory%
::Create Directory Application_App_Checkout
md Application_App_Checkout
::Into Directory Application_App_Checkout
cd Application_App_Checkout
tf workspace /delete %workSpacename% /noprompt
tf workspace /new %workSpacename% /noprompt /collection:%tfsURL% /login:%userName%,%password%
tf.exe get %tfsAddress% /recursive /login:%userName%,%password%
GOTO End
:NotDownloadTFSCode
echo "User does not wish to download latest code from TFS."
Pause
GOTO End
End Local

What is Risk-Based Testing?

2012-06-25T00:58:00.000-07:00

Risk-based testing (RBT) is a type of software testing that prioritizes the tests of features and functions based on the risk of their failure - a function of their importance and likelihood or impact of failure. In theory, since there is an infinite number of possible tests, any set of tests must be a subset of all possible tests. Test techniques such as boundary value analysis and state transition testing aim to find the areas most likely to be defective.

Assessing risks:

The changes between two releases or versions is key in order to asses risk. Evaluating critical business modules is a first step in prioritizing tests, but it does not include the notion of evolutionary risk. This is then expanded using two methods: change-based testing and regression testing.

Change-based testing allows test teams to assess changes made in a release and then prioritize tests towards modified modules.
Regression testing ensures that a change, such as a bug fix, did not introduce new faults into the software under test. One of the main reasons for regression testing is to determine whether a change in one part of the software affects other parts of the software.

These two methods permit test teams to prioritize tests based on risk, change and criticality of business modules. Certain technologies can make this kind of test strategy very easy to set-up and to maintain with software changes.

Types of Risks

Risk can be identified as the probability that an undetected software bug may have a negative impact on the user of a system.
The methods assess risks along a variety of dimensions:

Business or Operational

High use of a subsystem, function or feature
Criticality of a subsystem, function or feature, including the cost of failure

Technical

Geographic distribution of development team
Complexity of a subsystem or function

External

Sponsor or executive preference
Regulatory requirements

E-Business Failure-Mode Related

Static content defects
Web page integration defects
Functional behavior-related failure
Service (Availability and Performance) related failure
Usability and Accessibility-related failure
Security vulnerability
Large Scale Integration failure

Visual Studio 2010 Coded UI

2012-06-24T22:53:00.003-07:00

A coded UI test class is identified by a CodedUITestAttribute applied to the class.
Each coded UI test is a test method in a coded UI test class. You can add multiple test methods to each coded UI test class and identify each coded UI test method by using the TestMethodAttribute.
Your test method can also add validation code for a UI test control to obtain the value of a property of a UI test control. The test method can use an Assert statement to compare the actual value of the property to an expected value. The result of this comparison determines the outcome of the test. Every time that you run a coded UI test, you can analyze the test result and if the test fails, you can see or store the details of which assertion failed.
When you create a coded UI test, these files are added to your test project:

File	Description
CodedUITest1.cs	Contains the coded UI test class, test methods and assertions.
UIMap.uitest	Contains the XML model for the UIMap class, including all windows, controls, properties, methods, parameters, actions, and assertions.
UIMap.Designer.cs	Contains the code representation of the XML contained in the UIMap.uitest file. Do not edit this file.
UIMap.cs	Contains more of the code for the UIMap class. You can put any customizations for the UI map in this file.

These assemblies are added as references to your test project:

Microsoft.VisualStudio.QualityTools.CodedUITestFramework
Microsoft.VisualStudio.QualityTools.UnitTestFramework
Microsoft.VisualStudio.TestTools.UITest.Common
Microsoft.VisualStudio.TestTools.UITest.Extension
Microsoft.VisualStudio.TestTools.UITesting

Automation Testing / Visual Studio 2010 CodedUI

2012-06-24T22:40:00.001-07:00