Software Engineering - Performance Automation

Basic SpecFlow Automation tests in visual studio express using C# ,Selenium webdriver,nUnit console

2013-10-19T20:14:00.001+01:00

Specflow is layer that would sit on the top of C# and one can specify tests cases in normal English readable format and build automation using these test cases. Writing test cases in format that is easily readable enables appropriate reporting and the direct mapping user story into automated test cases.Instead of digging down what test case is doing, here from reading the test case itself, it gives understanding of what it is about and what it is doing. Thus enables one to describe a test case on how the desired behaviour should be specified.
Using above behaviour specification structure one can write a story like
Feature: Google Search
As an end user,
I would like to visit the google search page
And then I would like to search an item so that
I can view the search results
Scenario: Google Search
Given I am on the Google home page
When I search for text Webdriver
Then I should see the search results with title WebDriver – Google Search

Now the above defined test cases needs to glued to test automation code such that each of above Given When Then steps would get executed and make appropriate calls to application (browser here). Specflow helps to achieve this gluing of the code to BDD structure test cases. Lets see how we can do this using C#, selenium webdriver and nUnit. If you have not done setup of these tools, in my previous blogs I had describe how to setup Visual studio express with nUnit, selenium (or refer my another blog where I have explained how to install NuGet and download selenium/nunit etc)and how to create basic script. You can refer these blogs if you have not done the setup and want primary understanding on Selenium/C# and nUnit work together.

1. Setting up specflow and generate C# file having gluing code

Download Specflow binaries from https://github.com/techtalk/SpecFlow/downloads which will have name in form of Specflow_V<versionnumber>_bin.Zip. Extract this file into location like “C:\Data\seleniumCsharp\”. Once extracted, we can now add the path of binary exe file in PATH variable of windows environment. For this go to Start –> Rightclick on Computer –> Select properties. Next Click on Advance settings . If you are using windows 7, you will be prompted to enter admin password. Next click on Environment variables. Under System variables edit the PATH variable by appending the path something like “C:\Data\seleniumCsharp\SpecFlow_v1.9.0_bin\tools\” with semi colons as separator
After this open a new command window and enter specflow and then you should see following output in command window which confirms specflow exe is been located using system PATH
C:\>SpecFlow
usage: specflow <subcommand> [args]
Type 'specflow help <subcommand>' for help on a specific subcommand.
Available subcommands:
generateall - Generate tests from all feature files in a project
stepdefinitionreport - Generates a report about usage and binding of steps
nunitexecutionreport - Formats an NUnit execution report to SpecFlow style
mstestexecutionreport - Formats an MsTest execution report to SpecFlow style

From above output , there are two subcommands which we are interested - generateall and nunitexecutionreport. Using subcommand generateall lets try to create step definition class file in C#.

Go to Visual studio express(if you dont have one download it from Download Visual studio express for Windows Desktop), and create file –New project – SpecflowTutorial.
Open a new project of type Class library. Open Tools –>Library Package Manager Console > Package Manager Console to open NuGet. In the powershell command window type following command to download the packages
Install-Package SpecFlow
Install-Package SpecFlow.NUnit
Install-Package Selenium.WebDriver
Install-Package Selenium.Support
If you do not have NuGet refer my previous blog where I have explained how to install NuGet and download selenium/nunit etc
Next copy the feature and scenario marked in bold into a txt file and rename its extenstion from “.txt” to “.feature” (for example Google.feature). Save this file in project directory which is same location as .csproj file is stored for your project. Then in solution explorer of VS Express add this file by right clicking project –> Add –> Existing item. Change the filter to All files and you should see the new feature file. Then select the file and click add. Next save the project as SpecflowTutorial.
First I will show you how to create a step definition file using the specflow command and then we can create a batch file that can be executed from external tools which helps us to generate step definitions in just one click. Navigate to project directory where your .csproj and .feature files are stored and copy the path.
Next open a command window and type

specflow generateall <ProjectPath>\<Projectname.csproj> /force /verbose
where <Project Path> is the path that you copied in previous step and <Projectname.csproj> is the name of the project and click enter. You should get following output in command window

Processing project: SpecflowTutorial
Google.feature -> test updated

After this click on project in solution explorer –> Add –> Existing item and select All files filter . You should see a file with same name a feature but .cs extension. Select this file and click Add. Save the project.
Open up this .cs file which has feature file name a prefix (for example Google.feature.cs) and you can see a gluing code that joins the feature file. This file looks like as below.

// ------------------------------------------------------------------------------//  <auto-generated>//      This code was generated by SpecFlow (http://www.specflow.org/).//      SpecFlow Version:1.9.0.77//      SpecFlow Generator Version:1.9.0.0//      Runtime Version:4.0.30319.18052// //      Changes to this file may cause incorrect behavior and will be lost if//      the code is regenerated.//  </auto-generated>// ------------------------------------------------------------------------------#region Designer generated code
#pragma warning disable
namespace SpecflowTutorial
{
    using TechTalk.SpecFlow;
    
    
    [System.CodeDom.Compiler.GeneratedCodeAttribute("TechTalk.SpecFlow", "1.9.0.77")]
    [System.Runtime.CompilerServices.CompilerGeneratedAttribute()]
    [NUnit.Framework.TestFixtureAttribute()]
    [NUnit.Framework.DescriptionAttribute("Google Search")]
    public partial class GoogleSearchFeature
    {
        
        private static TechTalk.SpecFlow.ITestRunner testRunner;
        
#line 1 "Google.feature"#line hidden
        
        [NUnit.Framework.TestFixtureSetUpAttribute()]
        public virtual void FeatureSetup()
        {
            testRunner = TechTalk.SpecFlow.TestRunnerManager.GetTestRunner();
            TechTalk.SpecFlow.FeatureInfo featureInfo = new TechTalk.SpecFlow.FeatureInfo(new System.Globalization.CultureInfo("en-US"), "Google Search", "As an end user,\r\nI would like to visit the google search page\r\nAnd then I would l" +
                    "ike to search an item so that\r\nI can view the search results", ProgrammingLanguage.CSharp, ((string[])(null)));
            testRunner.OnFeatureStart(featureInfo);
        }
        
        [NUnit.Framework.TestFixtureTearDownAttribute()]
        public virtual void FeatureTearDown()
        {
            testRunner.OnFeatureEnd();
            testRunner = null;
        }
        
        [NUnit.Framework.SetUpAttribute()]
        public virtual void TestInitialize()
        {
        }
        
        [NUnit.Framework.TearDownAttribute()]
        public virtual void ScenarioTearDown()
        {
            testRunner.OnScenarioEnd();
        }
        
        public virtual void ScenarioSetup(TechTalk.SpecFlow.ScenarioInfo scenarioInfo)
        {
            testRunner.OnScenarioStart(scenarioInfo);
        }
        
        public virtual void ScenarioCleanup()
        {
            testRunner.CollectScenarioErrors();
        }
        
        [NUnit.Framework.TestAttribute()]
        [NUnit.Framework.DescriptionAttribute("Google Search")]
        public virtual void GoogleSearch()
        {
            TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo("Google Search", ((string[])(null)));
#line 7
this.ScenarioSetup(scenarioInfo);
#line 8
testRunner.Given("I am on the Google home page", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Given ");
#line 9
testRunner.When("I search for text Webdriver", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "When ");
#line 10
testRunner.Then("I should see the search results with title WebDriver – Google Search", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Then ");
#line hidden
            this.ScenarioCleanup();
        }
    }
}
#pragma warning restore
#endregion

It is not necessary to understand the complete code for time being, but few things to point out. You can see in method name GoogleSearch(), there are method calls with name testRunner.Given, testRunner.When and testRunner.Then. The text inside these method calls is the text we need to use when we create step definitions

2. Writing step definitions

Click on Project in solution explorer and right click – > Add –> new class file. Name this class file as TutorialStepDef.cs. Then save the project.

Open the TutorialStepDef.cs add following text in the top of code to existing “Using” code lines

using TechTalk.SpecFlow;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Support.UI;

Create a class name as TutorialStepDef as shown below

[Binding]
public class StepDefinition1
{ }

Inside this class we shall write a method that binds to call “Given I am on the Google home page”. If you look at Google.feature.cs file the testrunner call has the text “I am on the Google home page” and method name as Given. This method would look like this below with attribute as [Given]. The text that is passed to Given attribute should match with the one in testruuner call of Google.feature.cs file.

namespace SpecflowTutorial
{
    [Binding]
    class TutorialStepDef
    {
        [Given(@"I am on the Google home page")]
        public void GivenIAmOnTheGoogleHomePage()
        {
            driver.Navigate().GoToUrl("http://www.google.com");
            Assert.AreEqual("Google", driver.Title);
        }
    }
}

Similarly one has to write the code to When and Then steps. The final code in TutorialStepDef .cs file is as shown below.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using TechTalk.SpecFlow;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Support.UI;

namespace SpecflowTutorial
{
    [Binding]
    class TutorialStepDef
    {
        IWebDriver driver = new InternetExplorerDriver(@"C:\Data\seleniumCsharp\Drivers\");

        [Given(@"I am on the Google home page")]
        public void GivenIAmOnTheGoogleHomePage()
        {
            
            driver.Navigate().GoToUrl("http://www.google.com");
            Assert.AreEqual("Google", driver.Title);
        }

        [When(@"I search for text Webdriver")]
        public void WhenISearchForATextSelenium()
        {
            // Find the text input element by its name
            IWebElement query =driver.FindElement(By.Name("q"));
            // Input the search text
            query.SendKeys("Webdriver");
            // Now submit the form
            query.Submit();

            // Google's search is rendered dynamically with JavaScript.
            // Wait for the page to load, timeout after 5 seconds

            WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(5));
            IWebElement title = wait.Until<IWebElement>((d) =>
            {
                return d.FindElement(By.ClassName("ab_button"));
            });
        }

        [Then(@"I should see the search results with title WebDriver – Google Search")]
        public void ThenIShouldSeeTheSearchResultsForSelenium()
        {
            String title = "Webdriver - Google Search";
            //Check that the Title is what we are expecting
            Assert.AreEqual(title.ToString(), driver.Title);
        }
    }
}

Then next build the project and there should not be any build errors. Now how to run the test in nUnit.

3. Execute test in nUnit and generate specflow report.

nUnit should be installed in you machine and the Path variable of system should have nUnit bin directory. If this has been done, then in any new command window on typing nUnit and pressing enter should open up nUnit. If this is not the case, you can refer my previous blog on nUnit

Lets try to run tests through commandline first. Open up a command window and navigate to bin directory of the project. And then type nunit-console SpecflowTutorial.dll . This should start a IE browser and do the steps. Below output is seen in command window.

Runtime Environment -
OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1
CLR Version: 2.0.50727.5472 ( Net 3.5 )

ProcessModel: Default DomainUsage: Single
Execution Runtime: net-3.5
.Given I am on the Google home page
-> done: TutorialStepDef.GivenIAmOnTheGoogleHomePage() (6.3s)
When I search for text Webdriver
-> done: TutorialStepDef.WhenISearchForATextSelenium() (4.0s)
Then I should see the search results with title WebDriver - Google Search
-> done: TutorialStepDef.ThenIShouldSeeTheSearchResultsForSelenium() (0.0s)

Tests run: 1, Errors: 0, Failures: 0, Inconclusive: 0, Time: 11.2006407 seconds
Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

Now you have successfully run the Specflow test. But the report from the command window is not sufficient.There is XML file generated by nUnit which can be formatted into specflow report. This where we are going to use second subcommand option of specflow – nunitexecutionreport. For this command one has pass the project file and the xml file as parameters. In command window navigate back to project directory where .csproj file is located. And then enter following command in command window

specflow.exe nunitexecutionreport SpecflowTutorial.csproj /xmlTestResult:.\bin\Debug\TestResult.xml

After this command is successfully executed , there is html file created in the project directory which is as shown in below snapshot.

4. Automate generation of nunit testrunner file having step defintion.

Thus so far you have created a sample scenario in Given,When and Then format and execute a test case in specflow plus generated report. But there were many steps that required you to hop in and out of Visual studio and command line. To avoid all these we can automate the generation of binding file from feature file and execution of nunit console/report generation using external tool feature visual studio express. Below step automates generation of *.feature.cs files.

Create a batch file say Generatesteps.bat and add the below content. And then save it.

echo Starting to generate SpecFlow unit tests
SpecFlow generateAll %1 /force /verbose
echo Finished generating SpecFlow unit tests

Open Tools –> External Tools and Click Add.
Give title as Generate Steps
In Command window, browse to the GenerateSteps.bat file and select it.
In Arguments type $(ProjectFileName)
In Initial Directory type $(ProjectDir) and check option Use Output window.Click Apply and Ok

Now this external tool is setup and can be accessed from Tools –> Generate Steps. So instead of running a command line like specflow generateall <ProjectPath>\<Projectname.csproj> /force /verbose , you can just click this tool and it automatically picks up all the feature files you created in the project directory and generate *.feature.cs files

5. Automate Execution of Specflow tests in nUnit and generation of report

Create a batch file and name it as RunSpecflowtests.bat. Add the below content to this batch file and save it

@echo off
nunit-console %1
if NOT %errorlevel% == 0 (
    echo "Error running tests - %errorlevel%"
    GOTO :exit
)
specflow.exe nunitexecutionreport %2 /xmlTestResult:%3
if NOT %errorlevel% == 0 (
    echo Error generating report - %errorlevel%
    GOTO :exit
)
if %errorlevel% == 0 TestResult.html
:exit

Go to Tools –> External tool and click Add
In Title give it as Run Specflow tests.
In Command, browse the batch you just created and select it.
In Arguements box, type “$(TargetName)$(TargetExt) $(ProjectDir)$(ProjectFileName) $(BinDir)TestResult.xml”.
In Initial Directory type “ $(BinDir)”. And select option Use Output window. Then click apply and Okay.
Once you have build the project instead of running command lines to execute nUnit-Console and specflow , you can just click Tools->Run Specflow tests and it should execute the tests and generate the Html file also.

In this test case, currently in the first method – Given, we are creating a driver. But best approach is create webdriver when feature gets initiated and close the browser when feature closes. I shall explain about this in further blogs.

How to invoke nUnit from Visual studio express and execute automation tests

2013-10-06T18:12:00.002+01:00

nUnit is Unit testing framework tool for testing C# unit test cases. With the invent of selenium webdriver, this C# code intergrated with nUnit framework can be used to run acceptance tests. Simple way to integrate commands into Visual studio express to call nUnit tests is to use external tool feature.
Before that lets explore what options has nUnit has when you run it from a command line. Type ‘nUnit /help’ in command window which is in the directory where nUnit bin is pointing to, then it pops up below window .

Basically it has two types of argument. First is the name of inputfile which is the DLL file that you want to run. Second is the number of different options.
So if you type “nUnit <Projectname>.dll /run, this basically would open up nUnit and the project dll and start executing. This is exactly we want, but how to use this command from visual studio to avoid all other headache of entering path etc. That is by using external tool option and with just one click you can get nUnit tests executed from visual studio.
In visual studio express, go to Tools –> External tools. This Opens up a pop window with option to create a new external tool. Click on Button to Add this new tool and enter the options as shown in the screen shot

1. Enter title as Execute nUnit
2. In Command, click the button and browse to directory where nUnit.exe is stored and select it.
3. In Arguments text box , Enter the text as shown in the screenshot. “${BinDir}${TargetName}.dll /run”
4. In Initial director enter ${BinDir}
5. Select option prompt for arguments so that you can add additional options like categories if want to include. Then Apply and Ok.
After you have any nUnit tests created and complied, all you need to do is go to tools –>and click Execute nUnit (Click Okay on the prompt) and this would start nUnit and nUnit would load the dll file of the project you are on visual studio express and then start executing tests.
To add more flavor to above, say you require to run a particular tagged test instead of all the test cases in the dll. For that you need not go back to defining tool. As we chosen option to prompt for arguments, when you click Execute nUnit, the pop window allows you add the /include the tags you want or exculde the tags that you dont want to run as shown below. I have include the tests with a Tag attribute “NewTest”. The syntax of the argument that needs to be appended is
/Include:NewTest
or
/Exclude:Newtest

Capturing screen shots using remote/local webdriver in C#/Selenium webdriver automation test cases

2013-10-06T16:17:00.002+01:00

This blog provides details on how can you capture a screenshot using webdriver. You may run webdriver as remote webdriver or local webdriver . Both the webdriver has a method called GetScreenshot() which captures the screenshot of the browser’s current state. In tests it would be useful to capture the screen shot just to make sure the test has executed successfully or capture screenshots when error occurs. So this blog will provide you information on how basically one needs to make calls to these methods and you can use this code or learning from this code to implement it your framework.

Local Webdriver
Lets first look at local webdriver and how can we capture the screenshot. To run this code, you need to include package named system.Drawing. Thus you the top most part of your code is as below

using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Support;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Chrome;
using NUnit.Framework;
using System.Drawing;

The line of code to capture screenshot is

Screenshot ss = ((ITakesScreenshot)driver).GetScreenshot();

From the above instance of screenshot object ss , code has to be written to save the file. For this you can use SaveAsFile method of screenshot object as shown below code. If you don’t provide the path, then file will be saved in default directory which is the directory where the dll is stored

ss.SaveAsFile("Localexplorer.jpeg", System.Drawing.Imaging.ImageFormat.Gif);

The test method having this implementation looks like this below

[Test]
public void Test_ScreenShotLocalwebdriver()
{
    // Step b - Initiating webdriver
    IWebDriver driver = new InternetExplorerDriver();
    //Step c : Making driver to navigate

    driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");
    //Take the screenshot
    Screenshot ss = ((ITakesScreenshot)driver).GetScreenshot();
    //Save the screenshot
    ss.SaveAsFile("Localexplorer.jpeg", System.Drawing.Imaging.ImageFormat.Gif);

    //Step d 
    IWebElement myLink = driver.FindElement(By.LinkText("Download"));
    myLink.Click();


    //Step e
    driver.Quit();

}

Remote webdriver

For the remote webdriver, the same line of code using GetScreenshot() does not work and the complier complains about wrong typecasting. You may get error such as “

System.InvalidCastException : Unable to cast object of type 'OpenQA.Selenium.Remote.RemoteWebDriver' to type 'OpenQA.Selenium.ITakesScreenshot”

It can be overcome by creating a new class derived from orignial remotewebdriver and interface ITakesScreenShot. In this new drived class the constructor would use the base constructor , but the method getscreenshot can be overidden with new method. This derived class would like this as shown below

    public class ScreenShotRemoteWebDriver : RemoteWebDriver, ITakesScreenshot
    {
        public ScreenShotRemoteWebDriver(Uri RemoteAdress, ICapabilities capabilities)
            : base(RemoteAdress, capabilities)
        {
        }

        public Screenshot GetScreenshot()
        {
            // Get the screenshot as base64. 
            Response screenshotResponse = this.Execute(DriverCommand.Screenshot, null);
            string base64 = screenshotResponse.Value.ToString();
            // ... and convert it. 
            return new Screenshot(base64);
        } 
    }

The complete class file below shows example of two tests . One of them uses local driver and other one uses remote webdriver

using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Support;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Chrome;
using NUnit.Framework;
using System.Drawing;   


namespace NUnitSelenium
{
    [TestFixture]
    public class UnitTest1
    {

        [SetUp]
        public void SetupTest()
        {
        }
        [Test]
        public void Test_ScreenShotRemoteBrowser()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new ScreenShotRemoteWebDriver(new Uri("http://127.0.0.1:4444/wd/hub"), DesiredCapabilities.InternetExplorer());
            //Step c : Making driver to navigate
            
            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");
            //Take the screenshot
            Screenshot ss= ((ITakesScreenshot)driver).GetScreenshot();
            //Save the screenshot
            ss.SaveAsFile("Remote browser.jpeg", System.Drawing.Imaging.ImageFormat.Gif);

            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();
            
            
            //Step e
            driver.Quit();

        }

        [Test]
        public void Test_ScreenShotLocalwebdriver()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new InternetExplorerDriver();
            //Step c : Making driver to navigate

            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");
            //Take the screenshot
            Screenshot ss = ((ITakesScreenshot)driver).GetScreenshot();
            //Save the screenshot
            ss.SaveAsFile("Localexplorer.jpeg", System.Drawing.Imaging.ImageFormat.Gif);

            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();


            //Step e
            driver.Quit();

        }

    }

    public class ScreenShotRemoteWebDriver : RemoteWebDriver, ITakesScreenshot
    {
        public ScreenShotRemoteWebDriver(Uri RemoteAdress, ICapabilities capabilities)
            : base(RemoteAdress, capabilities)
        {
        }

        public Screenshot GetScreenshot()
        {
            // Get the screenshot as base64. 
            Response screenshotResponse = this.Execute(DriverCommand.Screenshot, null);
            string base64 = screenshotResponse.Value.ToString();
            // ... and convert it. 
            return new Screenshot(base64);
        } 
    }

}

Complie the code and then run it in the nUnit . Refer my blog on How to invoke nUnit from Visual studio express and execute tests
At the time writing this blog, for some reason, google chrome driver was not capturing full website snapshot while IE was doing full website page snapshot.

How to invoke/run different type of selenium web driver browser using remote webdriver in C# automation scripts

2013-10-06T00:09:00.002+01:00

Remote web driver feature in selenium is very advantages and helps to achieve distributed testing. Distributed testing comprises testing different browsers on different machine and simultaneous run of tests on number of machines. As a basic lets try to understand how we can we invoke a test case in different browser that is located on different machine using remote webdriver. Declaration for remote webdriver is as below for c# code

IWebDriver driver = new RemoteWebDriver(new Uri("http://127.0.0.1:4444/wd/hub"), DesiredCapabilities.InternetExplorer());

The first parameter Uri is the remote address on which selenium is running. In the above example it is running on machine with IP 127.0.0.1(i am running it locally) on port 4444. The second parameter is the capabilities of the browser you going to use. To have successful execution it requires to run selenium on the remote machine and that instance selenium should know where the browser driver are located. This can achieved by using parameter –DWebdriver.ie.driver and assigning it to that of IE driver . Similarly for one can use –DWebdriver.chrome.driver for Chrome driver. Below are the steps and command to be executed to achieve this

1. Open a new command prompt window

2. Navigate to the directory where you have selenium standalone server jar is downloaded .(if you dont have one already you can download from link
3. Then run the following command in the command window ( the path value shown in blue has to replaced with the path of the drivers where you stored them)

C:\Data\seleniumCsharp\Drivers>java -jar selenium-server-standalone-2.35.0.jar -Dwebdriver.chrome.driver=C:\Data\seleniumCsharp\Drivers\chromedriver_win32_2.3\c
hromedriver -Dwebdriver.ie.driver=C:\Data\seleniumCsharp\Drivers\IEDriverServer.
exe

This should get the selenium server up and running.One can fire requests from different machine using IP address of machine thats get listed when run ipconfig in command window, alternatively you can fire request from same machine using IP 127.0.0.1. .In this case although selenium standalone server is running on same machine, from your scripts if you access it via IP address 127.0.0.1, it mimics the same process as if you were accessing a standalone server on different machine. Once you started the server, you can check if it is running by opening this url http://127.0.0.1:4444/wd/hub (or from different machine using http://<machineIP>/wd/hub) and it should not throw any error.

So you have so far have setup selenium which can recieve commands through IP 127.0.0.1 and port 4444. Next lets create a class library project in Visual studio express. Say you name it as LanuchRemoteBrowser. The below code shows the declaration of both types of browser and how to invoke it with desired capabilities.

using System;
//Step ausing OpenQA.Selenium;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Chrome;
using NUnit.Framework;
using OpenQA.Selenium.Remote;

namespace LanuchRemoteBrowser
{
    [TestFixture]
    public class UnitTest1
    {              [SetUp]
        public void SetupTest()
        {
        }  
        //Test that uses IE Driver
        [Test]
        public void Test_OpeningHomePageUsingIE()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new RemoteWebDriver(new Uri("http://127.0.0.1:4444/wd/hub"), DesiredCapabilities.InternetExplorer());
            //Step c : Making driver to navigate
            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");       
            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();
            //Step e
            driver.Quit();

        }
                 
         // Test that uses Google chrome driver
         [Test]
         public void Test_OpeningHomePageUsingChrome()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new RemoteWebDriver(new Uri("http://127.0.0.1:4444/wd/hub"),DesiredCapabilities.Chrome());
            //Step c : Making driver to navigate
            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");
            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();
            //Step e
            driver.Quit();
            }
         }
    }

Before compiling this code, you need to reference the appropriate dlls from selenium and nunit. For traditional method of doing this, you can refer my previous blog - How to setup C#,nUnit and selenium client drivers on VSExpress for Automated tests. Alternatively there in another method of adding packages or references to you project using NuGet. Installation of NuGet is straight forward and can installed from Tools > Extentions and Updates. On Extension and Updates window, click on Online –> visual studio gallery and then in search box enter Nuget. Then from list, select NuGet Package manager and install it.

Now from Tools –> library package manager, click on Package Manager console.

From package manager power shell windows run following commands to add the dependencies/packages using NuGet

1. Install-Package Selenium.WebDriver
2. Install-Package Selenium.Support
3. Install-Package Selenium.nUnit

Once you run the above commands you would be having the output in powershell window as shown below

Next compile the above code by clicking Build > Build solution and open the dll created in debug directory after compilation via nUnit to run the tests.

How to invoke locally different types of browser web driver using selenium and c# for test Automation scripts

2013-10-02T23:25:00.000+01:00

This blog is in sequence to How to blogs related Selenium in C#. In previous blog of this series , we had created a basic web driver test . Different browsers have different webdrivers and invoking them is all nearly similar but there are slight changes.Of three main browsers – Firefox, Google Chrome and IE – firefox comes inbuilt. Hence in the test written in previous blog we could straight away start using firefox browser without downloading driver file.
As seen ,the statement below would be all just necessary to create instance of firefox browser.

IWebDriver driver = new FirefoxDriver();

The same statement for IE or Google chrome would not work as the driver files for these drivers are separate files and do not come embedded with selenium jar. For this one needs to download drivers for IE ( 32 bit or 64 bit ) and Google chrome . These are available from selenium at this location

Save them in desktop location something like C:\WebDrivers. After that they can be invoked by passing IE or Google chrome driver path as shown below

IWebDriver driver = new InternetExplorerDriver(@"C:\WebDrivers\IEDriverServer.exe");
IWebDriver driver = new ChromeDriver (@"C:\WebDrivers\chromedriver.exe");

Using above syntax for passing paths of drivers during driver creation, we can write two simple c# selenium tests in visual studio express as below

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
//Step a
using OpenQA.Selenium;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Chrome;
using NUnit.Framework;

namespace NUnitSelenium
{
    [TestFixture]
    public class UnitTest1
    {      

        [SetUp]
        public void SetupTest()
        {
        }
        
        //Test that uses IE Driver
        [Test]
        public void Test_OpeningHomePageUsingIE()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new InternetExplorerDriver(@"C:\WebDrivers\IEDriverServer.exe");

            //Step c : Making driver to navigate
            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");
          
            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();
     
            //Step e
            driver.Quit();

            )
         }
         
         // Test that uses Google chrome driver
         [Test]
         public void Test_OpeningHomePageUsingChrome()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new ChromeDriver (@"C:\WebDrivers\chromedriver.exe");;

            //Step c : Making driver to navigate
            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");
          
            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();
     
            //Step e
            driver.Quit();

            )
         }
    }

Another way in windows is to append Path variable with path of drivers. On how to change the path, you can refer link –> How to change Path

Once done, you can test this is working by running a new command window. And it command window just type in IEDriverServer.exe. Then you should get the following out put

You can test the same for google chrome and it should work. Once this setup is done, then on the computer where this PATH has been set, we need to have the code pointing to location of driver exe files, instead then can just have no arguments . Then the tests would like as below

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
//Step a
using OpenQA.Selenium;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Chrome;
using NUnit.Framework;

namespace NUnitSelenium
{
    [TestFixture]
    public class UnitTest1
    {      

        [SetUp]
        public void SetupTest()
        {
        }
        
        //Test that uses IE Driver
        [Test]
        public void Test_OpeningHomePageUsingIE()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new InternetExplorerDriver();

            //Step c : Making driver to navigate
            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");       
    
            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();
     
            //Step e
            driver.Quit();

            )
         }
         
         // Test that uses Google chrome driver
         [Test]
         public void Test_OpeningHomePageUsingChrome()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new ChromeDriver ();;

            //Step c : Making driver to navigate
            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");
          
            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();
     
            //Step e
            driver.Quit();

            )
         }
    }

Previous Blogs related selenium webdriver with C#
1. How to setup C#,nUnit and selenium client drivers on VSExpress for Automated tests
2. Creating Basic Selenium web driver test case using Nunit and C#

Quality Center test instances /test cases that are blocked and corresponding linked defects

2013-09-03T23:17:00.001+01:00

One time or the other during the test execution phase, one comes across a question on how to find the test cases that are blocked by defect. This is key because

1. It helps to identify the defects that are blocking the test cases and highlight them to defect management teams

2. It helps to identify any defects that are closed recently and would have been inappropriately linked to a test case.

To determine this result, a SQL query can be executed in Quality center to retrieve the results. Generally any user can execute this query in Quality center by feature called excel query. Before moving onto this excel query , few points needs to be understood

Difference between test case and test instances : A single test case can be extracted into multiple test sets and also within same test set. If one pulls multiple tests of same test case into a test set, this becomes test instances of that particular test case.

Linked defect : This is the defect that is linked to a particular test instance . For instance multiple defects can be linked to many test cases or multiple test instance can be linked to many test cases.

*Note - Defects in Quality center can be linked at many levels to different entities.Such as one can link a defect to requirement and then change the status to blocked, etc. In this blog, I am referring to linked defects that are linked to test instances in the test set. This is generally many people follow during the test execution

In Quality center, the BUG ID for the linkage is stored in table called link. Bugs can be linked to test case, test instances, and requirements etc. These are called entities in quality center. As we are trying to find linked defects to test instances, the entity type for us in question is “TESTCYCL” (the table that holds test instances)
Below is the ER diagram for LINK table and BUG table.
.
TESTCYCL table consisting test instances is linked to TEST table (that holds test cases) by common field TEST_ID. Using these information we can construct the query as below which retrieves test cases, corresponding test instances and linked defect.

SELECT T.TS_NAME, TC.TC_TEST_INSTANCE,LN_BUG_ID FROM
TEST T, TESTCYCL TC , LINK L
WHERE
T.TS_TEST_ID=TC.TC_TEST_ID -- Condition joining TEST (testcases) and TESTCYCL (Test instances) table
AND L.LN_ENTITY_TYPE='TESTCYCL' -- This condition makes sure that defects linked to test instaces only need to retrieved
AND TC.TC_TESTCYCL_ID=L.LN_ENTITY_ID -- Condition to join LINK and TESTCYCL table
AND TC.TC_STATUS = 'BLOCKED'

In addition to above, it may be also necessary to know whether those defects are closed are not. Now that means in addition to retrieval of result from above query, you would also like to get the status of the defect . For this it requires to join BUG table. The below listed is the query that will also retrieve status of the defect.

SELECT T.TS_NAME, TC.TC_TEST_INSTANCE, LN_BUG_ID,B.BG_STATUS FROM
TEST T, TESTCYCL TC , LINK L ,BUG B
WHERE
T.TS_TEST_ID=TC.TC_TEST_ID
AND L.LN_ENTITY_TYPE='TESTCYCL'
AND LN_BUG_ID = BG_BUG_ID
AND TC.TC_TESTCYCL_ID=L.LN_ENTITY_ID
AND TC.TC_STATUS = 'BLOCKED'

Apart from the above cases, you might also be interested to retrieve those test instances which are assigned to closed defects so that you can set the status back appropriately and remove the linkage. Below query that helps to achieve that

SELECT T.TS_NAME, TC.TC_TEST_INSTANCE, LN_BUG_ID,TC.TC_STATUS FROM
TEST T, TESTCYCL TC , LINK L ,BUG B
WHERE
T.TS_TEST_ID=TC.TC_TEST_ID
AND L.LN_ENTITY_TYPE='TESTCYCL'
AND LN_BUG_ID = BG_BUG_ID
AND TC.TC_TESTCYCL_ID=L.LN_ENTITY_ID
AND B.BG_STATUS = 'Closed'

The other useful blogs related excel queries, Audit tables and QC history can be found in following locations

First time Fix faliures or Reopen defects

How to retrieve Quality Center bug details and Audit tables information

How to retrieve QC status history for closed defect

How to retrieve test instances and test sets within particular folder

How to retrieve requirements and test coverage and linked test cases

Quality center defect history – To track the defects that have undergone severity change

2013-08-25T19:56:00.000+01:00

This blog is in response to comments from one of my previous blog. I was away from quality centre for long time and had not got the oppurtunity to reply to the comment. Now I feel I should blog about this if it could help someone else and I know how it can be done.

As mentioned in my previous blogs, history of any entity in the QC can be retrieved from AUDIT_PROPERTIES and AUDIT_LOG tables. In the previous blog QC History for closed defect – Status History, there is explanation on how to join these tables. I will quickly jump to main of query on how to retrieve defects that have undergone severity change

To find this, you need to basically pick defects that have changed from one status to another status. This is recorded in AUDIT_PROPERTIES table as AP_OLD_VALUE and AP_NEW_VALUE. So if defect severity is changes the AP_OLD_VALUE will have old value and AP_NEW_VALUE will have new value. If the severity is just set for first time then , AP_OLD_VALUE is null. Thus we need to find defects that do not have NULL value in AP_OLD_VALUE

AP.AP_OLD_VALUE IS NOT NULL

Since we are interested in the defects that have undergone change for severity, we need to put a where condition that just filters out defects having severity change.

AP.AP_FIELD_NAME = ‘BG_SEVERITY’

Using these two above conditions, you can arrive at the query as shown below that will retrieve all defects which have undergone change in severity levels.

SELECT B.BG_BUG_ID,AL.AU_TIME ,AL.AU_USER ,AP.AP_OLD_VALUE  as OLD_STATUS ,AP.AP_NEW_VALUE AS NEW_STATUS FROM BUG B
INNER JOIN AUDIT_LOG AL
ON AL.AU_ENTITY_ID = B.BG_BUG_ID
INNER JOIN AUDIT_PROPERTIES AP
ON AL.AU_ACTION_ID = AP.AP_ACTION_ID
AND AL.AU_ENTITY_TYPE = 'BUG'
AND AL.AU_ACTION_ID= AP.AP_ACTION_ID
AND AP.AP_FIELD_NAME = 'BG_SEVERITY'
AND AP.AP_OLD_VALUE IS NOT NULL
ORDER BY AU_TIME DESC

Below is the snapshot showing the output

Another user of this query is , say if it is required to extract the defects that have undergone severity change in last three months, then all you need to do this add one more condition to where clause as shown in below and and you should retrieve the results as expected.

SELECT B.BG_BUG_ID,AL.AU_TIME ,AL.AU_USER ,AP.AP_OLD_VALUE  as OLD_STATUS ,AP.AP_NEW_VALUE AS NEW_STATUS FROM BUG B
INNER JOIN AUDIT_LOG AL
ON AL.AU_ENTITY_ID = B.BG_BUG_ID
INNER JOIN AUDIT_PROPERTIES AP
ON AL.AU_ACTION_ID = AP.AP_ACTION_ID
AND AL.AU_ENTITY_TYPE = 'BUG'
AND AL.AU_ACTION_ID= AP.AP_ACTION_ID
AND AP.AP_FIELD_NAME = 'BG_SEVERITY'
AND AP.AP_OLD_VALUE IS NOT NULL
AND AL.AU_TIME > '20130501'
ORDER BY AU_TIME DESC

The other useful blogs related to Audit tables and QC history can be found in following locations

First time Fix faliures or Reopen defects

How to retrieve Quality Center bug details and Audit tables information

How to retrieve QC status history for closed defect

How to retrieve test instances and test sets within particular folder

How to retrieve requirements and test coverage and linked test cases

QC history for closed defect – Status History

2013-08-23T14:25:00.000+01:00

For some defects one would like to know the status history of defect. This can be easily found in the QC when you click on particular defect and then click history tab. But if you want to have this information for multiple defects on single sheet or page, then Excel query option in the QC is the best option.

The AUDIT_LOG table has all the events registered for a entity and AUDIT_PROPERTIES TABLE has the details of these events. Both these tables can be used to retrieve the history of any defect

Let’s try simple query to extract details for a particular bug. The bug id of BUG table is linked to AUDIT_LOG table with field AU_ENTITY_ID and hence create below inner join to begin with
SELECT * FROM BUG B
INNER JOIN AUDIT_LOG AL
ON AL.AU_ENTITY_ID = B.BG_BUG_ID

The AUDIT_PROPERTIES table is linked to AUDIT_LOG by Action_id values. The new query will be
SELECT * FROM BUG B
INNER JOIN AUDIT_LOG AL
ON AL.AU_ENTITY_ID = B.BG_BUG_ID
INNER JOIN AUDIT_PROPERTIES AP
ON AL.AU_ACTION_ID = AP.AP_ACTION_ID

Next comes the where conditions so that we filter out appropriate records. The new query will be
SELECT * FROM BUG B
INNER JOIN AUDIT_LOG AL
ON AL.AU_ENTITY_ID = B.BG_BUG_ID
INNER JOIN AUDIT_PROPERTIES AP
ON AL.AU_ACTION_ID = AP.AP_ACTION_ID
AND AL.AU_ENTITY_TYPE = 'BUG'
AND AL.AU_ACTION_ID= AP.AP_ACTION_ID
AND AP_FIELD_NAME='BG_STATUS'
AND B.BG_BUG_ID='26'
ORDER BY AU_TIME DESC

The above query would give values for all the columns of three tables. For status history , typically interested columns are who changed the status of defect, from what status to what status, and when. To retrieve these values the query will be now
SELECT AL.AU_TIME ,AL.AU_USER ,AP.AP_OLD_VALUE as OLD_STATUS ,AP.AP_NEW_VALUE AS NEW_STATUS FROM BUG B
INNER JOIN AUDIT_LOG AL
ON AL.AU_ENTITY_ID = B.BG_BUG_ID
INNER JOIN AUDIT_PROPERTIES AP
ON AL.AU_ACTION_ID = AP.AP_ACTION_ID
AND AL.AU_ENTITY_TYPE = 'BUG'
AND AL.AU_ACTION_ID= AP.AP_ACTION_ID
AND AP_FIELD_NAME='BG_STATUS'
AND B.BG_BUG_ID='26'
ORDER BY AU_TIME DESC

Below is the result value for defect that I have been querying

The above query can be then generalised by using B.BG_BUG_ID IN (list of defects)

SELECT AL.AU_TIME ,AL.AU_USER ,AP.AP_OLD_VALUE as OLD_STATUS ,AP.AP_NEW_VALUE AS NEW_STATUS FROM BUG B
INNER JOIN AUDIT_LOG AL
ON AL.AU_ENTITY_ID = B.BG_BUG_ID
INNER JOIN AUDIT_PROPERTIES AP
ON AL.AU_ACTION_ID = AP.AP_ACTION_ID
AND AL.AU_ENTITY_TYPE = 'BUG'
AND AL.AU_ACTION_ID= AP.AP_ACTION_ID
AND AP_FIELD_NAME='BG_STATUS'
AND B.BG_BUG_ID IN (list of defects)
ORDER BY AU_TIME DESC

Https Recording with New Jmeter

2013-05-22T20:13:00.001+01:00

Https traffic recording with Jmeter had been a issue for quite a long time. With recent version of Jmeter releases, it has tried to fill the gap on this shortcoming. It had previously https spoof until Jmeter version 2.6 which didnt quite work well with redirects etc. The latest release of Jmeter (currently 2.9) doesnt use this anymore. Jmeter https recording works normally. But for some URL's , it does work straight forward. Below are the steps that needs to be applied so that you can make sure Https recording works.

1. Importing Jmeter proxy certificate.
I had looked into this on google, but couldn't find a source to tell how this can be done. But my colleague found out that there is a file in extras directory of Jmeter.. Its called proxycert.cmd(sh for linux) . Import this file firefox by clicking on Options->Advanced –>View certificates>Authorities –> Import. After this you should see the Jmeter Proxy certificate as shown below

Then click on the edit button and tick the all boxes as shown below and click Ok

Try recording in Jmeter now. You may be are aware of normal recording process in Jmeter. If you not, refer this link Proxy Setup
Even after this if you still facing issue recording the https traffic, which I too had, then try to install firefox 3.6. When it didnt work for me after trying the above steps, I downloaded the older version of firefox 3.6. After installing it, I had following options unchecked in the firefox browser options. After this changes, https recording for Jmeter through firefox worked charmingly.

Creating Basic Selenium web driver automation test case using Nunit and C#

2013-05-19T23:06:00.000+01:00

To create selenium automated tests using C#, you would require nUnit and an IDE like VS express 2012 . To setup these, you can refer my previous blog on setup. In this blog article, I would write on how to create a basic selenium C# script and execute them on VS Express IDE itself.
At the end my previous blog, you would have the code on VS express IDE as below

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace UnitTestProject1
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
        }
    }
}

Say you got a test case that would

1	Opens up firefox browser
2	Navigates to selenium website
3	Clicks on link named Download on the page
4	close the browser

Below are few steps you need update this class file to run it as automated test.

a. Add code lines for importing the packages.
b. Create instance of webdriver
c. Add lines to webdriver so that it navigates to the url say “www.seleniumhq.org”
d. Add lines to click on link on the webpage.
e. Add lines to close the browser.

Step a :
Add below lines just above namespace. These lines are meant to import the classes that are part of reference libraries
using OpenQA.Selenium;

using OpenQA.Selenium.Support;
using OpenQA.Selenium.IE;
using NUnit.Framework;

Step b :

Inside the TestMethod1 , enter following code to call firefox webdriver
IWebDriver driver = new FirefoxDriver();

Step c:

After the above line of code, type the code that uses the driver instances that was created and navigates to selenium website
driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");

Step d:
Steps b and c basically create a webdriver and invoke URL. They represent a manual user actions like opening a browser and typing URL in address bar and then hitting enter. Next step is to simulate user action of clicking on link named download on the webpage. For this one needs to create a instance of webelement that points to that link element. And using this instance, call a method to click it. So the code would look like below
IWebElement myLink = driver.FindElement(By.LinkText("Download"));

myLink.Click();

Step e:

The code from above all steps basically would execute the required steps for the test case. The last step of the test case is to exit the browser. This can be done by using following code
driver.Quit();

After all the above steps you code should look like below.

using System;

using Microsoft.VisualStudio.TestTools.UnitTesting;
//Step ausing OpenQA.Selenium;
using OpenQA.Selenium.Support;
using OpenQA.Selenium.Firefox;
using NUnit.Framework;

namespace NUnitSelenium
{
    [TestFixture]
    public class UnitTest1
    {      

        [SetUp]
        public void SetupTest()
        {
        }
        [Test]
        public void Test_OpeningHomePage()
        {
            // Step b - Initiating webdriver
            IWebDriver driver = new FirefoxDriver();
            //Step c : Making driver to navigate
            driver.Navigate().GoToUrl("http://docs.seleniumhq.org/");
          
            //Step d 
            IWebElement myLink = driver.FindElement(By.LinkText("Download"));
            myLink.Click();
     
            //Step e
            driver.Quit();

            )
           }
    }

Now the test case is ready to run within IDE. To execute this test, first you need to build the project. For this click on Project-> build. Once build is successful, right click inside the method and select run test.

Other way to run the tests in nUnit is open to the DLL file generated from the above project after build (which will be stored in debug directory having name like <projectname.dll>) in nUnit as shown in below snapshot

Once opened in nUnit the test will appear as shown below

Click on Run to execute the test successfully.

Other blogs that may be interesting to you and related to selenium webdriver in C#

On how to setup the IDE (VS express), nUnit and selenium refer How to setup C#,nUnit and selenium client drivers on VSExpress for Automated tests
On Creating simple script that launches a browser does few steps refer Creating Basic Selenium web driver test case using Nunit and C#
On how to Load the same webpage on a number of different browsers suggest referring How to invoke locally different types of browser driver using selenium and c#
On Loading the same webpage on a number of virtual machines(which I have set up) for this, you need to use Remote webdriver instead of normal webdriver. Also with remote webdriver, you can launch different types of browser. Refer this webpage How to invoke/run different type of web driver browser using remote webdriver in C#
To take snapshot on different browser you can refer the link Capturing screen shots using remote/local webdriver in C#/Selenium webdriver

How to setup C#,nUnit and selenium client drivers on VSExpress for Automated tests

2013-05-19T22:11:00.000+01:00

Selenium can be used in conjunction with C# and Nunit to build robust automated test cases. This blog is one of the How to series on selenium-C# that help novice automation tester to learn and build automation test cases.
In this blog, I would be explaining the how to setup tools for Selenium web driver automation.
To build selenium webdriver automation test cases in C# and Nunit, you would require to setup

IDE for C# - Download Visual studio express for Windows Desktop. This is free for 30 days and You must register to obtain a free product key for ongoing use after 30 day. Installation of this tool is pretty straight forward
Nunit - Download NUnit and installation is pretty straight forward
Selenium Dot net client drivers - Download it from selenium download website . The latest version at this time is found as shown below. The drivers are nothing but dll files which needs to referred when you build the test cases which is described later in this blog

Once the VS express is installed, open the tool and it will display the welcome page. To create a selenium test cases in C#, we need to use C# project of VSExpress. For that on file menu click new project and this open new project window. Open Unit test project under Visual C# and click OK This will open a class file as shown below

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace UnitTestProject1
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
        }
    }
}

This is basic test project template in C#. For this projects lets add the references and these include the Selenium web driver which you downloaded and the nUnit dll file from Nunit bin folder. To do this on VS express menu bar, click on project-> add reference. And using browse button browse to directory where you downloaded the selenium c# drivers. Within the directory you may have directory called net40( latest version at the time blog was written) and under this directory select all files and click Ok . Below is the DLL files that I have with current version of C# selenium client libraries.

Once this done we are now set to write automation test cases using Selenium web driver and C#. To Verify whether the selenium client reference libraries are added, try typing at the top “using OpenQA” and press . to see if the selenium packages are displayed.

How to schedule Rational Performance tester(RPT) tests or execute them from command line

2013-05-06T13:22:00.001+01:00

For some cases, one might find himself in a situation where in , he may require to schedule a test to run late night or later point in time. To achieve this with Rational Performance tester, it can be done by scheduling a window task to run a batch file(or crontab to run a shell script) containing the command line that execute the tests

To get cracking on this, first thing is to get the command line written up to execute the rational performance tester test or schedule. For this you need to get the following parameter values that later needs to passed to command through command line.
-workspace : Path to workspace in which the test or test schedule is present. If you are not aware of this, quick way to get the path value of your workspace is to get this from properties window of the project. Right click of the project node and select properties from the menu. Then the value in the location tab of resource node is the value of the workspace path.

-project : This is the name of the project
-schedule: This is the schedule or the test that is required to executed. To get the value for the test that needs to be executed is to right click of the schedule or the test and select properties from the right click menu. On Properties window, copy the value that is displayed for “Path”.

Once you have this value, you got the all the information you required to execute the command line. The command you need execute depends is available in plugin directory. There are both shell and batch version. This typically something like “This is typically C:\Program Files\IBM\Rational\SDP\6.0\rpt_prod\eclipse\plugins\com.ibm.rational.test.lt.cmdlineexecute_6.1.2.” In this directory, there are files with .bat and .sh extension which can be used for windows and Unix respectively
To check if the command line works, navigate to above directory and then issue following command
cmdline –workspace “workspace “–project “testproj” –schedule “schedule_or_test”
Once the above command works, you can create a shell script or batch file that can referenced in cron tab or scheduler to execute is automatically.

Tips on Rational Performance Tester (RPT) Datapool

2013-05-03T23:14:00.001+01:00

After all the time, being used to csv/excel files , when I had to face Datapool in RPT , it wasn’t easy at first on how to manipulate data pool. It became trickily more difficult to handle when the data files grew for larger tests. Below are few of the tricks that helped me to overcome the clumsiness I had with datapool.

1. How to increase the rows of datapool.
Generally way of increasing the datapool row is doing a right click and then selecting option insert row. But this doesnt help much when you want say increase a file by 1000 rows . That means you need to do the right and selecting option to insert row 1000 times. Easy way to do this, is the select the last row, and then keep pressing enter (you can literally hold it pressed) to append news rows to the datapool.
2. How to Import Data pool as CSV file into existing data pool file.
This option is life saver in manipulating the data pool of RPT. One can prepare a datapool file outside the RPT say in a spreadsheet. And then save this as csv file. You can import data into existing datapool file by import export option. Below are the steps to do it.

Right click on the existing datapool file. Select import option

This opens up import dialog box . In the text box, type in CSV and this filters out an option called CSV File as Datapool

Then click next which will take you to “Import CSV file as datapool “ dialog. Click browse and select the file that you want to import. Tick/untick the options depending on how your csv is already prepared. ( I generally untick the both of them as I do not have column names in first row and no first column with record numbers.) After that click next to move to last step
In this step , check the option import into existing datapool. . Then select the datapool that you want to import the data. Then in records section , you can either select replace or append data option and click finish. After this your datapool should have new data from the csv file which you just imported.

3. How to delete multiple rows from a datapool file.
General procedure is to select a row and right click and then select delete row options. For some reason or the other, you may require to delete multiple rows from the datapool file. In that case, then this option becomes more painful. Easy trick to do this, is to select the first row, and press hold shift , scroll down and then select the row till which you want the data to be deleted. With Shift still pressed , right click the mouse on the selected area to open up right click menu with option to delete rows. Like this you can delete multiple rows in few small steps.

Running Rational Performance tester with agents on other side of firewall

2013-04-30T09:02:00.000+01:00

Rational Performance tester in terms of software installations as three components
1. RPT master or workbench
2. RPT License server
3. RPT Agents
RPT master is similar to controller in Loadrunner and plays central role in configuring and executing tests. In a case, where you got RPT agents inside the firewall and master outside or vice versa, the ports with which this communication generally happens becomes key as these ports need to be opened to make the test execution possible. Also if license server is on other side of firewall with respect to workbench , it also requires to open appropriate firewalls.

RPT workbench and RPT license server
RPT workbench and RPT license server communicate through port number 27000 for the purpose of license check. If license check is not working , first thing to look at whether the port 27000 is open by doing telnet to each other on this port. This can be reconfigured to use different port by changing the port number at both license key administrator on work bench and on license server.
To change this, open up the license administrator and then click Settings > Client/Server Configuration. This opens up the client server configuration window as shown below when the port number can be changed from default to something else

RPT workbench and RPT agent controllers
The forward communication between RPT workbench and RPT agent controllers is on ports 10002,10003 and 10005. The firewall on agents machines should be modified to allow these three ports.
The backward communication from RPT agent to RPT workbench in on range of dynamic ports. Say if you have three agents, then each of the agent would communicate back on three different ports . In total there are 9 ports 3 each for one agent. Since this number is dynamic, you can define the range beginning with certain port number. On version 8.0 and above , you can define this on following setting
Windon –> Preferences –> Agent Controller.

As in above snapshot, the port number to begin with is 25000, and if you had two agents, then first agent would communicate back on 25000, 25001, 25002 and second agent would communicate back on 25003, 25004, 25005. Thus all these ports should be open on the firewall to make the playback of test successful.

How to setup Multiple RPT agents on high end server

2013-04-29T16:55:00.001+01:00

Rational Performance tester agents are driver process built on Java. Rpt agent doesn't scale up directly even though you throw more hardware in terms of memory/cpu on it. The reason being the memory constraint for Java heap size of the driver process on agent machine. The maximum heap size for RPT_VMARGS property on linux OS is 3000m. So if you have a big iron having multiple cores and high memory as much as 32GB, then running one agent handicaps you from using the real power of the machine and all the memory goes wasted. To overcome this, there is way you run many RPT agent drivers that can help to use the power of the high end server and drive more user load during the test.

To run multiple drivers through a Rational agent controller server , there is no need to change the installation as it is built to handle this. But on the agent machine, you need to setup multiple IP alias. Say if the IP address for your agent is 192.140.10.1 then one can setup more IP aliases like 192.140.10.10, 192.140.10.11 etc., as many as agent driver you require. Also on the agent machine, make new deployment directories for each of the new multiple driver you plan to run on the server.
Then in Workspace on RPT master, configure locations and each location having one of those IP alias like below in mentioned table

Name	Hostname/IP	Deployment Directory
Driver1	192.140.10.1	/opt/IBM/SDP/AgentController/tmp
Driver2	192.140.10.10	/opt/IBM/SDP/AgentController/tmp1
Driver3	192.140.10.11	/opt/IBM/SDP/AgentController/tmp2

Below are Steps in RPT workspace to set this up.
1. Creating new Agent location

Select location from the above window and click OK. Select the path where this agent should be saved in workspace.
2. Configuring the agent according to above table
Go to the path where the agent is saved in workspace, and double click to open it. On first tab – General properties , in Name text box, enter the name and in "Hostname or IP address text box", enter the IP. Click test connection to verify the connection works.
3. Click general Properties to open up the properties table and create the properties as shown below( linux for linux OS)

Once these steps are done, the RAC (agent controller) is configured to use multiple agents. You now can choose these agents in user groups in schedule and execute test

Memory considerations In Rational Performance tester (RPT)

2013-04-26T14:50:00.001+01:00

Rational Performance tester tool is built on Java/Eclipse technology. Unlike Loadrunner, there are few memory implications that one has to be aware when using Rational performance tester.
RPT is GUI based tool and main GUI is enhanced eclipse. This is called workbench and mirrors controller in Loadrunner. The memory settings of the eclipse is key in having responsive workbench.
Other region of interest with regards to RPT memory is the driver memory . This memory is very key is executing higher number of users otherwise, you will face the issue of Java Out of memory very frequently. As evidently this two setting both relate to JVM memory settings.

1. JVM memory settings for work bench
The settings for this JVM can be done in the eclipse.ini file found in two directories . One in C:\Program Files\IBM\SDP and another one in C:\Program Files\IBM\SDP\rptse. The file in the former directory corresponds to IBM performance tester - full eclipse and latter one for is plain version. Depending on what version you use you can make change to eclipse.ini file in respective version. To add memory to this JVM, add the startup parameter -Xmx1200m into eclipse.ini file which mean maximum heap size for JVM is 1200 MB. And then restart the rational performance tester. After than you also view how much memory is been used by workbench by enabling the progress bar view for it in Window – > Preference: Show Heap Status

This JVM setting doesn't depend on number of users you run, but its useful when you open up many scripts, or open report from the test with large number of errors or with long duration.
2. JVM memory settings for driver in RPT agent controller
This is key memory setting and plays major role in the number of users that can be tested without Out of memory issue. This memory setting is for each driver in Agent machines that gets intialised when you kick of the test. The test can carry on if these drivers have more power in terms of memory and CPU. The memory for each of the agent can be specified by passing on parameter RPT_VMARGS

Open the agent controller location in the workspace by double clicking it.
Then define new parameter as shown below which means the JVM heapsize for driver on agent machine is 1500MB. The maximum value it can take (as of with RPT 8.2 version ) is 3000MB. If you have big server with huge memory and multiple processors, then you might want to run more agents on these machine because, 3000MB limit of JVM settings will cut down advantage for you from using all the power of agent machine. On How to run multiple agents on one machine, you can refer my other blog How to setup Mutliple agents

Apart from number of users, the other things that have implication on this memory are , large gif file, long script, responses that take long time, SSL connections.

How to create new Rational Performance tester Custom report?

2013-04-26T11:59:00.001+01:00

Rational Performance tester (RPT) has good feature to create custom reports. The main thing with default reports was the challenge to filter the report for each individual test scripts and have these are readily available in RPT.
Workaround is to create your own reports that you can fetch whenever required with just couple clicks.
Below are the steps that can help to create the your own Rational performance tester custom report.
1. Creating new report template
2. Creating the tabs as many as you want that have the charts,table or graphic the way you want
3. Adding graphic filters – like top 10 page hits etc
4. Adding the performance counters that you want the report to present like average page response time, or total completed transactions etc

To go through this process, lets take a standard requirement which most performance engineer would have. Say, one requires a report that shows in each tab, the average response vs time detail chart and the corresponding table for each of the scripts/User journey that are executed in the test
To create a timeline chart for average response vs time, we would require a chart graphic and counters that in interest are Pages counters – Page response time-> Average response for all Pages. And same counter can be used for tabular data too, but the graphic you require is table.
Having done this table work for tab, lets get started on
1. Creating new Report template

Click on any old performance report and this report appears in Performance test run window as shown below

Click on Manage report button which is as shown below

This Opens up the Select report dialog box .Click create new button

This opens up Create new report dialog. Enter the report name and click insert button

2. This opens up create new tab window. Enter the title of the tab for example – feature1 . In the frame, “ select graphic template”, it require to choose the template that can suit the measurements you want the tab to report. For example, if you are looking for a time base chart like average response time over the test period , you can choose “Response vs time detail tab” . After that, click on button Next

3. Next window that appears showing the option for graphic template that has been selected which is Graphic 1 of 2. Graphic 1 appears on the top and followed by Graphic 2 . Here the selected graphic is line chart . One can add filters here by clicking Add.Since this tab is only for feature1 and say the page names in the your feature1 script has FP1 as prefix, then one can filter the page response time counters that contains “FP1” by selecting the options as shown in below snapshot and filter value. Then Click Ok

4. You will come back to Report tab window and then click next. You will see, report tab with frame showing performance counters. One can add any counter that is available by click add and then selecting the values. For this example, I am going to add Page response counter for average values. Lets remove the counter that is selected by default by selecting it and clicking remove. Then click Add and this pops up small menu with different categories of counter. Select Page counters in the option which opens up the Add/Remove Page Performance counters. Select the option as shown below and click Finish

5. Click Next and you will be presented with dialog saying Graphic2 of 2 which has by default selected graphic as Table. You have to remove the same above steps as step 3 and 4 and then click finish
Now at this step, you have created new custom report and new tab within that report for Feature 1. You can do repeat adding of tabs for different feature untill you done with what you require. To open this report for any test you have executed and then select Display report option which will present with window showing the report – Report1 that you just created. Double click to open the report and it all ready now. You can fetch these report whenever you want with just click of two buttons .

RPT : Extracting value based on the occurrence that is passed dynamically

2012-04-15T13:07:00.000+01:00

Rational Performance tester does correlation by extracting dynamic values in response by creating a reference. But it lacks ability for an user to specify if the occurrence dynamically that needs to extracted .
For ex: Say in an application, after user logins , he views displayed list of cards. He picks first card which is default and picks any other card alternatively sometimes. You got a task cut out to create a script which picks 60 percent default card and other 40 percent of time any of the alternate cards. You cannot select 'Random Occurrence ' option because it defeats purpose of simulating default card 60 % of time or vice verse.

So to overcome the limitation in RPT, one has to find a way to pass the occurrence you want to pick and pick the card by that occurrence value(not just random occurrence). Here in this blog I would focus on extracting that value based on dynamic occurrence which is passed to a custom code interface . For this below are steps that need to be done

Create a reference to extract entire response that contains the html text
Create a custom code and pass the this reference as an argument along with the occurrence value
Write java code for this custom code interface that extracts value matching the pattern and occurrence and return the card number extracted

Step 1 : Create a reference to extract entire response that contains the html text containing all the cards
For this click on the response and from the right side of pane, click on content multiline text box(Do not select any text).

Right click and RPT displays menu as shown below with an menu item named create reference. Create field reference is different in a way from Create reference , that it extracts complete response into a reference field. While create reference will extract only part of text value that would have been highlighted by cursor. And then click create field reference and give it a appropriate name.

Step 2: Create a custom code and pass the this reference as an argument
Create a custom code element and Pass this reference as an argument to custom code interface as shown below.I have described in another blog on How to add reference item to custom code. In same, add the another argument that specifies the occurrence that needs to be extracted

Step 3 : Write java code for this custom code interface that extracts value matching the pattern and occurrence and return the card number extracted .
There are two ways to return this information to main test script from customcode. One by assigning value to already defined variable using put method call or by passing as return value of the custom code. Below code has both the methods shown

package customcode;

import java.util.regex.Matcher;

import java.util.regex.Pattern;

import com.ibm.rational.test.lt.kernel.IDataArea;

import com.ibm.rational.test.lt.kernel.services.ITestExecutionServices;

/**

* @author unknown

public class RegExonResponse implements

com.ibm.rational.test.lt.kernel.custom.ICustomCode2 {

/**

* Instances of this will be created using the no-arg constructor.

public RegExonResponse() {

}

/**

* For javadoc of ICustomCode2 and ITestExecutionServices interfaces, select 'Help Contents' in the

* Help menu and select 'Extending Rational Performance Tester functionality' -> 'Extending test execution with custom code'

public String exec(ITestExecutionServices tes, String[] args) {

String RegExpStr = "\"cardId\":\"(.*?)\"";

int i = 0;

int reqOccurence = Integer.parseInt(args[1]); // args[1] because this second argument passed to custom code

String CardBasedOnOccurence = null;

Pattern pattern = Pattern.compile(RegExpStr,Pattern.DOTALL);

IDataArea userDataArea = tes.findDataArea(IDataArea.VIRTUALUSER);

Matcher matcher = pattern.matcher(args[0]);

//tes.getTestLogManager().reportMessage( "Total " + matcher.groupCount());

while(matcher.find())

{

i = i + 1;

tes.getTestLogManager().reportMessage("First occurrence" + matcher.group());

if (i==reqOccurence )

{

CardBasedOnOccurence = matcher.group();

/*Use this step if you have defined a variable named SelectedCardID in the

test else, you can skip this and pass value as return value from this custom

code as shown in last few lines of code*/

userDataArea.put("SelectedCardID",CardBasedOnOccurence);

}

if (CardBasedOnOccurence==null)

{

return null;

}

else

{

return CardBasedOnOccurence;

}

What does RPT Lacks for a Loadrunner User

2012-04-15T00:39:00.000+01:00

I have been a HP Loadrunner user most of the time for Performance/Load Test. Off late, I ventured into task to use IBM Rational Performance tester (RPT). I did had very pleasant surprises from this tool . Below are few of those to mention

Automatic correlation feature got me quite amazed when it worked flawless all the time. RPT Uses Perl type of regular expression patterns to extract data from respone. I was impressed with perl regular expression back in time when I first used Jmeter and still find they give robust way of defining pattern. RPT does all related to automatic extraction expression pattern, instance number etc. Thus makes life easy when creating a new script.
Not just records thinktime, also records client processing delays . Hence represents right way of simulating secondary requests

But I find that many features ,which are very common to Loadrunner or any scripting tool, are missing here in RPT. RPT does make maintaining scripts bit harder without simple scripting interface . Below are the few of those missing features that made RPT annoying to me at times.

You cannot call a test within a test.

This meant that I cannot refractor the common steps (like login, etc). This meant that I cannot maintain common http calls across the scripts. Frequent RPT users advice another way on how to go about this. They suggest to break up one test in many tests and then arrange them in performance test schedule. I did manage to refractor this way, but this meant lot of overhead work of creating variables, assigning them . And every time I make change in one test, then I need to go to schedule and execute it to verify. Wouldn't the world be easy if we could be able to call a test within a test. This would solve many small pestering problems like I just mentioned

Cannot verify another test script, if there is already a performance schedule is running.

While using Loadrunner, there were situation, where I executed a test in controller for a Project and at the same time, verified the scripts from another project. This helped me to make use of those idle time when test execution happens in background. But in RPT, you cannot do that. You can execute one test at a time. If you executing a main test in RPT master, then you cannot verify another script

No Vuser end method

In Loadrunner, if one stops a test , then users exit gracefully and execute vuser end code. This is kind of feature is missing in RPT. Hence it means, when you have an application which requires user to logoff , there is no way you can achieve this, when you stop the test in middle. Users won't exit gracefully.

I have an application which requires user to logoff else it won't allow to login back for another 10 mins. If I find for some reason that I need to stop the test, then I had to wait for 10 minutes, verify all those users are okay (imagine time and effort spent to verify 10000 users if I require to) and then restart the test. This missing feature has made me to loose lot of time in the middle of tests and is very annoying for all who are involved in the tests

Cannot stop particular user during execution

This thing I used to frequently in Loadrunner, where in I find some user had corrupt data (because they did not logoff properly ) or I require to bring down a particular number of users in a particular group , then I had means to pick and choose user(s)and then stop them. While in RPT, you can still stop users, but you cannot pick and choose particular user(s) and then stop them. On this front, RPT just fares same as Jmeter which is sad. I admire Jmeter although it lacks this feature.

Capturing response, data substitution on error or failure

This capability was one of very good troubleshooting asset of Loadrunner. During a performance test with many users doing lot of interaction, if an error or failure occurs, you would definitely like to know what was request made, response received, data susbituted etc ,and also at same moment itself you view them. RPT has bit complex logging method, which I still need to get my head around. But what I found ,when eyeball over it, there is no obvious way to get this. Only way is you have to log a datapool value(parameter in loadrunner), for which you have write a custom code and write the value in Test Log. This gets logged irrespective of failure or not. Imagine in Http request where you are sending quite number of correlated values plus datapool values, then you have write Java custom code to log all of these. Wish, it has some option which allows to capture response, data substitutions etc only on error.

Rational Performance tester(RPT) custom code to log user value fom datapool

2012-04-15T00:38:00.000+01:00

Below is the custom code in Rational Performance tester that I managed for logging username value from datapool into test log. In Loadrunner, very often one uses commands like lr_message,lr_log_message to log username that was used by a particular virtual user in particular iteration. This helps performance tester to check whether the data was used was appropriate one, of if error occured, then what was the data used by the users etc. This this can be achieved in RPT by writing custom java code.

Custom Code class implements interface ICustomCode which has method called exec with method parameters of object type ITestExecutionServices and type String array

The trick is understand the classes/method/interfaces of this Interface ITestExecutionServices. This object has information about a virtual user in interface name IDataArea . Below is the syntax to retrieve this Virtual user dataarea.

IDataArea dataArea = tes.findDataArea(IDataArea.VIRTUALUSER);

From this Interface, one has to get another interface instance named IVirtualUserInfo. This interface contains that virtual user information.Below is the syntax code for that

IVirtualUserInfo virtualUserInfo = (IVirtualUserInfo) dataArea.get(IVirtualUserInfo.KEY);

And then from this Interface, one can get virtual userID by method named called getUserName()

To log he datapool value of user actually used , I passed that datapool value as argument to custom code interface. Since it was only one argument passed, I was able to extract the value of it in custom code by code syntax 'String UserName = args[0];' . Below is the complete code for this custom code interface

package customcode;

import com.ibm.rational.test.lt.kernel.IDataArea;

import com.ibm.rational.test.lt.kernel.services.ITestExecutionServices;

import com.ibm.rational.test.lt.kernel.services.IVirtualUserInfo;

/**

* @author unknown

public class LogUserName implements

com.ibm.rational.test.lt.kernel.custom.ICustomCode2 {

/**

* Instances of this will be created using the no-arg constructor.

public LogUserName() {

}

/**

* For javadoc of ICustomCode2 and ITestExecutionServices interfaces, select 'Help Contents' in the

* Help menu and select 'Extending Rational Performance Tester functionality' -> 'Extending test execution with custom code'

public String exec(ITestExecutionServices tes, String[] args) {

String UserName = args[0];

IDataArea dataArea = tes.findDataArea(IDataArea.VIRTUALUSER);

IVirtualUserInfo virtualUserInfo = (IVirtualUserInfo) dataArea.get(IVirtualUserInfo.KEY);

String User = virtualUserInfo.getUserName();

tes.getTestLogManager().reportMessage(User + "=" + UserName);

return null;

}

Executing Selenium TestNG tests using ANT

2011-10-31T21:00:00.000+00:00

ANT is software build automating tool for Java code. Uses XML file named ‘build.xml’ as the input for the build tasks. If one wants to run TestNG task using ANT, XML should contain information for testng calls and this can be built in two ways

1. By defining a external task of type TestNG and referring this task in another target ANT call further down in the code

2. By defining as Java command in ANT XML

The command line syntax to run TestNG task is java org.testng.TestNG testng1.xml [testng2.xml testng3.xml ...]. The second option takes this approach of calling testNG using command line syntax but using ANT. Lets have look at the first option here in this post

Defining TestNG task

The ANT xml tag to define external Task is by <taskdef> and this takes two attributes @name and @classname. And the tag <Classpath> includes the TestNG jar from the defined path .Hence below is the way to define a ANT task of type TestNG
1:    <taskdef name="testng" classname="org.testng.TestNGAntTask" >
2:      <classpath>
3:        <pathelement location="lib/testng-6.2.jar"/>
4:      </classpath>
5:    </taskdef>

The path in line 3 for me is lib/testng-6.2,jar. This is because have this TestNG jar file in lib folder and my build xml is same folder as my lib folder is as shown is figure beside. so the value for location attribute of pathelement should be relative path of TestNG jar file to your build file

Once the above taskdef is done, next define the actual target for ANT call. Any ANT task are referred by tags <target>. Within this <target> container, we will create a output directory for the testng reports, and then call the testng task that we just have defined

1:    <target name="runTest">
2:      <mkdir dir="testng_output"/>
3:      <testng outputdir="testng_output" classpathref="class.path">
4:        <xmlfileset dir="." includes="TestNG1.xml"/>
5:        <xmlfileset dir="." includes="TestNG2.xml"/>
6:      </testng>
7:    </target>

In above XML, the attribute classpathref actually refers to container of type path having name as ‘class.path’ . So for you , you need to define this path as per below xml before the target tag “runTest”. Below is the xml for defining path containing all classes and jar files from my project folder. And below is the folder structure I have for my project

1: <property name="src" location="src" />
2: <property name="bin" location="bin" /> 3: <property name="libs" location="lib" />
3: <path id="class.path">
4:      <pathelement location="${libs}/testng-6.2.jar" />
5:      <pathelement location="${libs}/selenium-java-client-driver.jar" />
6:      <pathelement location="${libs}/selenium-server-standalone-2.6.0.jar" />
7:      <pathelement location="${bin}" />
8: </path>

Final XML

Hence the final XML, after defining project name etc, it should look like this below

1: <project basedir="." default="runTest" name="Ant file for TestNG">
2:
3: <property name="src" location="src" />
4: <property name="bin" location="bin" />
5: <property name="libs" location="lib" />
6:
7: <path id="class.path">
8:      <pathelement location="${libs}/testng-6.2.jar" />
9:      <pathelement location="${libs}/selenium-java-client-driver.jar" />
10:      <pathelement location="${libs}/selenium-server-standalone-2.6.0.jar" />
11:      <pathelement location="${bin}" />
12: </path>
13:
14:    <taskdef name="testng" classname="org.testng.TestNGAntTask">
15:      <classpath>
16:        <pathelement location="lib/testng-6.2.jar"/>
17:      </classpath>
18:    </taskdef>
19:
20:     <target name="runTest">
21:
22:      <mkdir dir="testng_output"/>
23:
24:      <testng outputdir="testng_output" classpathref="class.path">
25:        <xmlfileset dir="." includes="TestNG1.xml"/>
26:        <xmlfileset dir="." includes="TestNG2.xml"/>
27:      </testng>
28:
29:    </target>
30: </project>

Now to execute this ant file, go to the directory where the build.xml resides in a new command line window, And then enter command “ant” and ant will execute the testng tests. If your ant file is having different name than build.xml, then you need to run command “ant –f < yourfilename>” to run the testng tests. Since we defined the default task is “runTest” in project attributes, running ant without specifing target name will execute default target (here runTest). For more options on ant command check this link for running ant

Running Selenium TestNG test from command line

2011-10-24T13:30:00.001+01:00

When I tried to run TestNG command from command line, I get following error. This is mainly because java cannot find class path and jar file of TestNG

C:\Selenium\Selenium_tests\DemoProject java org.testng.TestNg
Exception in thread "main" java.lang.NoClassDefFoundError: org/testng/TestNg
Caused by: java.lang.ClassNotFoundException: org.testng.TestNg
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
Could not find the main class: org.testng.TestNg.  Program will exit.

C:\Selenium\Selenium_tests\DemoProject

So below are the steps to get it working

1. Navigate to directory of your project (if you are using Eclipse IDE).Create and set a path variable named “ProjectPath” by entering following command. Do echo command to check what is the value that we just set is right

C:\Selenium\Selenium_tests\DemoProject set ProjectPath=C:\Selenium\Seleni

um_tests\DemoProject

C:\Selenium\Selenium_tests\DemoProject echo %ProjectPath%
"C:\Selenium\Selenium_tests\DemoProject"

2. Now we after setting this project path, we need to create classpath that points to selenium/Testng class files and jar files along with testng jar file. My project setup is like below and I keep jar files in folder called lib and eclipse creates complied classes in directory called bin automatically when you execute your test cases from eclipse

Below is command to set the classpath for jars in lib folder and binary classes/folders in bin folder and do a echo to check the path that is been set

C:\Selenium\Selenium_tests\DemoProject set classpath=%ProjectPath%

\bin;%ProjectPath%\Lib\*

C:\Selenium\Selenium_tests\DemoProject echo %classpath%
C:\Selenium\Selenium_tests\DemoProject\bin;C:\Selenium\Selenium_tes

ts\DemoProject\Lib\*

3. Now execute the java command for TestNG as shown below

C:\Selenium\Selenium_tests\DemoProject java org.testng.TestNG

%ProjectPath%\IEandFirefoxTests.xml

[TestNG] Running:
  C:\Selenium\Selenium_tests\DemoProject\IEandFirefoxTests.xml
===============================================
IEandFirefox
Total tests run: 6, Failures: 0, Skips: 6
Configuration Failures: 4, Skips: 12
===============================================

How to retrieve the test sets - test instances - results within particular folder/path in Quality Center

2010-05-07T10:54:00.002+01:00

Many times , you would be running the tests in multiple test sets and with multiple instances. Those multiple test sets would be in different folders under and under different hierarchy

Quality center has option to get this information using Analysis graphs, and thus you would be able to find what is the progress of each test set. But is has its own limitations. Like, if you want to find which test in each of sets is failed, or who has been the responsible tester for those tests , or any more information, then we are handicapped using Analysis reports.

Well , if you learn the excel reports, you life would be made easy. I have briefly mentioned on How to access excel reports in one of previous blog

So how shall we do go ahead?

Let me explain how the test sets, test lab folders, test cases are arranged. Below diagram from Quality Help file shows the Entity relationship diagram

The TESTCYCL table is the one that stores test set details. Hence it has information on instances of design tests within test set. If you not aware of instance, Instance is use of a test in a test set. If the test set contains a test more than once , then it will have same numbers of instances in the test set. This value is stored in column TC_TEST_INSTANCE column of TESTCYCL table.

The TEST table contains all the tests that are defined in test plan. The main and bit complex folder is the one that stores the path. It is CYCLE table, where it contain entry for each folder that is created in tree view. But how is the tree structure of quality center folder stored?Each row in this table corresponds to folder and each of them have a Father ID indicating ID of another folder under which it falls.

Suppose we have folder structure like
Folder1/Folder2/Folder3/Folder4
Then for folder1
ID = 1 , Father ID = 0
For folder 2
ID = 2, Father ID = ID of folder1 = 1
For folder 3
ID = 3, Father ID = ID of folder2 = 2
etc.

Hence the table has to be self joined to retrieve exact test cases/sets within a folder.

From the diagram, we can easily write down the following join conditions
TESTCYCL.TC_CYCLE_ID = CYCLE.CY_CYCLE_ID
TEST.TS_TEST_ID = TESTCYCL.TC_TEST_ID

Now say you want to retrieve all the test sets/cases within Folder2 till the level of folder 4. Here you have 4 levels of folder. Assume for lowest level Folder4 as representing z, then folder3 as y, then folder2 as x, then folder1 as w

Then we have write the self joins as below

and CYCLE.CY_FOLDER_ID = z.CF_item_id
and z.cf_father_id = y.CF_item_id
and y.cf_father_id = x.CF_item_id
and x.cf_father_id = w.CF_item_id
and x.cf_item_name = 'Folder2'
and w.cf_item_name = 'Folder1'

The final query would be

select Test.TS_NAME as TestCase, TESTCYCL.TC_STATUS as Status,CYCLE.CY_CYCLE as TestSet, TESTCYCL.TC_ACTUAL_TESTER as Tester
from CYCLE,TEST,TESTCYCL,CYCL_FOLD z ,CYCL_FOLD y,
CYCL_FOLD x,CYCL_FOLD w
where TEST.TS_TEST_ID = TESTCYCL.TC_TEST_ID
and TESTCYCL.TC_CYCLE_ID = CYCLE.CY_CYCLE_ID
and CYCLE.CY_FOLDER_ID = z.CF_item_id
and z.cf_father_id = y.CF_item_id
and y.cf_father_id = x.CF_item_id
and x.cf_father_id = w.CF_item_id
and x.cf_item_name = 'Folder2'
and w.cf_item_name = 'Folder1'

How to retrieve requirement - test case - linked defect using excel reports

2010-05-05T20:35:00.001+01:00

QC 9.2 has feature called excel reports. This feature is very versatile as you can retrieve complex data which simple reports or graphs from QC will not be able to do. One of the report that may be useful for any testing project is to find which all defects are linked to test cases and these test cases are covering which of the requirements. This will give you data that can be used to address which defects are more severely effecting your testing progress and resolving a defect what requirements/test cases you can quickly progress.

Here below is entity relationship diagram that you can find in HP quality center database reference. It is one of the help documentation available in QC.

For folks who not sure how to access excel reports in QC , follow below steps you will be amazed to find this feature.

Login into QC with project and domain. Then click on Tools and next select excel reports. It will open up another window. In the top menu , there is + button which on clicking will open report. You can make that report public or private.

TEST - REQUIREMENT ER DIAGRAM

So the box highlighted in Red is the table which stores Test details ( from Plan). The TEST table is connected to Requirement (REQ) table via requirement coverage table REQ_COVER..

At the start , your query would be simple one as below which will retrieve test details

Select * from TEST t

Next we need join both REQ and REQ_COVER tables. Join condition between REQ and REQ_COVER is

rq_req_id = rc_req_id. Join condition between REQ_COVER and TEST is rc_entity_id = ts_test_id and rc_entity_type = 'TEST'. Using these lets construct below query

Select r.rq_req_name, t.TS_NAME from TEST t, REQ r, REQ_COVER rc

where rq_req_id = rc_req_id

and rc_entity_id = ts_test_id

and rc_entity_type = 'TEST'

The above query will list all the tests and requirements attached to those tests.

Next is how to find linked defects. Basically the defects are linked to tests which are pulled in the test sets of QC. We have to join test sets (TESTCYCL) to test (TEST) first, then join linked defects table LINK

Join between TESTCYCL and TEST tables is as below

TEST..ts_test_id = TESTCYCL.tc_test_id

The join between test set (TESTCYCL) and LINK is as below

LINK.ln_entity_type = 'TESTCYCL' and TESTCYCL.TC_TESTCYCL_ID = LINK.ln_entity_id

Here you can notice the condition "LINK.ln_entity_type = 'TESTCYCL' " has value equated to TESTCYCL which is table name containing test sets. Basically if you link defects to test sets or tests within test set then, ln_entity_type will have the value TESTCYCL

So combining all the conditions we can make the query which will retrieve requirements/test cases/linked defects

Select r.rq_req_name, t.TS_NAME,ln_bug_id from TEST t, REQ r, REQ_COVER rc, TESTCYCL tc, LINK l

where rq_req_id = rc_req_id

and rc_entity_id = ts_test_id

and rc_entity_type = 'TEST'

and t.ts_test_id = tc.tc_test_id

and l.ln_entity_type = 'TESTCYCL'

and Tc.TC_TESTCYCL_ID = l.ln_entity_id

If you run the above query in excel reports, you should be able to retrieve the mapping of link defects - test cases- requirements.

*Note : This will not retrieve those defects which are linked to test cases but the test cases are not covered to a requirement.

The other useful blogs related to Audit tables and QC history can be found in following locations

First time Fix faliures or Reopen defects

How to retrieve Quality Center bug details and Audit tables information

How to retrieve QC status history for closed defect

How to retrieve test instances and test sets within particular folder

First time fix failures or Reopen defect from QC

2010-05-02T07:19:00.005+01:00

In quality center, for your project analysis, it would be useful to find how many defects reopened. As you might be aware that , due to defects and not so good work from developers, your testing progress gets hampered. But how will you be able to quantify like the defects most of them were reopened and users had to do more of regression and testing? For this, metrics of Reopen defect is quite good and will justify that there was more of testing required than expected.

In my previous blog How to retrieve Quality center bug details ? ,

I have explained the information of Audit tables in QC. In case,if you are not aware of Audit tables, it would be worth going through it first.

So how to find reopen defects? The audit_properties table as said before holds the properties of the QC entity for any action that happened. So when a defect gets a new status, for this action there is entry made in audit_log table saying an action is performed for the bug. Next for this action, audit_properties table stores the properties that have been changed.
This means audit_entity column of audit log table will have value saying it is for BUG ( the table name of defects). and ap_field_name column of audit_properties will have the value of BG_status ( the column in the defect table that stores Defect status). Hence the value for the column ap_new_value of audit properties table will have has Reopen( Check the exact text for your project for Reopen status. It may vary)

Thus we have three conditions
1. audit_log.AU_ENTITY_TYPE = 'BUG'
2. audit_properties.AP_FIELD_NAME = 'BG_STATUS'
3. audit_properties.AP_NEW_VALUE = 'Reopen'

As per relationship between Audit_log and audit_properties , we have following condition . You can check the Relationship Diagram in my previous blog How to retrieve Quality center bug details ?

audit_log.AU_ENTITY_ID = audit_properties.BG_BUG_ID
audit_properties.AP_ACTION_ID = audit_log.AU_ACTION_ID

Thus from above conditions you can find all the defects that have been reopened.

SELECT
Bug.bg_project,bug.BG_BUG_ID as Defect_ID
FROM BUG Bug, AUDIT_LOG , AUDIT_PROPERTIES
WHERE audit_log.AU_ENTITY_ID = Bug.BG_BUG_ID
and audit_log.AU_ENTITY_TYPE = 'BUG' and
audit_properties.AP_ACTION_ID = audit_log.AU_ACTION_ID
and audit_properties.AP_FIELD_NAME = 'BG_STATUS'
and audit_properties.AP_NEW_VALUE = 'Reopen'

* 'BG_STATUS' is default column in BUG table where the status of the defect is stored. But in many cases, or in your QC project , the status might have defined to be stored in some other column. You can try locate that field in BUG table when you have glance at it in QC excel reports, because user need to give appropiate name while setting up this.

Software Engineering - Performance Automation

Basic SpecFlow Automation tests in visual studio express using C# ,Selenium webdriver,nUnit console

1. Setting up specflow and generate C# file having gluing code

2. Writing step definitions

3. Execute test in nUnit and generate specflow report.

4. Automate generation of nunit testrunner file having step defintion.

5. Automate Execution of Specflow tests in nUnit and generation of report

How to invoke nUnit from Visual studio express and execute automation tests

Capturing screen shots using remote/local webdriver in C#/Selenium webdriver automation test cases

How to invoke/run different type of selenium web driver browser using remote webdriver in C# automation scripts

How to invoke locally different types of browser web driver using selenium and c# for test Automation scripts

Quality Center test instances /test cases that are blocked and corresponding linked defects

Quality center defect history – To track the defects that have undergone severity change

QC history for closed defect – Status History

Https Recording with New Jmeter

Creating Basic Selenium web driver automation test case using Nunit and C#

How to setup C#,nUnit and selenium client drivers on VSExpress for Automated tests

How to schedule Rational Performance tester(RPT) tests or execute them from command line

Tips on Rational Performance Tester (RPT) Datapool

Running Rational Performance tester with agents on other side of firewall

How to setup Multiple RPT agents on high end server

Memory considerations In Rational Performance tester (RPT)

How to create new Rational Performance tester Custom report?

RPT : Extracting value based on the occurrence that is passed dynamically

What does RPT Lacks for a Loadrunner User

Rational Performance tester(RPT) custom code to log user value fom datapool

Executing Selenium TestNG tests using ANT

Defining TestNG task

Final XML

Running Selenium TestNG test from command line

How to retrieve the test sets - test instances - results within particular folder/path in Quality Center

How to retrieve requirement - test case - linked defect using excel reports

First time Fix faliures or Reopen defects How to retrieve Quality Center bug details and Audit tables information How to retrieve QC status history for closed defect How to retrieve test instances and test sets within particular folder

First time fix failures or Reopen defect from QC

First time Fix faliures or Reopen defects

How to retrieve Quality Center bug details and Audit tables information

How to retrieve QC status history for closed defect

How to retrieve test instances and test sets within particular folder