Software Creations - Orn Kristjansson

Selenium browser UnitTesting from TeamCity

orn — Fri, 30 Jul 2010 22:32:45 +0000

I just setup browser testing framework utilizing Selenium from TeamCity project. The usual suspects are involved, Gallio, MBUnit, Nant and even C# Unittests.

The usual scenario when it comes to automating browser testing is that the QA / testers will create some scripts to run a browser against your website. Somehow those tests are usually not maintained very well and often are run by hand. There is not much value in browser testing scripts if you have to run them by hand.

As I needed browser testing on one of the projects I’m on I decided to look into using more of a automated setup to run the browser tests. There seem to be two big players Waitn and Selenium, Selenium lends itself to broader range of testing, naturally we will go with Selenium.

Here is the scenario I want, the tester installs a recorder on his computer, in this case a FireFox plugin. The tester records the tests and runs them in the browser using the plugin tool. Once the tester is happy with the tests the tester checks them into the repository. After check-in tester lets a developer know that there are new or changed tests. The developer takes the script and turns it into C# UnitTest, simply has Selenium convert it to UnitTest code. Then the developer takes and updates or adds the tests that resulted from the scripts and checks it into the repository. The conversion step could be automated in the future once Selenium supports that. The next step is to run it from TeamCity and after it runs you get email with the results.

So let’s take a closer look at what is needed. We need the UnitTest to be able to run against different servers using different browsers. We will pass values from TeamCity to the Nant script that is responsible for compiling and running the tests. This is how your test C# configuration file might look like.

key=“SeleniumAddress” value=“localhost” />
key=“SeleniumPort” value=“4444″ />
key=“SeleniumSpeed” value=“0″ />

key=“BrowserType” value=“*firefox” />
key=“BrowserUrl” value=“http://10.9.169.198/” />
key=“BaseUrlPath” value=“IPCA.Dev/” />

Then the base test class will look something like this.

[FixtureSetUp]
public virtual void TestFixtureSetup()
{
// Read from config
msBrowserType = getConfigSetting(“BrowserType”, msBrowserType );
msBrowserUrl = getConfigSetting(“BrowserUrl”, msBrowserUrl);
msBasePath = getConfigSetting(“BaseUrlPath”, msBasePath);
//
msSeleniumAddress = getConfigSetting( “SeleniumAddress”, msSeleniumAddress );
miSeleniumPort = int.Parse(getConfigSetting(“SeleniumPort”, miSeleniumPort.ToString()) );
msSeleniumSpeed = getConfigSetting( “SeleniumSpeed”, msSeleniumSpeed );

// Start up the selenium session, using config values
selenium = new DefaultSelenium(msSeleniumAddress, miSeleniumPort, msBrowserType, msBrowserUrl);
selenium.Start();
// Clean errors
verificationErrors = new StringBuilder();

// sets the speed of execution of GUI commands
selenium.SetSpeed(msSeleniumSpeed);
}

[TearDown]
public void TeardownTest()
{
try
{
selenium.Stop();
}
catch (Exception)
{
// Ignore errors if unable to close the browser
}
Assert.AreEqual(“”, verificationErrors.ToString());
}

And a sample Selenium C# Unittest

//
using System;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
//
using Gallio.Framework.Assertions;
using MbUnit.Framework;
//
using Selenium;

namespace SeleniumTests
{
[TestFixture]
public class LoginPage : WebTestBase
{

[Test]
public void TheLoginPageTest()
{
selenium.Open( this.msBasePath + “TestLogin.aspx”);
selenium.Click(“lbAdmin”);
selenium.WaitForPageToLoad(“50000″);
selenium.Click(“loginLink”);
selenium.WaitForPageToLoad(“50000″);
try
{
Assert.IsTrue(selenium.IsTextPresent(“my responsibilities regarding permissible access”));
}
catch (AssertionException e)
{
verificationErrors.Append(e.Message);
}
selenium.Click(“ctl00_pageContent_btnSubmit”);
selenium.WaitForPageToLoad(“50000″);
try
{
Assert.IsTrue(selenium.IsTextPresent(“Total Unassigned Web”));
}
catch (AssertionException e)
{
verificationErrors.Append(e.Message);
}
}
}
}

After the Nant script compiles the tests and is getting ready to run the UnitTests it needs to startup the Selenium engine. Make sure to spawn in order for the Selenium engine to exist on another thread than your tests.

name=“SeleniumExec” value=“java” />
name=“SeleniumPath” value=“C:\apps\selenium\selenium-server-1.0.3\” />
SeleniumParams” value=”-jar ${SeleniumPath}selenium-server.jar” />

${SeleniumExec}“
commandline=”${SeleniumParams}” workingdir=”${path.base}${WebTest}“
spawn=”true” failonerror=”true” verbose=”true“
/>

3000” />

In order to run the tests using different browsers, change the configuration file of the tests before run.

file=“${path.base.test}${assembly.test.config}”
xpath=“/configuration/appSettings/add[@key='BrowserType']/@value”
value=“*firefox”
verbose=“true”/>

target=“runTests” />

name=“runTests”
description=“runs tests using Gallio.” >

message=“*** Start runTests: “/>

result-property=“exitCode”
failonerror=“false”
report-types=“Html;Xml”
report-directory=“${artifacts}”
report-name-format=“gallioresults”
show-reports=“false”
application-base-directory=“${path.base.test}”
>

>
name=“${path.base.test}${assembly.test}”/>
>
>

Set error for email injector to pick it up and GlobalFailBuildMessage for
the end target to fail the build after cleanup
–>
test=“${int::parse(exitCode)!=0}”>
name=“GlobalFailBuildMessage” value=“*** One or more tests failed. Please check the log for more details” dynamic=“true” />
message=“EmailInjectMsg=${GlobalFailBuildMessage}” />
>

message=“*** End runTests: “/>
>

And after the run of the Unittests Selenium needs to be shut down

src=“http://localhost:4444/selenium-server/driver/?cmd=shutDownSeleniumServer”
dest=“shutdown.txt” failonerror=“false”
/>

As I had a need to setup different configurations in TeamCity to run against different locations on the webserver I used a couple of Nant variables that are passed on the command line from TeamCity, like you normally would do when running Nant script -D:BaseUrlPath=/Test/ etc.

message=“*** Location variables passed from TeamCity”/>
message=“*** BrowserUrl=${BrowserUrl} “/>
message=“*** BaseUrlPath=${BaseUrlPath} “/>

Of course you get the Gallio UnitTest report as well

With this setup once we deploy to a server we can run all the browser tests on it using different browsers with one click of a button from TeamCity.

#swBoulder 2010 - Startup Weekend

orn — Sun, 25 Apr 2010 23:37:33 +0000

What does the #swBoulder hashtag stand for you may ask. sw stands for Startup Weekend the one I attended last weekend was held in Boulder they are held all over the country and in other countries well. Andrew Hyde is the founder of Startup Weekend and so much more, check him out.

Let me give you first a little bit of information what it is all about. Simply work in startup mode for a weekend on a project of your choice and see what you can produce with your team. It’s a event geared towards technology / software, however you don’t have to be developer. Most of the teams are made up of people with different backgrounds marketing, sales, business, developers and other skills as well. Everybody brings something to the table, it doesn’t matter what it is. If you like to contribute to a project in a startup settings this event is for you. Don’t worry about what you can do or what you can not do, there is room for everybody and everybody contributes. Some of the people working on the project do not have time to stay for the whole weekend that is fine as well, you contribute what you can. Of course it’s best to give the lead that information up front in order for the lead to know how you can be used as a resource over the course of the weekend.

And here is how it went down for me. The event was held at CU Leeds School of Business, people started gathering in around 17:00 on Friday. There was pizza and beer, the usual menu that most people can deal with. To kick it off Cameron Burgess and then Rich Grote ( SnapImpact ) had entertaining and informative talks to set the tone for the event.
Next up those who had an idea that they wanted to pitch got the stage for 5 minutes each. I did have an idea of my own but decided not to push it as the crowd of 40-50 people ended up with 13 ideas presented, there was plenty to choose from. Next the crowd voted on the ideas, which ones were to be worked on over the weekend, 8 of those were chosen. At this point you had to choose which idea you wanted to work on. There were two ideas I liked and I talked to both presenters then chose lloyddobllers idea from there. As the groups start to form the people in the group start figuring out plan of attack and take ownership of different aspects of what needs to be done. This is your opportunity to work on something that you feel is exciting or new. At this point our group broke up and we started poking at what needed to be done each in our own corners, I had an Avalanche game to watch and did not get much done rest of the night. We figured we would show up around 9 on Saturday morning.

Saturday morning we showed up early ready to work going as fast as we could. As in a normal startup, time is of essence and things move quickly, just keep going. For lunch Ayush Agarwal came by and had a talk about Startup Weekend and gave some tips about what the judges are looking for which benefited us greatly and then took questions. Then back to work, before you know it it’s getting close to midnight. We actually made the mistake of forgetting to have dinner and were running pretty low when we headed home for the night. Home to get some food and recharge the batteries.

Sunday morning after some hours of sleep we showed up again ready to rock it for the rest of the day. We really wanted to ship a product before the end of the day. It was recommended to us to ship on Saturday night, but how could we, we had plenty of hours to work in more features on Sunday. Along with a prototype of the application we put together a business plan and a slide presentation. Even a promotional video was made along the way. As expected it came down to the wire on the development side, we wanted certain features and finished up with 20 minutes to spare. we demoed the website live on our own domain during our presentation, no crashes…

During the presentations, the judges got busy both with questions and forming their opinions of the projects. 6 out of 8 projects made it to the finish line, props goes to one of the teams which lost all but two of their people through the course of the weekend but still presented a viable idea, way to stick with it. Apparently our project kicked ass and was chosen the best one by the judges. That was the icing on the cake for our hard work through the weekend, but regardless I think we would have been very proud of what produced regardless of what place we ended up at. Not only was the idea business viable but we managed to ship, have a good business plan, presentation, live demo of the product and exit strategy we even signed up couple of partners along the way.

In my case I had never done Twitter OAuth authentication and offered to work on that, now I know how that works. As we decided to use .net to code, I found code to reuse from .Net solution by Shannon Whitley thanks.

Here are pictures from the event from Lev Mazin from AskYourTrgtMrkt which was on hand allowing the teams to market research their ideas as they formed.

John Sheehan was also there from Twillio they sponsored the event. John happened to be helping out on my team and was a great asset.

This was my first Startup Weekend and I didn’t really know what to expect, sometimes as with this event I did little research by choice as I wanted to see how I would react to it myself. For me it was a breath of fresh air. Working hard for a weekend and to see the result at the end of it was a blast. Then coming back to work on Monday feeling energized ! It brought back some good memories from startups I had in the past. It is really a startup mode for a weekend very much like in the real world just more compact.

TeamCity replacing build agent

orn — Thu, 01 Apr 2010 21:37:41 +0000

We have the Professional version of TeamCity at work which allows 3 build agents / machines and 20 users. It’s big enough for our efforts. We just put an old build machine to bed and a new one is coming online. However I could not find how to remove agent from the TeamCity interface. First I uninstalled the agent on the old build machine, but the profile was still on the server. I ended up going into the Oracle TeamCity database and deleting the profile from the AGENTS table. After that I was able to add the new build agent in the TeamCity server console.

Custom file extenstion in Visual Studio C#

orn — Tue, 02 Feb 2010 18:14:58 +0000

I was using some files that have among other things some C# code in it in VS. When it dawned at me that I wasn’t getting the syntax highlighting which makes it much harder to read the code. Visual Studio just treats it as any other text file. In order to add a new one you just need to set new file association within VS. Tools - Options - Text editor - File Extensions. Add your extension and the editor you want to use, in my case Microsoft Visual C#. That will give you the code syntax, much better.

Custom Nant Task

orn — Sat, 23 Jan 2010 03:18:37 +0000

When using Nant there is not much you can say is lacking. Sometimes you find yourself wondering why you are cutting and pasting good amount of code between different scripts. There has to be a better way of reusing code and sure enough there is. Just write your own Nant extension and it’s pretty simple, let’s take a look.

First create a class that extends the nant Task then declare the name of the task and setup the properties you want available to the user. Lastly you will have ExecuteTask procedure that will be called by Nant when your Task executes. That’s all the magic !

[TaskName(“setVersionAssembly”)]
public class CxSetVersionAssembly : NAnt.Core.Task
{
[TaskAttribute(“file”, Required = true)]
[StringValidator(AllowEmpty = false)]
public string file
{
get { return msAssemblyFile; }
set { msAssemblyFile = value; }
}

[TaskAttribute(“buildNumber”, Required = true)]
[StringValidator(AllowEmpty = false)]
public string buildNumber
{
get { return msBuildNumber; }
set { msBuildNumber = value; }
}

// Executes the nant task
protected override void ExecuteTask()
{
// Don’t catch errors, it will display in the build log if any
Project.Log(Level.Info, “start setVersionAssembly”);
ChangeAssemblyVersionNumber(msBuildNumber, msAssemblyFile);
}
}

When you use it in your Nant script you will have to load your assembly first. Once your assembly has been loaded you can start using your custom Task.

assembly=“bin\debug\DE.Nant.Extensions.dll”/>

file=“${tempFile}”
buildNumber=“3.333.3.3000″
/>

This is a very brief and simple example, there are bunch of properties and other goodies you can use as you build out your custom Nant Tasks.

The Custom Nant Task I wrote is used to update the version of our .net assemblies during build. Attached is the complete code including Nant script and unittests. Get the code

Code coverage from Nant script using Gallio and nCover

orn — Thu, 14 Jan 2010 23:02:00 +0000

I decided to go with MbUnit for a new project, the newest version 3.x comes bundled with Gallio. Gallio is a test runner and can run loads of different flavors of tests. nUnit, msTest, MbUnit, etc. Of course once you have your tests running you wonder how much of your code gets coverage. To figure that one out I added NCover, here is a sample of how you can have nCover cover your Gallio UnitTest runs.

For best results install Gallio on the build machine and point to that directory when you load the task
loadtasks assembly=
also make sure your list of assemblies to be covered is just the assembly name, not the file name.
assembly.list=myAssembly1;myAssembly1;etc

name=“galliounittest”
description=“Runs MbUnit UnitTests using Gallio.” >
message=“*** Start Gallio unittest: “/>

assembly=“${path.gallio.task}Gallio.NAntTasks.dll” />

result-property=“exitCode”
failonerror=“false”
runner-type=“NCover”
report-types=“Html;Xml”
report-directory=“${artifacts}”
report-name-format=“gallioresults”
show-reports=“false”
application-base-directory=“${path.base.test}”
>
-property value=“NCoverArguments=’//w ${path.base.test} //a ${assembly.list}’” />
-property value=“NCoverCoverageFile=’${path.ncover.dir}${coverage.xml.file}’” />

>
name=“${path.base.test}${assembly.test}”/>
>
>
if=“${exitCode != ‘0′}” >One or more tests failed. Please check the log for more details>

message=“*** End Gallio unittest: “/>
>

name=“nCoverReport”
description=“Creates UnitTest Coverage report.” >
message=“*** Start nCoverReport: “/>

program=“${path.ncover.explorer.exe}”
projectName=“${PojectName}”
reportType=“ModuleClassFunctionSummary”
outputDir=“${path.ncover.dir}”
xmlReportName=“${coverage.xml.file}”
htmlReportName=“${coverage.html.file}”
showExcluded=“false”
verbose=“True”
satisfactoryCoverage=“1″
failCombinedMinimum=“true”
minimumCoverage=“0.0″>
>
name=“${path.ncover.dir}${coverage.xml.file}” />
>
>
type=“Assembly” pattern=“*.Tests” />
type=“Namespace” pattern=“*.Tests*” />
>
>

message=“*** End nCoverReport: “/>
>

NCover error

orn — Wed, 06 Jan 2010 22:50:20 +0000

I ran into a problem when adding NCover to our UnitTests on a new project the error I got when using NCover with Gallio was the following, Profiled process terminated. Profiler connection not established. The FAQ that is in the NCover directory bundled with Gallio suggested - If using the command-line, did you COM register CoverLib.dll ?

Sure enough, just run
>regsvr32 CoverLib.dll
from a command prompt, that fixed the problem on both our build machines.

Nant, Tf Unable to determine the workspace

orn — Tue, 22 Dec 2009 22:21:20 +0000

I was using Nant to checkout files in a build script and got the error “Unable to determine the workspace” from Team Foundation Server repository. The trick is to reset the cache first, run this command
tf workspaces /s:http://yourserver.com:8080 /noprompt
from your Nant script before doing any checkouts etc, then it should work.

C# download web page

orn — Fri, 11 Dec 2009 00:37:05 +0000

I’m sure you had the need sometime to download the content of a web page to be able to analyze it or work with the content. Here is a code snippet just for you, we will use simple http GET for the specified URL.

public static string getWebPage(string psUrl)
{
WebResponse result = null;
string sRet = string.Empty;

try
{
WebRequest req = WebRequest.Create(psUrl);
req.Method = “GET”;
req.Timeout = 3 * 1000; // 3 secs
// Explicit no caching, usually this is the default
req.CachePolicy = new RequestCachePolicy(RequestCacheLevel.BypassCache);

// Has to process the results if the responding service is spitting it out
result = req.GetResponse();
Stream ReceiveStream = result.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding(“utf-8″);
StreamReader sr = new StreamReader(ReceiveStream, encode);
// in case the caller is interested
sRet = sr.ReadToEnd();
}
finally
{
if (result != null) result.Close();
}

return sRet;
}

The solution was offline during its previous session and will remain offline

orn — Wed, 02 Dec 2009 18:33:02 +0000

If your .Net solution gets disconnected from your Team Foundation Server you might get the following message once you load your solution in Visual Studio. “The solution was offline during its previous session and will remain offline.” in the Output window and your solution is kept offline. Which means your local changes are not picked up automatically and you will need to do manual checkouts and checkins into the repository for local files you have changed.

The trick is to manually reset the values in the registry, where else ?
HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0\TeamFoundation\Servers\ ( Your TFS server name )

set the values
AutoReconnect = 0
Offline = 0

Next time you open up your solution you will be asked if you want to connect to your Team Foundation Server, yes please.

Update…
If that does not work, try this one