Steve Trefethen's Weblog

Fixing NameMapping issues in TestComplete when UI changes occur

Steve — Tue, 03 Nov 2009 11:20:42 -0800

Just posted a new video to tv.falafel.com illustrating how to resolve issues with automated tests when there have been UI changes to the application you’re testing. UI changes can be especially difficult for test automation when you’re relying on a static NameMapping of elements in the UI.

If you’re working with TestComplete be sure to checkout our other related videos.

Visual map of my hard drive

Steve — Sun, 01 Nov 2009 20:43:05 -0800

Lately, I’ve been combing through my drive and uninstalling old bits like VS.NET 2005. I’ve been using WinDirStat which is a great tool for performing such a task and it got me wondering how other people’s drives looked. There are a few things of note in this picture (click to enlarge) and one of those is the size of my winsxs (11.8GB!) and Installer (4.1GB!) folders. I’ve Googled on these and without spending a tremendous amount of time it appears there’s not much you can do to shrink these things which is rather disappointing.

Clearly, I haven’t marked every region of this image though I’ve highlighted what I think are the more interesting parts. I’d like to know if others have winsxs folders that are this large and if anything can be done to shrink it down. It seems to me that it would eventually force a reinstall of the OS and surely seems like a place bits go to die.

Thoughts?

CruiseControl.NET VS.NET Starter Kit for plugin development

Steve — Fri, 30 Oct 2009 10:59:42 -0800

At the Silicon Valley Code Camp 2009 I gave a talk called Extending CruiseControl.NET through the use of plugins. I discussed the necessary steps and illustrated with an example ISourceControl provider using the LinqToTwitter OS project on Codeplex. The provider polls a configurable Twitter account looking for Tweets that that start with “CI:” allowing the Tweeter to trigger a build of the project simply be tweeting something like: “CI:Start the build”.

It’s a simple example but illustrates how easy it is to create CC.NET plugins an extend the platform to uses beyond classic Continuous Integration.

Here is a link to the Starter Kit. Btw, you will need to update the ThoughtWorks assembly references to match your build of CCNET.

Switch from dasBlog to BlogEngine.NET

Steve — Wed, 28 Oct 2009 22:33:00 -0800

I finally pulled the trigger on switching to BlogEngine.NET. I tweaked it a bit to maintain all of my old blog post links and probably lost a few links here and there but the switch is made. I'm going to run with this for a few days and see how I like it.

Please let me know if you notice anything weird. Btw, if you see any $g(...) macros those were previously Google search links under dasblog which aren't supported here.

One of the major things I hope this will resolve is the issue of dasBlog losing comments.

Using Google Playground Examples in VS.NET

Steve — Tue, 06 Oct 2009 05:43:00 -0800

I’ve been working on ways to leverage Google’s Calendar API to publish Falafel’s training calendar entries to the web. One of the things I stumbled into was the Google Code Playground which is a cool tool for testing out various Google API’s. The Playground includes Firebug lite which is also something I hadn’t run into, I love finding new cool things!

At any rate, this post is about how to get from the Playground to something live. In the screenshot to the right you can see the JavaScript source code in a code editor at the top right hand side of the screen. The missing piece here is that to get the code from the Playground to work on your own site you need to do a few things. First, you have to get an API Key. Once you’ve done that you can then add a <script> tag to include the Google GData API in your page.

    <script src="http://www.google.com/jsapi?key=<your_key>" type="text/javascript"></script>

After, that you need to initialize the GData API and get the code from the Playground to execute. To do that I did two things.

Wrapped the JavaScript code from the Playground page in a function.
Added the following code to my page with a reference to call the JavaScript function

    <script type="text/javascript">
        //<![CDATA[

        google.load("gdata", "1");
        google.setOnLoadCallback(loadItems);
        //]]>
    </script>

”loadItems” above is the name of the function I created in Step 1. Next, I had to add a DIV tag to the page as a target for the new content:

<div id="content">
</div>

At this point, the page (plain HTML) can be debugged from VS.NET and/or uploaded to your domain and function correctly. Anyway, I learned a few things along the way and thought this might be useful.

dasBlog provider for use with BlogEngine.NET

Steve — Mon, 14 Sep 2009 08:36:00 -0800

I’ve used dasBlog since I first began hosting my own blog and previously considered a move to BlogEngine.NET (BE) but the URL’s aren’t compatible so there remains some investigation to figure out the best mechanism prior to switching. Unfortunately, dasblog seems to be on it’s final legs so spending time there at this point doesn’t seem wise.

To that end, I’ve been investigating various issues involved in a switch and in an attempt to make a transition easier I’ve created a BE provider which reads/writes blog posts and comments using dasBlog’s IBlogDataService rather than BE’s own XML provider. Basically, I’ve replace one XML provider with another, actually that’s not entirely true as this new provide is a descendant of XmlBlogProvider included with BE. Using this approach I avoid having to constantly re-import my blog data as I look into solving the URL issues.

I’ve made the code available via SVN on Google Code here so feel free to grab it and give it a go. I chose Google Code over Codeplex because the latter doesn’t want abandoned projects and I don’t see this as an ongoing project as it’s scope is quite narrow. What I’m interested in is any feedback regarding issues using the code and taking this sort of approach. In my initial playing around with BE it seems to function properly and allows me to use the “slug” feature of BE to “fix” at least part of the URL problem by simply removing spaces from Titles rather than replacing then with dashes.

Things I’ve tested (manually)

Adding/Editing/Deleting posts
Adding/Editing/Deleting comments

I haven’t played with images or attachments though my blog data seems to render fairly well. I can see some limitations such as no support for threaded comments though I don’t view this as a long term solution but I think it’s a step in the right direction.

Things that need testing/implementation:

DateTime conversions from dasBlog to BE
Image uploads
Windows Live support

Btw, I’m looking for help with the problem of retaining my existing URLs so if you’re a dasblog user and potentially interested in switching ping me and perhaps we can collaborate.

Using TestComplete Stores and Checkpoints

Steve — Fri, 11 Sep 2009 21:52:00 -0800

I’ve recorded a video for Falafel TV illustrating how to use TestComplete’s Stores and Checkpoints features including v7’s new Checkpoint Wizard dialog. In the video I demonstrate recording a Keyword test and leveraging various checkpoints to validate the test has performed as expected. The checkpoints illustrated in this video include:

Object
Table
File

If you have questions or have suggestions for future videos leave a comment. Also, if you’re looking for training or consulting on TestComplete feel free to contact me or call the Falafel office at (831) 462-0457.

One more note, Falafel is hosting a TestComplete Online Training Summit Oct 5-7th or November 9-11th which you can learn more about and sign up for here.

Other videos:
Distributed Testing Using TestComplete
HTTP Load Testing Using TestComplete

Text file processing with LINQ

Steve — Wed, 09 Sep 2009 07:15:00 -0800

After working on this problem the other day I started Googling looking for posts written about using LINQ for text file processing. I found the post Parsing textfiles with LINQ (or LINQ-to-TextReader) by Arjan Einbu.

LINQ shows us alternate ways to write code, introducing a more declarative coding paradigm. To use LINQ over the lines of a file, we can read all the lines in the file into a collection, and use LINQ over that collection. There’s some overhead to this; the need to read the entire file upfront and to fit the entire file in memory at once.

The solution was to create an extension method on TextReader for IEnumerable<string>. That post was followed up by another post, rather unfortunately titled, improving upon the solution using TextFieldParser class in the Microsoft.VisualBasic.FileIO namespace, something I wasn’t aware existed and now find it odd this class is stuck well off in left field.

One of the reasons this subject interests me is I’ve been working with EDI files for awhile now and querying data directly from this file format would be really nice. For example, given a PO with line item segments like this:

PO1*1*36*CA*11.15*PE***VP*RRSKRC85*PI*0001111091127~
PID*F****PRSL ROMNE BBY TRAY ORGNC~
PO1*2*84*CA*11.15*PE***VP*RSMKRC85*PI*0001111091131~
PID*F****PRSL SPRG BBY TRAY ORGNC~
PO1*3*84*CA*11.15*PE***VP*RBSKRC85*PI*0001111091128~
PID*F****PRSL SPNCH BBY TRAY ORGNC~
PO1*4*72*CA*11.15*PE***VP*RHEKRC85*PI*0001111091126~
PID*F****PRSL SPRG W/HRB CLM ORGNC~

You can calculate the total quantity, highlighted in yellow, of all line items using LINQ like this:

using (var reader = new StreamReader("c:\\edi\\inbound\\850_09022009_1311_89.txt"))
{
    var query = (from line in reader.GetSplittedLines("*")
                            where line[0].Equals("PO1") && line[2].Length > 0
                            select Convert.ToInt32(line[2])).Sum();
    ...
}

Using Arjan’s implementation of GetSpittedLines, that’s his name not mine for the extension method he wrote, you can apply logic to any of the columns from the file which is pretty cool.

Of course, there are a myriad of ways of doing the same thing but it’s interesting to have access to the columns allowing for calculations and querying. For my EDI work I’m using FileHelpers which works well though I really like this LINQ option. That said, I haven’t done any benchmarking so I’m not sure about the performance but most of the PO’s I’m working with are less than 4KB and the volume isn’t so great that this would be a major factor. At any rate, I hope you find useful for you too.

Btw, if you’re looking for custom EDI implementations feel free to contact me.

Distributed Testing Using TestComplete

Steve — Tue, 08 Sep 2009 04:46:35 -0800

I’ve recorded a video for Falafel TV illustrating how to use TestComplete’s distributed testing features. In the video I demonstrate recording a Keyword test and adding the necessary support to allow it to be executed in a distributed fashion. If you’re just getting started this is a good introduction and should get you headed in the right direction. I’ve got some notes for an advanced video on the subject but if you have suggestions let me know.

If you have questions or have suggestions for future videos leave a comment. If you’re looking for training or consulting on TestComplete feel free to contact me or call the Falafel office at (831) 462-0457.

One more note, Falafel is hosting a TestComplete online training Summit Oct 5-7th which you can learn more about and sign up for here.

Mail Server Log MIME Header Parsing

Steve — Sun, 06 Sep 2009 05:12:35 -0800

The other day a friend, whose not a developer, approached me looking for help with a problem they were having dealing a few large server log files. The file contained 100’s of MIME headers listed one after the other each with a starting comment and separated by a blank line. I was given a sample containing 1400 MIME headers which at first I opened in Windows Notepad and looked like what you see to the right. They had been trying to work with this file using Excel but not having much luck because the headers were inconsistent sizes and I imagine extracting the right fields was clearly a problem. The desired result was a format from which they could perform some analysis of the data and particularly of the X-Originating-IP field.

After about 10 seconds of staring at the data in Notepad I opened it in Notepad++ and things looked a bit more sane and it sort of dawned on me what my friend’s first thoughts probably were when they first glanced at this file. Excel looked better but didn’t make the process any easier.

A few choices entered my mind:

Write a simple parser, regex etc.
Look for an existing MIME parser
Use VS.NET editor Macros to extract the content
grep/findstr

My focus desire was to:

not spend much time
produce a CSV file
do something my friend could duplicate (lessen “support”)

I opted to search for a MIME parser largely because I figured one written in C# had to exist then write a tool to spit out a CVS file. My first Google search was “parse email header C#” which gave me a few interesting links but nothing that really caught my eye. The next attempt was “parse MIME header C#”:

Bingo.

The CodeProject article is largely code snippets and the first one looked interesting.

Mime m = Mime.Parse("message.eml");
// Do your stuff with mime

I thought, if there is .eml message parsing then I’m good regardless of the fact that it was expecting a file. I downloaded the source and it compiled without error, always encouraging. Next, I looked into the parsing support:

// Summary:
//     Parses mime message from byte[] data.
//
// Parameters:
//   data:
//     Mime message data.
public static Mime Parse(byte[] data);
//
// Summary:
//     Parses mime message from stream.
//
// Parameters:
//   stream:
//     Mime message stream.
public static Mime Parse(Stream stream);
//
// Summary:
//     Parses mime message from file.
//
// Parameters:
//   fileName:
//     Mime message file.
public static Mime Parse(string fileName);

Sweet.

For testing I saved off a single MIME header and created a simple console application to try and parse a fake .eml file which worked like a charm. All that was left to do was write some code to read the log file one header at a time and spit out a .CSV file.

I made one minor change to the MIME parsing code which was to change it’s HeaderFieldCollection from an IEnumerable to IEnumerable<HeaderField> so as to leverage LINQ to search for the “X-Originating-IP”. Of course, I later found out that the code attached to the article is outdated.

At any rate, I quickly had the file parsed, output to .CVS using a simple console application with input and output filename params which I mailed off. So, if you’re looking for MIME header parsing this library worked well for the 1400 headers I tried and I’m glad I could offer this tiny bit of help in a situation that sounds very serious for the folks involved.

Btw, kudos to Ivar Lumi for making this available, heck I think writing this post took longer than developing the solution.