Iain Hoult

We’re Hiring

noreply@blogger.com (Iain Hoult) — Wed, 29 Feb 2012 10:08:00 +0000

Open Projects is a software house based in Nottingham UK, specialising in software for the newspaper/magazine circulation industry. There are vacancies for two new people to join the team.

A developer with skills in VB.Net, C#, ASP and SQL to work on Windows, web and mobile solutions.
An IT Support professional to handle second line support and to make sure none of the servers are on fire.

For more details and how to apply, please go to the Open Projects Careers page.

How to Convert Strings to Integers in Visual Basic

noreply@blogger.com (Iain Hoult) — Mon, 06 Jun 2011 20:53:00 +0000

While recently reviewing a piece of code, I managed to spot three different methods being used for converting values stored in objects or strings into Integers. This is where Visual Basic takes the record for having the most methods for performing the same operation. Thinking about it I realised that I could name six different ways of converting strings to integers in Visual Basic…

This post is a "back to basics" investigation into which way is best to convert to an Integer.

Meet the methods:

CInt, CType, Int, Convert.Int32, Integer.Parse, & Integer.TryParse

CInt, CType and Int belong to the Microsoft.VisualBasic namespace. Convert.Int32, Integer.Parse and Integer.TryParse are from the main body of the .Net Framework.

To find the best way of converting a string to an integer I have run each of the methods through a series of tests; the speed of each method, its ability to round number, handle Nothing (null) and completely invalid values.

Test 1 – Speed

In order to find the fastest method of converting a string to an integer each method was put through its paces converting the string “100” into an integer. To be able to see the difference in timings the operation was performed 100 million times. Each test run was carried out multiple times to achieve an average timing. Each batch of test runs were performed against three different version of the .Net Frameworks, 2.0, 3.5 and 4.0.

(These timings are specific to my PC and should only be used for comparison with each other)

The fastest method for .Net 2.0 and .Net 3.5 is Integer.Parse at 13.12 seconds and 13.23 seconds respectively, but is overtaken for .Net 4.0 by Integer.TryParse at 13.19 seconds.

It is nice to know that using the TryParse method does not mean you are getting the performance hit you might imagine for the extra checking. (During testing a Boolean variable was being set to the result of the parse test).

CInt, CType and Int performed badly in comparison to the other methods. This is conceivably due to these methods being there for backwards compatibility with old VB6 code, although as you will see in the next few tests they do more than the other methods.

Test 2 - Rounding

Sometime it is desirable to round values that are not whole numbers into integers, for this test each method was passed “0.5” and “0.6” to determine if it could successfully convert the value to an integer, and whether it would round the number up or down.

Method	0.5	0.6
CInt	0	1
Convert.ToInt32	Exception	Exception
CType	0	1
Int	0	0
Integer.Parse	Exception	Exception
Integer.TryParse	Failed	Failed

Only the Microsoft.VisualBasic methods handled the test, the others threw exceptions apart from Integer.TryParse which, being designed for this scenario, reported a failure.

Also it is interesting to note that Int will always returns the integer portion of the number, and so always rounds down for positive numbers and up for negative numbers. In researching this post I came across, if you can believe it, a seventh method of converting a string to an integer called Fix, which is very similar to Int but handles negative numbers differently.

Test 3 – Nothing (null)

These are the results from attempting to convert a string set to Nothing into an integer.

Method	Result
CInt	0
Convert.ToInt32	0
CType	0
Int	ArgumentNullException
Integer.Parse	ArgumentNullException
Integer.TryParse	Failed (0)

Interestingly the first three methods consider Nothing to be the same as zero, while once again Integer.TryParse rejects the value.

Test 4 – Invalid Integer

Strings can of course contain text as well as numbers, the following results are when attempting to convert the string “abc” into an integer.

Method	Result
CInt	InvalidCastException
Convert.ToInt32	FormatException
CType	InvalidCastException
Int	InvalidCastException
Integer.Parse	FormatException
Integer.TryParse	Failed

Well I think from these results that it is fairly obvious that the language designers do not think it is a good idea to be able to pass any old string into an integer conversion. I agree, I would prefer my applications to fail fast rather than perhaps silently returns zeros when passed random text (as with Nothing above).

Overall Results

So which is best and why? Well it all comes down to how you are going to be using the function and what sort of data you want the function to handle.

Unless you want the specific rounding abilities of Int then I can’t see any reason to ever use it. It performed badly against every test, making it the worst overall.

If you need speed then you should ignore the Microsoft.VisualBasic methods, as they are at least twice as slow as the others.

If you need to handle general rounding then CInt and CType are the only functions for you.

If the data being presented for conversion may not be an integer, then use Integer.TryParse to successfully determine this without throwing an exception, in the fastest time. The only catch with this method is that you need to remember to handle the function passing back a failure result.

Finally if you want the speed without the extra coding complexity that goes with handling the result from Integer.TryParse, the Integer.Parse looks best.

Creating Deadlocks

noreply@blogger.com (Iain Hoult) — Fri, 13 May 2011 16:46:00 +0000

I wrote a post a while ago on how to handle database deadlocks in your code, where I described how to programmatically detect and handle your code being chosen as the deadlock victim by SQL Server.
Running up against some deadlocking issues recently, I needed a way to reliably produce a deadlock so that I could test various scenarios. I thought it might be useful to someone who needed to do the same thing, so this is what I did.

Step 1 – Create the resources to lock

To begin with we need two tables and data that we can lock. (This is a prerequisite to actually producing the deadlock.)

create table ResourceA (DataFieldA int not null)
insert into ResourceA (DataFieldA) values (1)
create table ResourceB (DataFieldB int not null)
insert into ResourceB (DataFieldB) values (1)

Now we need a method of getting Process 1 to lock Resource A while wanting Resource B, and at the same time having Process 2 lock Resource B while wanting Resource A.

Step 2 – Process One to Lock the data

begin tran Process1
update ResourceA set DataFieldA = 2
waitfor delay '00:00:05'
update ResourceB set DataFieldB = 2
commit tran Process1

What happens here is that Process 1 locks Resource A, then while the lock is still being held there is a delay which gives us enough time to execute step 3, then Process 1 tries to get a lock on Resource B

Step 3 – Process Two to Deadlock

While Process 1 is delayed after locking Resource A, we can execute Process 2 which will lock Resource B before Process 1 can get to it.

begin tran Process2
update ResourceB set DataFieldB = 3
update ResourceA set DataFieldA = 3
commit tran Process2

This will cause a deadlock and result in one of our two processes being rolled back. The actual process that is chosen as the deadlock victim, will be selected based on which one SQL Server determines is the easiest to rollback.
So we have got to the point where we can create a deadlock, but we need a way of picking the process that we want to be rolled back. This is achieved by setting the deadlock priority on Process 1, to specify that it should not be chosen.

SET DEADLOCK_PRIORITY HIGH
begin tran Process1
update ResourceA set DataFieldA = 2
waitfor delay '00:00:05'
update ResourceB set DataFieldB = 2
commit tran Process1

Now Process 2 will consistently be chosen as the deadlock victim, giving you a reliable way of creating deadlocks in your database.

Vacancy for Junior Developer

noreply@blogger.com (Iain Hoult) — Wed, 16 Feb 2011 16:08:00 +0000

Open Projects are looking to hire a new Junior Developer, primarily to do .Net development.

If you live in or around Nottingham, and you are interested in joining a team where you will learn a wide variety of skills doing a whole host of different jobs, why not send in your CV.

Full details can be found here

Using jQuery with frames

noreply@blogger.com (Iain Hoult) — Thu, 27 Jan 2011 20:42:00 +0000

I’ve been having lots of fun recently getting my jQuery skills up to speed, one of the things that I found tricky, and hard to find any information on, was how to use jQuery with frames.

As there isn’t much out there on the subject I thought it may be useful to someone if I wrote a quick blog post on my findings.

The good news is that it pretty easy to get jQuery to work with frames, the main obstacle I ran up against when trying to find out how to do it were “helpful suggestions” online that didn’t actually work. I have a suspicion that it might be down to using IE8, as I had another IE/jQuery issue, but that’s post for another day.

To start with you need a page containing your frameset. The example frameset below contains two frames, one on the left and one on the right. Note that each frame has an “id” attribute.

<frameset cols="160,*" border="0">
     <frame id="left" scrolling="no" noresize="true" src="left.aspx">
     <frame id="right" noresize="true" src="right.htm" border="0">
</frameset>

The jQuery will be in the frame on the left (in “left.aspx”), while we will be acting upon the frame on the right (“right.htm”).

To change a setting on the other frame we use the second argument (context) in the jQuery function to specify the frame. The example below will change the text in a paragraph (“p”) tag.

$("p", window.parent.frames["right"].document).text("something new");

window=the current frame
parent=the frameset
frames=the collection on frames held within the frameset
"right"=the id of the second frame
document=what’s in the second frame

If you wanted, the id “right” could be swapped for the frame number, this is zero based so the second frame will be number 1, as in the example below.

$("p", window.parent.frames[1].document).text("something else");

To make the other frame navigate to a different page we use a similar technique.

$(window.parent.frames["right"].document.location)
            .attr("href","http://iainhoult.blogspot.com");

Or we can revert to using straight JavaScript, which is actually slightly shorter.

window.parent.right.document.location.href = "http://iainhoult.blogspot.com";

Deadlock Handling

noreply@blogger.com (Iain Hoult) — Fri, 26 Nov 2010 16:53:00 +0000

I was rewatching a presentation given by Udi Dahan at TechEd 2010 earlier this year, and I was reminded by his questioning of the audience about who was correctly handling their deadlocks, that really I wasn’t.

Before I get started on what I did about deadlocks, let’s take a quick refresher on the problem:

A deadlock is a stalemate situation, the simplest example goes like this; process one is holding resource A while needing resource B, at the same time process two is holding resource B while needing resource A. Each process is blocking the other so neither process can progress, at this point the database locking manager breaks the stalemate by killing off one process allowing the other to continue.

I am not sure how the locking manager decides who should be the victim of the deadlock, I am sure people much smarter than I am know, to me it seems to be tossing a coin in the air. There are ways to try and skew the outcome of this decision, namely setting the deadlock_priority of a session. Using this setting allows you to try and emphasise important and non-important tasks, by letting you configure a deadlock priority value between -10 (please kill me) and 10 (don’t touch me), with 0 being the norm.

Getting back to handling a deadlock in code, or my lack of, the point is that I was treating a deadlock like any other error, logging it and stepping back to a safe distance. But that is missing the point, it’s all in the error message:

“Transaction (Process ID %d) was deadlocked on %.*ls resources with another process and has been chosen as the deadlock victim. Rerun the transaction.”

Specifically it is the part of the message that says “Rerun the transaction”, because this is not really a failure message it is more of an instruction, to just wait until it is clear to progress.

So that is what I implemented, it couldn’t be simpler, if the code gets a deadlock wait for a bit and then try again.

As an example, I started out with something like this basic call to the database returning a DataTable:

public DataTable ReturnDataTable()
{
     using (var dt = new DataTable())
     {
          dt.Load(ActiveCommand.ExecuteReader());
 
          return dt;
     }   
}

(Any errors that occur, including deadlocks, will bubble up and be taken care of at a higher level).

To modify this function to handle deadlocks just needs us to catch any SqlExceptions and determine if they are deadlocks, if they are pause and try again:

public DataTable ReturnDataTable()
{
     using (var dt = new DataTable())
     {
          try
          {
               dt.Load(ActiveCommand.ExecuteReader());
          }
          catch (SqlException ex)
          {
               if (ex.Number == 1205)
               {
                    System.Threading.Thread.Sleep(2000);
                    dt.Load(ActiveCommand.ExecuteReader());
               }
               else
               {
                    throw;
               }
          }
                 
          return dt;
     }
}

Note that we are specifically checking for a SqlException with an error number of 1205 to determine if it is a deadlock (if you look at the messages within SQL Server you will see it is for the deadlock error mentioned above).

select * from sys.messages where message_id=1205 and language_id=1033

The length of time to wait may depend on what sort of process is likely to be blocking you, waiting 5ms might well be long enough. It would probably be useful to insert some level of randomness to the time paused, or to repeat the call several times within a short period.

The above C# function is only an intermediate step, it has two problems; firstly the logic used to call the database is repeated, and secondly the deadlock logic would need to be repeated in every method.

The solution is to create a deadlock handler function, to sit between the calling method and the method that calls the database:

private T DeadlockHandler<T>(Func<T> innerFunction) 
{
     T result;
     try
      {
          result = innerFunction.Invoke();
     }
     catch (SqlException ex)
     {
          if (ex.Number == 1205)
           {
               System.Threading.Thread.Sleep(2000);
               result = innerFunction.Invoke();
          }
          else
          {
               throw;
          }
     }
     return result;
}

Implementing the deadlock handler function as a generic function that takes in a Func<T>, keeps the code DRY, the database logic isn’t repeated and neither is the deadlock logic.

To use the deadlock handler with our example would need, a calling function:

public DataTable ReturnDataTable()
{
     return DeadlockHandler<DataTable>(InnerReturnDataTable);
}

And a function to communicate with the database (the original function):

private DataTable InnerReturnDataTable()
{
     using (var dt = new DataTable())
     {
          dt.Load(ActiveCommand.ExecuteReader());
          return dt;
     }
}

And that’s it, deadlock handling is not bad at all.

Errors, what to tell the user

noreply@blogger.com (Iain Hoult) — Fri, 15 Oct 2010 15:43:00 +0000

As much as we would like to believe our code is error free, that we never make mistakes and that unit tests make us bullet proof, there will always be bugs. It is a fact of life, it doesn’t matter if you are writing code on your own or for a massive corporation like Microsoft.

Obviously you don’t want to have any bugs and you should do whatever you can to try and stop them creeping in, but you can’t control every circumstance or predict every situation.

So my question is what do you do about it when you get an error? What should you be telling the user? How much technical information should you give?

(Warning contains content that some may find offensive)

How much detail?

Ok, let’s compare what I consider to be the two different ends of the spectrum:

Full technical details

I have used websites that have crashed and given me technical details. Is it any use to me at all to know there is a problem with the website’s database, or to get a stack trace? As the user of the website with no power to fix it, I would say no. The best thing I could do is take down all the details and pass them along to the website owner.

No details

On the Apple iPhone if an application has a problem, it just gets closed by the operating system, and that’s it it’s just gone no message no anything. When this happen to me I think “oh the app’s died” and I reloaded it. Any detailed message would have been a waste, again all I could do with it is pass it on to the author of the app.

To not display any error message to the user at all though, the idea that “there’s nothing they can do about it so why tell them anything”, I feel is a bit disrespectful to the user. I’m jumping ahead a little here, but wouldn’t a message like “App has encountered an error” with a button for Close and another for Reload perhaps be a nicer experience? Perhaps that is just a question of perception, “I have never had an error on my iPhone, sure apps sometimes close by themselves but it not like my PC where I get error messages all the time”.

Somewhere in the middle

A message dialog that initially appears with a user friendly error message and then, should the user wish they can expanded the technical details, is something I see on the PC a lot.

So we start with a user friendly message, which is good as it’s user friendly, which is what we are aiming for. As for the technical detail, is there anything we can do with it? Well that opens up a whole new area of discussion.

In the first two examples the user had no power over the environment they were using (website & iPhone), but this message is the sort that you might see on a PC, where there is at least the possibility that they can.

Let’s take this as an example error; the user is trying to save what they are working on and they get the user friendly message “Can’t save document”, with a technical message “File ‘accounts.xls’ locked by another user”. It isn’t in user friendly speak anymore (what does “locked” mean), but if you have been around PCs for a while and you have a level of confidence with them, then you can read this message as “the file I’m working on is being messed around with by someone else in the office”, then all that needs to be done is find out who it is.

So that was admittedly a very simple error to deal with and still lots of people that just use computers in their job, would be stuck by it. Most technical errors are going to be way more technical, which would be useful for any IT people trying to help the user, but not really for the users themselves.

The developer’s point of view

Let’s change direction now and think about this from the developer’s point of view; there are two types of error, expected and unexpected.

Expected errors

The simple example above is an expected error, the document was locked by another user the developer knew it was likely to happen and coded for it.

Unexpected errors

Then there is the unexpected error, which is the one developers should really care about (e.g. an OutOfMemoryException thrown deep within the application).

What to do with an error

What the developer wants is as many technical details as possible to be able to narrow down and squish the bug. What we have to be careful of is thinking we should show that information to the user, during development it might be nice to see it, but once it gets to the users it just turns into techo mumbo jumbo. In fact showing too much information could be disclosing details that make the application less secure.

What the developer should be doing is logging the details and getting them back. For a website that’s easy as the logging is local, other applications might need to have a customer error reporting system. There are standard error handling system that can be tied in, such as ELMAH. Microsoft’s applications tie into their Windows error reporting system, which you as a developer can also use.

And for the user the developer should be displaying the user friendly message, there’s no technical information that can help the user.

In conclusion

Going back to the error message spectrum I think that less is more helpful, in fact I think that this error message I received when using Google Reader was just the right level, "Oops...an error occurred. Please try again in a few seconds".

Steal This Presentation

noreply@blogger.com (Iain Hoult) — Sun, 19 Sep 2010 19:01:00 +0000

Just like in the title, I’ve stolen this presentation from my old friend Garry’s blog.

This is a really good presentation produced by Jesse Desjardins, demonstrating some excellent design tips. Some of the magazine layouts aren’t quite to my taste, but I just wish I had half of Jesse’s design skills.

Using jQuery To Asynchronously Load Page Elements

noreply@blogger.com (Iain Hoult) — Tue, 27 Jul 2010 13:15:00 +0000

I like jQuery, but every time I get starting using it I unfortunately seem to get pulled onto a different project. Recently I was working on a little ASP.Net pet project something that I couldn’t get pulled off just as it’s getting interesting, as part of the coding I wanted to find out how to asynchronously load page elements.

I wanted to do it using jQuery and I assumed it would be pretty straight forward (it’s even easier than I thought), but I couldn’t find anything that matched exactly what I wanted to do. Searching on StackOverflow I found a link to a tutorial with various jQuery exercises including asynchronously loading PHP pages.

The PHP example was fairly close, and enough for me to work out how to use jQuery to asynchronously load page elements in ASP.Net.

The following code is really a very simple example so that I don’t forget how to do it again. It is not meant to be a guide to learning jQuery from scratch. The example will load up a webpage and then asynchronously load up the contents of a drop-down list and display it on the webpage.

To start with we are going to look at the webpage that will initially be loaded up:

  1:     <div id="control">
  2:         <img id="progress" src="ajax-loader.gif" alt="Loading..." /> 
  3:         <select id="loaded" style="display:none"></select>
  4:     </div>

This is all that the body of the HTML contains; an image that displays an animated GIF as a progress indicator (something to show the user that a part of the screen is still loading and hasn’t frozen up). And a drop-down list which initially has no items listed and is not being displayed. These two tags are wrapped up in a DIV to make things tidy.

The magic happens in the jQuery:

  1:     <script type="text/JavaScript">
  2:         $(document).ready(function() {
  3:             $("#control select").load("DataGrabber.aspx", function() {
  4:                 $("#progress").hide();
  5:                 $("#loaded").show();
  6:             });
  7:         }); 
  8:     </script>

Once the HTML document has finished loading (line 2) we use the jQuery .load() command to request the data from the server for our drop-down list element (line 3), this is retrieved from "DataGrabber.aspx". After the data has been received another function is called, that will hide the progress indicator and display (now fully loaded) the drop-down list (lines 4 & 5).

The complete listing for the webpage looks like this:

  1: <html>
  2: <head>
  3:     <title>Asynchronous jQuery</title>
  4:     <script type="text/JavaScript" 
  5:         src="http://ajax.googleapis.com/ajax/libs/jquery/1.4/jquery.min.js">
  6:     </script>
  7: 
  8:     <script type="text/JavaScript">
  9:         $(document).ready(function() {
 10:             $("#control select").load("DataGrabber.aspx", function() {
 11:                 $("#progress").hide();
 12:                 $("#loaded").show();
 13:             });
 14:         }); 
 15:     </script>
 16: </head>
 17: 
 18: <body>
 19:     <div id="control">
 20:         <img id="progress" src="ajax-loader.gif" alt="Loading..." /> 
 21:         <select id="loaded" style="display:none"></select>
 22:     </div>
 23: </body>
 24: </html>

The webpage is the first half of the equation, the second is getting the data. This is the entire code for the "DataGrabber.aspx" page:

  1: <script language="VB" runat="Server">
  2:     Public Sub Main()
  3:         Response.CacheControl = "No-Cache"
  4:         
  5:         For counter = 1 To 10
  6:             System.Threading.Thread.Sleep(400)
  7:             
  8:             Response.Write( _
  9:                 String.Format( _
 10:                     "<option>{0}</option>", _
 11:                     counter.ToString))
 12:         Next
 13:     End Sub
 14: </script>
 15: 
 16: <%  Main()%>

This is all server side code and for demonstration purposes is just outputting OPTION tags in a loop for a count of 1 to 10. Note that within the loop is a sleep of 400 milliseconds (line 6), this is to simulate a server side operation that takes a few seconds to execute, a few seconds that without this asynchronous call the user would have to be waiting for the page to load.

As an additional point (taken straight from the API documentation) as something to watch out for:

Due to browser security restrictions, most "Ajax" requests are subject to the same origin policy; the request can not successfully retrieve data from a different domain, subdomain, or protocol.

On a side note I am using a nice animated GIF file as my progress indicator, it has a customised image and colour scheme. I got it from the AjaxLoad website which will generate your custom GIF file for you in seconds.

Junior Developer Wanted

noreply@blogger.com (Iain Hoult) — Fri, 16 Jul 2010 10:59:00 +0000

Open Projects are looking to hire a new Junior Developer, primarily to do .Net development.

If you live in or around Nottingham, and you are interested in joining a team where you will learn a wide variety of skills doing a whole host of different jobs, why not send in your CV.

Full details can be found here

Code snippets in your blog

noreply@blogger.com (Iain Hoult) — Tue, 15 Dec 2009 17:02:00 +0000

Before I start on this subject I had better lay out the facts; My blog is hosted on Google’s Blogger and I use Microsoft’s Windows Live Writer to compose my posts, on the whole the two play nicely with each other.

Previously when wanting to drop code snippets into a blog post I have used the source code formatter available on the manoli.net website. It is quite user friendly, you go to the website paste in your code and hit the “format code” button. The website will then give you the html that you will need to implant into your post. The first time you use it, you will need to setup the stylesheet information it needs in order for the code snippet to be correctly formatted.

The result is a nice looking code snippet that looks like this:

pages.Add( 
    new Page   
    { 
         Address = browser.Url.AbsoluteUri, 
         FileName = browser.Url.AbsolutePath, 
         Text = browser.DocumentText 
    } 
);

Although I had previously setup the stylesheet information, when I came to writing my last post, I though to myself “there must be an easier way to do this”. What I found was the Source Code Formatter plugin for Windows Live Writer.

Once installed you paste your code into the plugin’s window and when you are happy with it, it just goes straight into you post, no need to cut and paste html.

I think it looks a bit nicer too, being in a box and having alternating coloured lines (both are configurable options):

pages.Add( 
    new Page   
    { 
         Address = browser.Url.AbsoluteUri, 
         FileName = browser.Url.AbsolutePath, 
         Text = browser.DocumentText 
    } 
);

The nice benefit here too, is that there is no need to setup any stylesheet information before you begin.

It simple and pretty, I’m going to be using this one for a while to come.

A good guide to getting setup with the plugin can be found on Skot’s blog, where he goes into detail about installing and using the plugin, with some nice screen shots to show what’s happening.

It’s a free website, but it’s only static content

noreply@blogger.com (Iain Hoult) — Fri, 11 Dec 2009 16:18:00 +0000

I’m not sure or not whether this is a well known fact, but those nice people at Microsoft have been giving away free domain names and web space to anybody that wants it for some time now.
They have started charging for custom domain names, although you can still operate your website as a sub domain of OfficeLive.com. The rest of the basics are still free, with just a few added charges should you want extra bits bolted on.

It is designed for small businesses, and the basic way of setting up a website is to use existing templates, and just tweak them to customise the look. Advanced users get the ability to do away with the templates and write their own website from scratch.

The only major downside I’ve found when writing your own website, is the fact that it has to be static. That’s right, nothing is permitted to operate server-side, forget asp.net mvc, you are forced back to the 90’s. I guess it is some sort of security thing, but even so IMHO it is a tad on the harsh side.

On the plus side there is a way of getting round this, well sort of. You still can’t do anything “clever” with your website, but you can make it dynamic in a scenario such as displaying information on pages that is pulled out of a database.

Write your website to work dynamically then save the pages it generates and use those as your static content on your live website.

Admittedly manually doing this stops being scalable at the point you decide you have enough content to justify making your website dynamic in the first place. So manually doing it is out before you begin, but coding something up to do it for you couldn’t be simpler.

Use a WebBrowser control to read a page from your dynamic site

browser.Navigate(new Uri(address));

Store its key details

pages.Add( 
    new Page   
    { 
         Address = browser.Url.AbsoluteUri, 
         FileName = browser.Url.AbsolutePath, 
         Text = browser.DocumentText 
    } 
);

Then read all of the links off of the page

browser.Document.Body.GetElementsByTagName("A");

And store them, remembering to only store links that go to pages on your own website (unless you plan on mapping the entire web).

Now you just have to navigate to the first link you have recorded and read that page, repeat until you have visited every link on your website. At this point you can save the pages you have been storing to files.

public void Save() 
{ 
    string folder = @"f:\webfiles" + DateTime.Now.ToString("yyyyMMdd"); 
    if (!Directory.Exists(folder)) 
        Directory.CreateDirectory(folder); 
    string fullPath = Path.Combine(folder, FileName); 
    if (File.Exists(fullPath)) 
        File.Delete(fullPath); 
    using (var sw = new StreamWriter(fullPath)) 
    { 
        sw.Write(Text); 
    } 
}

This process could be quickly extended to generate sitemaps to allow search engines to more easily map your site.

On the other hand you could, for a small fee, get some web space where you can use PHP and MySql and write a really dynamic website.

Small World

noreply@blogger.com (Iain Hoult) — Wed, 24 Jun 2009 12:28:00 +0000

Isn’t it a small world, well that is the conclusion I’ve come to anyway, after having the chance to catch up with an old friend who I hadn’t talked to for a while.

Both of us being software developers the conversion soon got round to discussing what sort of tool/technologies we were both working with, I’m in the .Net camp while my friend is in the Java camp. So your first thought might be that there is not much in common between what we do (or perhaps you wouldn’t, I don’t really keep an eye on what’s happening in the Java world), but as it turns out we are doing very similar things.

He is working with Groovy and Grails, which as he put it is essentially a blend of Java, Ruby/Rails, Spring and Hibernate. Groovy being based on Java and leveraging lots of the ideas from dynamic languages. Grails sits on top of Groovy and provides an MVC framework, together with built-in support for database-abstraction/ORM (Hibernate) and user-interfaces (Spring).

The names are different but the concepts are all starting to sound familiar; Hibernate = nHibernate, Spring (which I’ve got to admit to being a little fuzzy on, even though it is also available for .Net) provides an MVC framework = ASP.Net MVC Framework, and an IoC system = StructureMap (our IoC system of choice).

As the conversation went on, the language became exactly the same; Iterations, Design patterns, Coding principles (like convention-over-configuration) and Continuous Integration, they’re using CruiseControl while we are using Team City (mainly because we like Jet Brains’ ReSharper so much).

We even talked about source control, which was the spookiest point, as not only were we both using svn, but we were both using TortoiseSVN.

Is it strange that we are doing the same sort of things? Probably not, as we are both trying to achieve the same goals (writing quality, reliable and maintainability software) and there is cross pollination of ideas and technology between the two camps, apparently much more than I realised.

SQL Server Tricks to Remember

noreply@blogger.com (Iain Hoult) — Thu, 04 Jun 2009 15:08:00 +0000

This post is really just somewhere to keep these SQL Server stored procedures so that I don’t forget them. They are a couple of useful system stored procedures that ship with SQL Server.

sp_change_users_login

Used for mapping database users against SQL Server logins.

To determine what users exist on the database that are not mapped to logins on SQL Server:
sp_change_users_login 'report'
To map a database user to an existing SQL Server login:
sp_change_users_login 'auto_fix', 'username'
To map a database and create an SQL Server login at the same time:
sp_change_users_login 'auto_fix', 'username', null, 'password'
(The null parameter is the SQL Server login)

sp_MSforeachtable

Undocumented stored procedure used for executing the same command against every table in a database.

To get the number of rows from each table in a database:
sp_MSforeachtable "select count(*) as '?' from ?"
(The ? character is used as a placeholder for the table name)
To get the number of rows for each table whose name starts with I:
sp_MSforeachtable
@command1 = "select count(*) as '?' from ?",
@whereand = "and o.name like 'i%'"
(There can be 3 different commands executed against each table per run, specified as @command1, @command2 and @command3. @whereand allows the selection of specific tables.)
There is also the ability to run a command before and after all the tables are processed, @precommand and @postcommand respectively.

If you look, it might well be there

noreply@blogger.com (Iain Hoult) — Thu, 26 Mar 2009 17:48:00 +0000

When the .Net Framework first came out I can remember seeing several articles that said we, as developers, needed to learn the different namespaces within the framework.

This was brought back to mind recently when I was reviewing a few pieces of code, and I realised that the authors of the code had been doing their own code rather than using the functions available to them in the framework. Specifically they where reinventing the System.IO namespace, the two functions in question were:

System.IO.Path.Combine – For concatenating a path with a filename to get a valid path/filename string.

and

System.IO.Path.GetFileNameWithoutExtension – As the name suggests for getting the name of the file minus its extension from a full path.

I think we all know the main parts of the framework, all the bits that we use all the time, but I bet there's lots of really useful stuff in there that could be a real help if we just took a little time to explore what is available.

I say “we” because I think I'm going to take my own advise and have a look through some of the namespaces that I normally never use. Who knows, I might make life a little easier for myself.

Build Your Own “Fluent Interface”

noreply@blogger.com (Iain Hoult) — Sat, 31 Jan 2009 11:15:00 +0000

As I was putting together a fluent interface recently I decided it might be a good topic to blog about, to try and show why they are a very useful way of doing things.

A fluent interface is really just a nice way of passing information into an object though your code. It’s really a very simple technique, so to give you an idea of how one works I will give you some comparisons against the one I put together.

I was doing some black-box testing of an application that validated information held in a database, to help me with this task I created a helper object that would insert a record into my test database. There were various fields associated with the record that I needed to be able to configure easily to create the correct data for each test scenario.

There were three different ways that I could have set this up to work...

Properties

The test method is the one exercising the helper object shown below. We create an instance of our helper, set its properties and then call the Create() function to do the database work.

    public void Test1()
    {
        HelperProp helper = new HelperProp();

        helper.Validated = false;
        helper.Downloaded = true;
        helper.ComplexMode = true;

        int testRecord = helper.Create();
        // Test code removed for clarity
    }

The helper exposes the various properties that are needed by the coder, and has a method to store the test record in the database.

    public class HelperProp
    {
        public bool Validated { get; set; }
        public bool Uploaded { get; set; }
        public bool Downloaded { get; set; }
        public bool ComplexMode { get; set; }

        public int Create()
        {
            return DB.ExecuteScalar(
                "INSERT into TestTable ({0},{1},{2},{3})", 
                Validated, Uploaded, Downloaded, ComplexMode);
        }
    }

The downside with properties is that setting them, can get a tad verbose. It might not look that much here, but I have cut down the number of options in the example considerable, plus this is just one of the types of record I needed to be able to control, and this is just the one test.

Constructors

This test method does the same thing as the previous one, but this time the options are all set via the helper’s constructor.

    public void Test2()
    {
        HelperCtor helper = new HelperCtor(false, false, true, true);

        int testRecord = helper.Create();
        // Test code removed for clarity
    }

The helper now just takes in those options into it’s constructor.

    public class HelperCtor
    {
        private bool _validated;
        private bool _uploaded;
        private bool _downloaded;
        private bool _complexMode;

        public HelperCtor(
            bool validated, bool uploaded, 
            bool downloaded, bool complexMode)
        {
            _validated = validated;
            _uploaded = uploaded;
            _downloaded = downloaded;
            _complexMode = complexMode;
        }

        public int Create()
        {
            return DB.ExecuteScalar(
                "INSERT into TestTable ({0},{1},{2},{3})",
                _validated, _uploaded, _downloaded, _complexMode);
        }
    }

With this method we have moved the verbose code out of the tests and into the helper object.

We really only want to set a couple of options but with this constructor all of them need setting, and answer would be to have multiple constructor of different formats, as we are dealing with boolean values this could become a pain if we only want to set certain combinations.

There is another downside with using constructors for configuring our helper object and that is readability. With the helper we are setting a lot of boolean values, so the call to the constructor, as can been seen in the test, is only a collection of true/false values not exactly the most intuitive code to read.

Fluent Interface

The test method is now very concise, there’s really only one line for the setup of the helper, plus it is very readable.

    public void Test3()
    {
        HelperFI helper = new HelperFI();

        int testRecord = helper.Downloaded().ComplexMode().Create();
        // Test code removed for clarity
    }

The magic is in the helper’s functions, they do two simple things; store the setting and return a helper (in fact the same one). That’s it, because each function returns the helper it is possible to chain the functions, making a fluent interface.

    public class HelperFI
    {
        private bool _validated;
        private bool _uploaded;
        private bool _downloaded;
        private bool _complexMode;

        public HelperFI Validated()
        {
            _validated = true;
            return this;
        }

        public HelperFI Uploaded()
        {
            _uploaded = true;
            return this;
        }

        public HelperFI Downloaded()
        {
            _downloaded = true;
            return this;
        }

        public HelperFI ComplexMode()
        {
            _complexMode = true;
            return this;
        }


        public int Create()
        {
            return DB.ExecuteScalar(
                "INSERT into TestTable ({0},{1},{2},{3})",
                _validated, _uploaded, _downloaded, _complexMode);
        }
    }

With the fluent interface version of the helper, there is more code, but only about the same as if we had used the longhand version of properties (in fact we could have used properties, which would have got rid of the brackets in the test method).

Fluent Interface - Extended

Sometimes it is not logical to give the coder all the options at once, so you can achieve a sort of hierarchy by returning different classes which reduce/allow different options.

Lets say that we wanted to restrict the options from the example, so that the coder must first choose uploaded/downloaded, then validated/ComplexMode, and only after that be able to call the Create() function.

The test method would be virtually identical to the previous one.

    public void Test4()
    {
        HelperHierarchy helper = new HelperHierarchy();

        int testRecord = helper.Downloaded().ComplexMode().Create();
        // Test code removed for clarity
    }

The helper I have split into four classes; the first one is just a dumb DTO to hold the option settings.

    public class Values
    {
        public bool Uploaded { get; set; }
        public bool Downloaded { get; set; }
        public bool Validated { get; set; }
        public bool ComplexMode { get; set; }
    }

The second is the top level class that the coder will use. It only has the options of Uploaded and Download, both of these functions return the next level class passing into it the DTO.

    public class HelperHierarchy
    {
        private Values _values { get; set; }

        public HelperSubHierarchy Uploaded()
        {
            _values.Uploaded = true;
            return new HelperSubHierarchy(_values);
        }

        public HelperSubHierarchy Downloaded()
        {
            _values.Downloaded = true;
            return new HelperSubHierarchy(_values);
        }
    }

The third level is quite similar to the second, this time only allowing the Validated and ComplexMode options to be available, both returning the final level again passing along the DTO with all the settings.

    public class HelperSubHierarchy
    {
        private Values _values { get; set; }

        public HelperSubHierarchy(Values values)
        {
            _values = values;
        }

        public HelperCreator Validated()
        {
            _values.Validated = true;
            return new HelperCreator(_values);
        }

        public HelperCreator ComplexMode()
        {
            _values.ComplexMode = true;
            return new HelperCreator(_values);
        }
    }

The final level has the Create() function, that just pulls the settings out of the DTO to do its database work.

    public class HelperCreator
    {
        private Values _values { get; set; }

        public HelperCreator(Values values)
        {
            _values = values;
        }

        public int Create()
        {
            return DB.ExecuteScalar(
                "INSERT into TestTable ({0},{1},{2},{3})",
                _values.Validated, _values.Uploaded, _values.Downloaded, _values.ComplexMode);
        }
    }

Just as a final note of interest there is another tricks that you can do with fluent interfaces, by using Extension Methods you can give an existing object a fluent interface.

Self Documenting Code 101

noreply@blogger.com (Iain Hoult) — Fri, 04 Apr 2008 08:11:00 +0000

Something that I am very keen on is "Self documenting code" by which I mean code written in such a way that makes it easy to read and maintain.

Ok, I know what you are thinking "my code is easy to read", well good I hope it is, this post isn't aimed at the developer with years of experience, but saying that how many times have you had to work on a piece of code that takes you longer to decipher than to do what you need with? How often after you got mad at that code did you realise that you were the author?

The more readable your code, the more maintainable your code, the less stress for you as a developer.

Let's start with a very small example method:

        private int Calc(Order[] orders)
        {
            int i;
            int result = 0;


            for (i = 0; i < 7; i++)
            {
                if (orders[i].Sales > 10.1f)
                {
                    result++;
                }
            }

            return result;
        }

So what does it do?

It takes some orders and loops round them seven times adding up where the sales are greater than 10.1 and returning the result, anything else would really be guessing. Now how did we work that out, well the method parameter is called "orders" of type "Order" so it's something to do with orders and it uses a property called "Sales" and oh yes there's a number "7" in the loop. If the parameter was called "os" of type "O" we wouldn't have a clue.

So what should we do to make it more readable? Add some comments perhaps:

        private int Calc(Order[] orders)
        {
            int i;
            int result = 0;


            // loop round seven times
            for (i = 0; i < 7; i++)
            {
                // if the sales value is greater than 10.1 increment the result
                if (orders[i].Sales > 10.1f)
                {
                    result++;
                }
            }

            return result;
        }

Comments are good, they are probably the most obvious way of documenting your code, but do these comments actually help? "loop round seven times" as a comment for a loop that goes round seven times, we can all read what the code is doing but can we tell why it's doing it?

Lets take the comments back out and this time change that variable "i" to something better:

        private int Calc(Order[] orders)
        {
            int day;
            int result = 0;


            for (day = 0; day < 7; day++)
            {
                if (orders[day].Sales > 10.1f)
                {
                    result++;
                }
            }

            return result;
        }

Now that starts to make things clearer, the method is looking at different day's orders, one order for every day of the week. IMHO giving that one variable a descriptive name has made the code much more readable than adding the comments did. Repeat after me, "single letter variable names are evil".

The next step is to get rid of those "magic numbers", that is the random numbers mixed up in the middle of the code:

        private int Calc(Order[] orders)
        {
            int day;
            int result = 0;
            const int daysInWeek = 7;
            const float salesTarget = 10.1f;


            for (day = 0; day < daysInWeek; day++)
            {
                if (orders[day].Sales > salesTarget)
                {
                    result++;
                }
            }

            return result;
        }

Oh, a sales target, that's what the "10.1" was all about, this method calculates how many days in a week somebody hits their sales targets! Still no comments, but now we can tell exactly what this method is meant to be doing.

If you look at the start of the loop, you will see that I have left the initial value of the "day" variable within the code, there isn't anything to gain in changing this unless it is likely to be modified. You could make the same argument about the "daysInWeek", but then there might only be 5 days in the working week.

The last thing we will look at is the actual name of the method, and I'm going to be a little cheeky here and refactor the method into two:

        private int SalesTargetHitDuringWeek(Order[] orders)
        {
            int day;
            int result = 0;
            const int daysInWeek = 7;


            for (day = 0; day < daysInWeek; day++)
            {
                if (IsAboveSalesTarget(orders[day]))
                    result++;
            }

            return result;
        }


        private bool IsAboveSalesTarget(Order order)
        {
            const float salesTarget = 10.1f;


            if (order.Sales > salesTarget)
            {
                return true;
            }
            else
            {
                return false;
            }
        }

I have extracted the logic that handles checking if sales have hit their target and put it into its own method. Now both methods have names that properly describe their purpose, the methods are small and only have the one responsibility described in the name. The names that I chose are long enough to describe the methods without being verbose, and thanks to IntelliSense you don't need to wear your finger away when using them.

Hopefully this quick example has illustrated some very simple techniques for making your code "self documenting", and we didn't even have to write any comments.

Phrases to develop by

noreply@blogger.com (Iain Hoult) — Thu, 03 Apr 2008 12:53:00 +0000

Over time I have picked up various programming tips that I have taken to heart, some of these are well known and I am sure you can find countless article on them, others are ones that I have picked up where I have worked, all of these are commonsense.

Fail Fast

Failing fast may sound a little backwards, but when you are developing code it is a good thing to have errors show up and to show up fast. The faster you know there is an error the faster you can fix it, and the more likely you are to catch it while developing your code, rather than after it has gone live.

The main way to do this is not to allow every bit of code to have its own error handling. "No error handling, that's crazy talk!", no I am not talking about a complete absence on error handling, what I am suggesting is the error handling is left at responsible levels in your code structure, where you can see them easily and deal with them.

Having low level methods handling their own errors can also leave you applications in an unknown state, which would make the security guys very excitable.

Keep it simple stupid (KISS)

That's not me being rude, blame the guy who thought up this phrase and needed a second S.

This one is really easy; keep you code simple, don't write big complicated methods. Do you like debugging big complicated method? No, so don't write them, write much smaller methods that are restricted to doing one thing (have one responsibility).

If nothing else when you come to debugging a problem, that stack trace will point you to a method with only a few line of code in it, much easier than a method that is a few pages long.

You ain't gonna need it (YAGNI)

Does our code need to be all things to all people? No, it needs to do the task for which it is designed.

If you know that in six months there will be extra functionality needed in an area you are working on, should you include code to handle it? You could, but in six months time that extra functionality might not be needed, even if it is then the specification will probably have changed leaving you needed to rework any code you have already added.

If you know that something you are working on is likely to change, don't ignore it, treat it as a hint so you know not to put anything in place that would block it. But don't spend your precious time trying to integrate it now.

If you ignore it, it will come back and bite you on the bum

A phrase I was told to remember when I first started out as a junior developer, and one I thought would go well after YAGNI.

This is not the opposite of YAGNI as it might first appear, in the example above you are not ignoring anything, you are just leaving it until a more appropriate time.

This is that completely random, a billion to one scenario, that the user will never get. You know the one, it's the one that is going to keep you debugging late into the night when it makes your app go funny and your customers get upset.

Write self documenting code

Not really a phrase, more a request I guess.

You should be able to go to your code and read it, and what's more you should be able to go back to your code in six months and still be able to read it. You should be able to go to your code and not need to spend half your time trying to workout what it is doing, before you can add your new feature or fix that bug.

The whys and wherefores of this are fairly basic, but there are quite a few so I will leave them for their own post.

Don't Repeat Yourself (DRY)

A very good piece of advice, that is a basic tenant of good design.

If you need to do the same sort of task in multiple places, don't rewrite that similar code, just do it once and have the calling code handle any differences.

For example your code might need to access a database for various task, you could have lots of different methods take on the responsibility of creating command objects against connections and executing them to get your data. The alternative is to create one method that takes on this responsibility, that the methods who need the data call passing in some SQL or stored procedure details. (Hopefully you know exactly what I'm talking about, if not Google "database gateway").

Another example could be where you have two objects that do similar tasks, the duplicated operations could be moved to a base class, that both your original classes inherit.

Assumptions are the mother of disaster

I first remember hearing this phrase when watching a film, it had nothing to do with computers but it is the sort of statement that can apply to anything.

It is very simple, if you make an assumption and it is wrong then it is a bad thing, if that assumption is at the heart of what your developing then it is a very bad thing. We all have to make assumptions, it is a balancing act, little assumptions are normally ok to sort out, it is the big ones you want to avoid.

I hope you found some of these phrases useful, I certainly have over the years.

Extension Method – What, why, where, when?

noreply@blogger.com (Iain Hoult) — Tue, 18 Mar 2008 15:05:00 +0000

Ever since .Net 3.5 and Visual Studio 2008 came out there has been lots about its new features, such as Linq and Lambda Expressions, but a nice little feature that seems to be getting overlooked is the ability to create Extension Methods.

What is an Extension Method?

Have you ever been frustrated that an object (closed off to you) doesn’t have a particular method, something that would seem the most logical little method for the object to have, but it just doesn’t, I have.

Well now those nice folk at Microsoft have come up with a way for us to get that missing method, they have given us the ability to add methods to an object to extend it, the aptly name “Extension Methods”.

For our example below, I could have gone really basic with just enough code to demonstrate the principle, but I like to give fuller examples. Ok, time for a little code...

Here is a very simple email gateway class:

using System.Net.Mail;

namespace ExtensionMethods
{
    public class EmailGateway
    {
        private SmtpClient _smtpClient;
        private MailMessage _mailMessage;

        public EmailGateway()
        {
            // Create mail server and email objects
            this._smtpClient = new SmtpClient("mailserver");
            this._mailMessage = new MailMessage();
            // Set fixed sender's address
            this._mailMessage.From = new MailAddress("address@domain.com");
        }

        public void SendEmail(string toAddress, string subject, string body)
        {
            // Configure email with passed parameters
            this._mailMessage.To.Add(toAddress);
            this._mailMessage.Subject = subject;
            this._mailMessage.Body = body;
            // Use mail server to send email
            this._smtpClient.Send(_mailMessage);
        }
    }
}

With the code to exercise it:

private void EmailSomeone()
{ 
        EmailGateway emailGateway = new EmailGateway();
        emailGateway.SendEmail("someone@domain.com", "test email", "this is a test");
}

This code is fine, until you try to send an email to multiple people, for example:

emailGateway.SendEmail("someone@domain.com;other@domain.com", "test email2", "this is another test");

When we try this we get a FormatException because the MailMessage.To.Add() method is only able to handle one email address at a time. This is the point where I would like to call a nice AddRange() method, except unluckily for us there isn’t one.

There is no silver bullet to get more that one address added to our email, somewhere we are going to have to write some code that takes a collection of addresses and added them to the email. The question is how can we make our code as elegant as possible?

Behold the Extension Method:

using System;
using System.Linq;
using System.Net.Mail;


namespace ExtensionMethods
{
    public static class MailAddressCollectionExtensions
    {
        public static void AddRange(this MailAddressCollection collection, string toAddresses)
        {
            // Remove any existing addresses, so our range are the only ones
            collection.Clear();

            // Parse the passed addresses, adding each one to the colection
            toAddresses
                .Split(";".ToCharArray(), StringSplitOptions.RemoveEmptyEntries)
                .ToList()
                .ForEach(address => collection.Add(address));
        }
    }
}

The extension method is a simple static method with one special part, in the method signature the first parameter is the object we are extending, notice it is preceded by the this keyword. This has to be the first parameter, any other parameters we define are the ones that will be pass in, so we are extending _mailMessage.To and passing in toAddresses.

And to use it we simply need to change the "To" line in our email gateway code:

public void SendEmail(string toAddresses, string subject, string body)
{
        // Configure email with passed parameters
        this._mailMessage.To.AddRange(toAddresses);
        this._mailMessage.Subject = subject;
        this._mailMessage.Body = body;
        // Use mail server to send email
        this._smtpClient.Send(_mailMessage);
}

Why?

Extension methods were added to help Linq work with the existing System.Collections.IEnumerable and System.Collections.Generic.IEnumerable(T) types.
To tell if a method is an Extension method or not is really simple, just look at the little icon that Intellisense gives you, if it’s got a little blue arrow on it pointing straight down then you’ve got yourself an Extension method.

Where?

The example has the Extension method in its own class, you don’t need to do it that way, in fact I am always cautious about adding “utility” classes into projects, they have a tendency to start filling up with random code that should really be somewhere else.

When?

The purpose and use of Extension methods are to extend closed objects, that really says it, they are not there for us to extend our own objects, but ones that we have no control over and wish we did.