Doogal Bell's bloggy thing

I don’t rate Rated People

2012-05-24T21:44:00.001+01:00

You’ve probably seen the adverts where Phil Spencer, the man whose remarkable negotiating skills can sometimes get £5,000 knocked off a £500,000 house purchase, tells us about this remarkable website which can connect us punters with local tradesmen. To me this seems like a good idea for a website and I have some work that needs doing, so thought I’d give it a spin.

So I filled in my details, entered the job details and waited for a response. And then, nothing. OK, not entirely true, I received a couple of text messages and emails. A couple of days passed and I received an email telling me that I hadn’t had a response and maybe I should expand on my job description to get a response. So off I went and updated the description and hit submit and then I was presented with a 404 error (for the non technical, this means the website is broken). So I tried again and got the same result (definition of insanity - doing the same thing over and over again and expecting different results). And I gave up at that point.

OK, slightly annoying, but no worries, I’ll drop them an email and let them know and I’ll update the description later when it’s fixed. So I go to their contact page. I can ring them, but not at this time or I can send them a tweet or write on their Facebook page or add a comment to their blog. No feedback form, no email address. I didn’t particularly want to berate them on a public forum, maybe the problem is something specific to me, so none of those options appealed. Yes, yes, I see the irony, I am now berating them quite publically (hello my many reader), but I’m annoyed now.

So, so far, Rated People is something of a failure for me. I think if you’re going to be spending a bunch of money on TV advertising, quite a bit of effort should also be spent on the product…

Smart phone + app + bike mounting > bike computer

2012-05-13T21:14:00.001+01:00

I recently took possession of a new Nokia Lumia 800, thanks to work, and immediately thought it would make a great replacement for my Veloset GPS 600 bike computer. Hearing Windows Phone didn’t have many apps, I thought I’d have to write my own biking app, but it turns out there are already a couple available, both free.

The first is MapMyRide. This initially seemed pretty good, but it has one fatal flaw, it lets the phone go to sleep. If you want to look at the map as you ride it is pretty tricky. You really don’t want to be fiddling with your phone when you’re riding.

Next up is MyBikeMap, which has one big advantage in that it doesn’t let the phone sleep, so I can look at my map all the time during my ride. It’s also well designed, with a simple user interface and showing just the pertinent information on the screen displayed when riding. It’s not perfect, my main complaint is the lack of support for miles, but it’s hard to complain too much when it’s free and it does most of what I want.

The final piece of the puzzle is something to attach the phone to my bike. I went for this mounting, primarily because it came up first on a Google search. The phone fits it perfectly and I was pleased to realise I could leave the zip open slightly to still have access to the buttons and I was able to use the touch screen through the plastic cover. But it does have a somewhat major problem as I found out today. Although the coupling between cover and bracket on the bike is perfectly adequate when riding on a road, if it gets a jolt then the cover (and the phone) can go flying. And you might not even notice until you look down some time later.

The design of the mounting is kind of weird, the cover is attached to the bracket not once but twice. One of these couplings is pretty secure, but the other isn’t. So the solution I’ve come up with is to glue the weaker coupling and I’m hoping this solves the problem. But you may want to consider this issue before purchasing this mounting.

But other than the teething problems, I’m pretty happy with this set up. It’s great to have a map in front of me as I ride, since it gives me much more opportunity to try heading off down a road or track that I don’t know without worrying about getting completely lost. And it makes me think the market for high end bike computers may not last much longer. Why spend £200 on one of them when a smart phone can do the same job?

KmlLayer error handling in Google Maps

2012-05-12T21:09:00.001+01:00

Two years ago I noted that the KmLayer in Google Maps didn’t provide an event to inform me if there was a problem when loading the KML. Things have moved on and an event has been added to the API at some point. Usage is as follows.

  var kmlLayer = new google.maps.KmlLayer(url, { map: map });
  google.maps.event.addListener(kmlLayer, 'status_changed', function() {
    if (kmlLayer.getStatus() == 'OK')
      $('#status').html('');
    else
      $('#status').html('KML loading problem - ' + kmlLayer.getStatus());
  });

How to increase your AdSense revenue

2012-04-14T22:32:00.001+01:00

My proper website has been around for about 12 years. For most of that time it had very few visitors and the money I made from advertising was minimal. Then a couple of years ago advertising revenue started going up. It’s now plateaued but I earn a nice wedge of cash ever month, not enough to give up the day job but enough to add a decent amount to my income. So what’s my secret and how can you do the same thing?

Firstly, you can optimise the placement, colours and number of AdSense units. Now this may well improve your revenue but the problem is it is almost impossible to figure out if revenue increased due to changes to your ads or just due to random fluctuations. Take a look at my daily income over the past month.

There’s a lot of volatility in those figures. Admittedly you can slice the data in different ways which can produce better results but even with no changes to my ad setup, the daily numbers are all pretty volatile. And if you aren’t making a lot of cash from your ads, your figures are likely even more volatile. So I figure if you want to optimise you ad setup, then you need to be looking at weekly or monthly numbers, so the feedback loop on changes is going to be pretty slow. To be frank the limit of my optimisation of ads is to increase the number of ad units to three (the maximum allowed and also probably the maximum number that wouldn’t be too annoying to a user of you site).

So what did cause the increase in my income? Simple really, an increasing number of visitors. My graph of monthly visitors and my graph of monthly AdSense revenue are almost identical. So unfortunately the answer to the question of how to increase AdSense revenue is another question, how to increase the number of visitors to your site.

And the answer to that question is actually pretty straightforward I reckon. The first part is fully within your control - Produce a lot of content of a reasonably quality. i.e Content that search engines will consider to be of value and hence index. The second part isn’t completely within your control, gain some inbound links from reputable sites. I actually think this will naturally follow from producing content that people appreciate.

In some ways I can’t help thinking this is the business model that Facebook has adopted, get a shedload of content (although in their case its helpfully created by their own users) and shove ads on it.

Fun with flags

2012-04-13T20:18:00.001+01:00

The Big Bang Theory hasn’t been quite the same since the geeks started to pair off with girlfriends (since half the humour was about them failing spectacularly in their endeavours with the opposite sex), but Sheldon’s recent video podcast ‘Fun with flags’ was rather splendid.

Creepy ads

2012-04-07T22:07:00.001+01:00

I’m used to visiting a website and their adverts then following me round the web afterwards. It’s happened with Dell and some property website. They even seem to make an effort to show me relevant stuff (laptops I was looking at or properties in the area I looked at). Then after a few days they disappear. I find it a bit creepy and it’s a bit of an eye opener to realise what the ad companies know about me (or about the cookie that’s sat on my machine).

But there’s an advert that’s been following me around for months now, for LeanKit Enterprise Kanban. I’m pretty sure this ad can’t be getting shown to everybody on the web, since this is a pretty niche product, so I can only assume it’s appearing because I visited their site ages ago. This has gone beyond creepy, this is just plain weird. Is showing me the same ad for months on end actually effective? Are the advertisers paying a premium for it?

Cross browser selectSingleNode for XML in Javascript

2012-04-06T10:53:00.001+01:00

Internet Explorer has a non-standard method available in XML documents called selectSingleNode. Pass in some XPath and it returns the first node that matches the XPath. Other browsers have ways of doing the same thing, but they are more long winded. So in short, I like the IE implementation and since I don’t fiddle with XML in JavaScript that often I often forget that selectSingleNode is not supported on all browsers (and if you’re wondering why I use IE as my primary browser, it’s because it’s still, just about, the most popular browser out there).

So here’s a cross browser version of selectSingleNode (not my own work, copy and pasted from somewhere I can’t remember on the web)

    function SelectSingleNode(xmlDoc, elementPath) {
      if (xmlDoc.evaluate) {
        var nodes = xmlDoc.evaluate(elementPath, xmlDoc, null, XPathResult.ANY_TYPE, null);
        var results = nodes.iterateNext();
        return results;
      }
      else
        return xmlDoc.selectSingleNode(elementPath); 
    }

Business Optix blog now live

2012-03-19T20:53:00.001Z

The blog of my employer Business Optix has gone live. We’ll be posting there as we roll out new features to our web and desktop products, along with tips and tricks to use the current software and any other company news. Keep up to date by signing up to the RSS feed.

Veloset GPS 600 bike computer review

2012-03-10T17:35:00.001Z

My first foray into the world of bike computers was a Sigma 1609 device. I quite liked it, since it was simple to use with a clear display, big chunky buttons and a very low price. The only minor downside was a certain amount of faffing around required to get it configured initially since it needed to know how big my wheels were.

But one day on my ride to work, a driver decided it would be a good idea to pull out at a junction without looking to see if anybody was coming and knocked me to the ground in the process. As I dusted myself off at the side of the road I heard a crunching sound and realised me and my bike had successfully got to the road side but my Sigma hadn’t.

So it was time to get another bike computer, since they are quite addictive. Being a map geek, I thought it was time I bought one with GPS, so I could map my routes on my PC. The Veloset GPS 600 caught my eye, since it was probably the cheapest GPS enabled bike computer available.

And it when it arrived first impressions were good, coming in a nice attractive box. But firing it up was disappointing. For some reason it uses a butt ugly font to display text and since it’s only a small device it’s not only ugly but difficult to read as well.

And things got worse when I fired up the accompanying software, which is possibly the ugliest software I’ve seen in my life. For people used to fondling iPads and the like, this will probably be a grave disappointment. But it does what it needs to do, importing data from the computer and displaying it in various graphs (speed, altitude) and on a map. It does consistently show my average speed as being higher than my maximum speed, which my basic grasp of maths suggests is incorrect, but other than that it seems to function correctly.

Which can also be said for the bike computer itself. It’s not the prettiest device ever but it does what it needs to do, displaying current speed, average speed (this time accurately!), distance covered, altitude, journey time etc. I’m not particularly keen on the touchpad buttons, since it’s easy to press them by accident and it’s impossible to press them at all when wearing gloves. Also on occasion the GPS seems to take a while to switch itself on, so the starts of journeys are sometimes lost.

So in conclusion, it’s the cheapest GPS enabled bike computer available and it shows in the presentation. It’s not a device you’ll fall in love with, but it does do everything it needs to do.

Finding a UserPrincipal for an email address

2012-03-06T21:16:00.001Z

I wanted to let users log in to our application using their Windows user name or their email address. In order to achieve this aim, I needed to get hold of a UserPrincipal from the user’s email address so I could then test the password they entered was correct. It took quite a lot of searching to find the relevant code, so I thought I’d post the pertinent part here.

      PrincipalContext context = new PrincipalContext(
        ContextType.Domain, Environment.UserDomainName);
      
      UserPrincipal user = new UserPrincipal(context);
      user.EmailAddress = "test@test.com";

      // create a principal searcher for running a search operation
      PrincipalSearcher pS = new PrincipalSearcher(user);

      // run the query
      PrincipalSearchResult<Principal> results = pS.FindAll();

      foreach (Principal result in results)
      {
        // do something useful...
      }

The scale of the universe

2012-02-14T21:31:00.001Z

If like me you find it hard to grasp how the tiny the tiniest things are and how massively huge the biggest things are, then have a look at this

http://htwins.net/scale2/

I’m not sure our minds can ever fully comprehend these different distances and sizes, but maybe it’ll help…

Let’s outsource the bankers

2012-01-30T22:20:00.001Z

The main argument for bankers earning vast amounts of money seems to be that in a global market, they’ll just bugger off somewhere else if we can’t match the money available elsewhere. But the odd thing is, for the rest of us, globalization has led to stagnating wages as jobs have been outsourced to countries with cheaper labour. Odd that the free market doesn’t work in the same way for the rich as it does for everyone else.

So if we assume that maybe this argument is a bit of a fib, and assuming we can find decent replacements in some far off land who are willing to work for a much smaller pay packet, can’t we just outsource all our bankers along with some of our well paid CEOs? That’s some offshoring I wouldn’t mind seeing.

Developer interview questions

2012-01-28T21:20:00.001Z

A while back I had to interview some people for a developer role at work so came up with a few questions, combining a few from the web with some of my own. This is essentially a note to myself for next time I’m interviewing.

jQuery
What's a jQuery selector? How would you select an item by its ID? By its class?
Give some examples of JQuery UI effects and widgets and what they could be used for

.NET
What's an interface? Compare and contrast with an abstract class
How does memory management differ between .NET and a non-managed language? How can we make .NET behave more like a non-managed language?
What are generics? Why use List<> instead of ArrayList?
What's a virtual function? How does it relate to OOP?

Web
What is a RESTful web service? Why are they preferred to SOAP web services?
What are the common data formats returned by an AJAX web service call? Is one better than the other? What about if you wanted to call it from a fat client?
Name and describe several HTTP status codes
Name the various HTTP verbs and when they are used

General
Your application has a performance problem, how would you investigate the issue?
An error occurs in your web application but only in production and only happens occasionally, how do you go about tracking down the problem?
What are some of the issues around multi-threaded applications?
Discuss some ways of ensuring code quality remains high in a project

Do it yourself inbound link alerts

2012-01-21T21:33:00.001Z

Embedded Analytics provide a nice service that will email you whenever somebody clicks on a new link to your site. I’ve been signed up for a while and it’s interesting to see who’s linked to my site. But I received an email last week informing me that my site had so many inbound links that I would have to start paying for the service. To be fair the amount they were going to charge me wasn’t a lot, but I couldn’t really justify spending money on something that is essentially just a way to waste a bit of time for me. And I also figured I could probably do the same thing myself through the Google Analytics API, since this is what Embedded Analytics uses.

I’m assuming that Embedded Analytics uses the source for visitors to your site to spot new links. There is a downside to this since it won’t spot links that have been added but have not been clicked on, but generally these won’t be that interesting, since they presumably are links on low traffic sites.

So to implement this requires a few steps. Pull out the data from Google Analytics and store this data somewhere (DB, XML file, whatever). Then next time we pull the data out of Google, check for new URLs in the returned data and send a notification of these new URLs. Embedded Analytics also goes a step further and validates that the links are valid and that the pages containing them are available from the web. I was only really interested in the first part of this solution so have written a piece of code to pull out the URLs using the Google Data API for .NET. The rest of the work is left as an exercise for the reader!

using System;
using Google.GData.Analytics;

namespace GoogleAnalytics
{
  class Program
  {
    static void Main(string[] args)
    {
      AccountQuery feedQuery = new AccountQuery();
      AnalyticsService service = new AnalyticsService("DoogalAnalytics");
      service.setUserCredentials("email", "password");

      DataQuery pageViewQuery = new DataQuery("https://www.google.com/analytics/feeds/data");
      pageViewQuery.Ids = "ga:202885";
      pageViewQuery.Metrics = "ga:visits";
      pageViewQuery.Dimensions = "ga:source,ga:referralPath";
      pageViewQuery.Sort = "ga:source,ga:referralPath";
      pageViewQuery.GAStartDate = DateTime.Now.AddMonths(-1).ToString("yyyy-MM-dd");
      pageViewQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");

      DataFeed feed = service.Query(pageViewQuery);
      for (int i = 0; i < feed.Entries.Count; i++)
      {
        DataEntry pvEntry = (DataEntry)feed.Entries[i];
        string host = pvEntry.Dimensions[0].Value;
        string path = pvEntry.Dimensions[1].Value;
        Console.WriteLine("http://" + host + path);
      }

      Console.ReadLine();
    }
  }
}

Another take on peer to peer lending

2012-01-06T21:44:00.001Z

It seems a little strange to me that at a time when banks are meant to be desperately trying to increase their balance sheets, they are offering such meagre interest rates. I dunnow but maybe offer some decent rates and people will shove their money in your bank? But it seems they have decided it’s better to offer crap rates and hope we are too stupid to realise we can get better returns elsewhere.

So for a few years I’ve been stashing cash in Zopa and getting a return that manages to beat inflation by lending money directly to people, with the caveat that there is a higher level of risk than having money in the bank.

But I’m obviously not the only person to choose this option and rates have been falling of late. And although I’m happy to finance personal loans, i did have a hankering to lend money to businesses, especially since this seems to be something else banks are failing to do properly*

Which is where Funding Circle comes in. This is peer to peer lending but the other party is a business. Interest rates are currently some what better than Zopa. Time will tell how bad the default rates are but I’m going to drip feed some money in there and see how it pans out.

*I always thought banking was pretty straightforward, take in money from deposits, then lend it out to other people and skim a bit of profit from the transaction. Simple and a bit boring. Maybe they should have stuck to this slightly dull job, rather than inventing, buying and selling insane derivatives that nobody understands and nobody can quantify the associated risk. Get back to doing what you’re meant to do and people might get off your back a bit…

Why all the free ads for Facebook?

2011-12-30T11:16:00.001Z

A while back I picked up a copy of the Evening Standard and was surprised to see Facebook and Twitter logos at the top of every page, suggesting readers follow ES on these two websites. And this is just an extreme example of what I’m seeing more and more. Ads on the telly and in newspapers no longer show the URL of the company’s website, but the URL of their Facebook page instead.

But the thing that confuses me is why big companies would choose Facebook as their main point of contact with their customers? Sure, social networks are the big thing at the moment and getting users to follow you or like your product might have some benefit, but there seem to be a number of downsides.

First, who owns all the data being collected about your customers? My guess is Facebook. Can you extract that data if Facebook decide they don’t want you anymore or you decide to move? I guess it’s probably possible via the Facebook API but it seems somewhat risky. And even if you do extract it, Facebook will no doubt keep hold of it as well.

Then there’s the question of ads. The company pages I’ve seen on Facebook seem to have the same ads as any other page. I’ve found no indication that companies get any of the income from those ads, so why drive traffic to Facebook so they can make money from your brand? And what if Facebook decide to show ads for one of your competitors?

Frankly it all seems a bit odd. Big companies have big IT departments and generally have their own websites, fully under their control. It is pretty simple to add some Facebook widgets to your own site and get integration that way which seems a more sane approach if you want to get hooked into Facebook.

For a one man band kind of company, I can see the sense in putting you web presence on Facebook, it’s a lot simpler and cheaper than building your own website, but for multi-nationals my prediction for 2012 is that this is something they do much less of.

I am Sheldon…

2011-12-26T19:46:00.001Z

…with a somewhat larger waste line, somewhat smaller IQ and hopefully with less OCD tendencies.

Goodbye Google Friend Connect

2011-12-02T20:49:00.001Z

I’ve been disenchanted with Google Friend Connect for a while. I only used it for commenting on my website but it has a number of weaknesses

A crappy user experience
No notifications of new comments, which is a pain if you only receive a handful of comments
Ignoring query strings in URLs, so comments don’t stick to the right page
Weird date formatting and no control over how they appear

For a long while I assumed Google would actually update the widgets but I’m not sure anything ever got changed after the initial release. It was released and then, nothing. Even a visit to the site shows a copyright notice from 2009, which suggests they haven’t done much with it for some time.

So I had been meaning to convert to some other system but just hadn’t got round to it. Then I noticed Google have decided to can it (which of course hasn’t been mentioned anywhere on the Friend Connect site itself), so I decided it was time to finally do something. Google’s suggested solution is to hook into Google+, but that seems like a pretty useless way to add commenting to a website. So my suggested solution is to sign up to Disqus, it takes about 10 minutes to plug it in to your website and looks pretty good straight out of the box.

Google Maps in a desktop app

2011-10-11T21:51:00.001+01:00

I’ve seen one or two examples of using Google Maps in a WinForms desktop app, but the ones I’ve seen seem to involve loading image tiles from the Google server directly. There’s nothing wrong with that approach but I thought it would be a lot simpler to simply host a local web page using the Google Maps API in a WebBrowser control in an application. Here is a very simple example of this idea.

If you want to extend this example, it’s possible to call scripts in the page via the WebBrowser.Document.InvokeScript method and the application can respond to events in the page via the WebBrowser.ObjectForScripting property.

As an aside, similar ideas can be applied to hosting other Javascript web components, such as an HTML editor like CKEditor.

Google Maps and Friend Connect weirdness in IE8

2011-07-14T22:08:00.001+01:00

I received a bug report about my website. The maps that appear on my UK postcodes pages weren’t working in IE8. I hadn’t noticed since they were working fine in IE9 but when I switched to compatibility view in IE9 I started to see the same problem. This helped because I was then able to debug my JavaScript (much as I love IETester, I’d love it even more if I had access to the developer tools for each version of IE). And debugging the script revealed google.maps was null.

At this point I assumed this was a problem with my Google Maps script, so tried loading Google Maps asynchronously then tried specifying an older version of the API. Neither helped. I tried inserting the script in my HTML head but with no luck.

Finally I had a look at the google object and saw the only thing defined in there was friendconnect. Now my suspicions moved away from Google Maps to Google Friend Connect. my Friend Connect stuff appears at the bottom of each page and the script reference for it was also down the bottom of the page. So I thought I’d tried moving the Friend Connect script reference to the html header. And voila, the maps started working again.

So the conclusion? I’m not really sure, although I suspect Friend Connect is removing the google.maps object, although it seems odd that it only happens on IE8.

Retrieving the most popular pages using Google Analytics API

2011-06-04T22:43:00.001+01:00

For a long time I’ve shown the most popular pages on the home page of my website. I did this by logging every page that was viewed to the MySql database on the back end. This kind of worked but had a few problems. First, it wasn’t very clever since it couldn’t tell the difference between a real visitor and a search engine bot. Second, since I’ve started to get quite a few visitors (no, really), it was writing a large amount of data to the database.

So I thought there must be a better solution. Figuring that all the information I needed was already being collected by Google Analytics, I thought I could grab this data and dump it into a much smaller with just the page URL and the number of visits (rather than adding a row for every visit). So I coded up a solution using the .NET wrapper around the Google Analytics API. And this is what it looks like (with the database access code removed for clarity). You’ll need to provide your own email address, password and Google Analytics account table ID to get this to work, for obvious reasons.

using System;
using Google.GData.Analytics;

namespace GoogleAnalytics
{
  class Program
  {
    static void Main(string[] args)
    {
      AccountQuery feedQuery = new AccountQuery();
      AnalyticsService service = new AnalyticsService("DoogalAnalytics");
      service.setUserCredentials("email address", "password");

      DataQuery pageViewQuery = new DataQuery("https://www.google.com/analytics/feeds/data");
      pageViewQuery.Ids = "Google Analytics account table ID";
      pageViewQuery.Metrics = "ga:visits";
      pageViewQuery.Dimensions = "ga:pagePath";
      pageViewQuery.Sort = "-ga:visits";
      pageViewQuery.GAStartDate = DateTime.Now.AddMonths(-1).ToString("yyyy-MM-dd");
      pageViewQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");

      DataFeed feed = service.Query(pageViewQuery);
      for (int i = 0; i < 20; i++)
      {
        DataEntry pvEntry = (DataEntry)feed.Entries[i];
        string page = pvEntry.Dimensions[0].Value.Substring(1);
        string visits = pvEntry.Metrics[0].Value;

        Console.WriteLine(page + ": " + visits);
      }

      Console.ReadLine();
    }
  }
}

Poor man’s XSLT profiling for .NET

2011-05-30T21:08:00.001+01:00

If you’ve ever looked round for a profiler for XSL transformations then chances are you’ve found the Microsoft add-on for Visual Studio, which looks like it’s just the ticket, if you happen to have Visual Studio Team System. But if you don’t happen to own that version, then it might look like you have to upgrade your VS license or buy some other XSLT profiler.

But if you happen to own a .NET profiler (I highly recommend AQTime) then there may be another solution. Visual Studio comes with the XSLTC tool that can be used to generate an assembly from an XSL transformation. Once we’ve got an assembly, then we can build a small wrapper application that loads up the assembly, passes it to an instance of the XslCompiledTransform class and calls the transform. And once we’ve got that, we can use a standard .NET profiler to find bottlenecks.

And as I understand it, the XSLT profiler add-on for Visual Studio works in just this way so profiling using this technique should be just as effective as the Microsoft version.

Spotify not too good to be true anymore

2011-04-14T22:05:00.001+01:00

Apparently I’ve been using Spotfiy for over two years. Funny, it seems longer than that. It was the perfect music service for me, unlimited music of my choosing on my PC, which is where I listen to music most of the time, with the only minor downside being some adverts that play occasionally between tracks. But it looks like it won’t be quite so perfect anymore. Free users can only listen to a track a total of five times and total listening time will be limited to 10 hours a month.

As a frequent user of Spotify I can see why they are doing this. First, it’s obvious that advertising revenue is not what they were hoping for, most of the ads are still for Spotify itself. Second, using it has had a perhaps not unexpected effect on my music buying behaviour. First example, U2 put their last album up on Spotify before its official release. I had a listen and realised it was rubbish, so as a marketing exercise I doubt it was a huge success. Second example/s, quite a few new releases are put onto Spotify Premium upon release. On a couple of occasions I’ve then purchased the album before it’s become available on the free version (Elbow, Arcade Fire if you’re wondering). I’m guessing this isn’t what Spotify wanted me to do, they were presumably hoping I’d pony up the Premium version. And then when those albums did become available on the free version (generally only a few weeks after release) I was hit with a mild feeling of regret for spending money that I didn’t really need to and deciding to think twice before making another purchase. Again, probably not what the music industry sponsors of Spotify were hoping for.

So now I’ve got a choice, sign up for a tenner a month and continue on as I am at the moment or spend that tenner on a CD every month. I guess the music industry don’t care too much which way that ten quid gets to them, so it’s purely a personal dilemma. But the people who can’t or won’t spend a tenner (teenagers, students mostly I guess) will probably rediscover the skill of searching for pirated albums on Google. The music industry is still caught between a rock and a hard place.

Updating the Code-Point postcode datataset in MySql

2011-03-12T10:49:00.001Z

Some time ago I imported the Ordnance Survey Code-Point postcode dataset into MySql. It looks like there’s a new version of that dataset available which includes new postcodes so I wanted to update my database. I guess I could just empty the table and re-import the data, but since it takes some time import and the data is live on the web, this wasn’t the ideal solution. Fortunately, MySql has a useful IGNORE keyword which will ignore failed inserts so any old postcodes will be ignored (since the postcode is used as the primary key on the table) whilst new ones are inserted. Of course, this assumes that the latitude and longitude of old postcodes doesn’t change, which I’m hoping is a reasonable assumption. So my new code looks like this.

using System;
using System.IO;
using DotNetCoords;
using LumenWorks.Framework.IO.Csv;
using MySql.Data.MySqlClient;

namespace ImportCodepoint
{
  class Program
  {
    static void Main(string[] args)
    {
      string[] files = Directory.GetFiles(@"C:\Users\Doogal\Downloads\codepo_gb\Code-Point Open\Data");
      foreach (string file in files)
      {
        ReadFile(file);
      }

    }

    private static void ReadFile(string file)
    {
      using (StreamReader reader = new StreamReader(file))
      {
        CsvReader csvReader = new CsvReader(reader, false);
        using (MySqlConnection conn = new MySqlConnection(
          "server=server;uid=username;pwd=password;database=database;"))
        {
          conn.Open();
          foreach (string[] data in csvReader)
          {
            string postcode = data[0];
            // some postcodes have spaces, some don't
            if (postcode.IndexOf(' ') < 0)
              postcode = data[0].Substring(0, data[0].Length - 3) + " " + data[0].Substring(data[0].Length - 3);
            // some have two spaces...
            postcode = postcode.Replace("  ", " ");
            
            double easting = double.Parse(data[10]);
            double northing = double.Parse(data[11]);

            // there are some postcodes with no location
            if ((easting != 0) && (northing != 0))
            {
              // convert easting/northing to lat/long
              OSRef osRef = new OSRef(easting, northing);
              LatLng latLng = osRef.ToLatLng();
              latLng.ToWGS84();

              using (MySqlCommand command = conn.CreateCommand())
              {
                Console.WriteLine(postcode);
                command.CommandTimeout = 60;
                command.CommandText = string.Format(
                  "INSERT IGNORE INTO Postcodes (Postcode, Latitude, Longitude) " +
                  "VALUES ('{0}', {1}, {2})",
                  postcode, latLng.Latitude, latLng.Longitude);
                int count = command.ExecuteNonQuery();
                if (count > 0)
                  Console.WriteLine("Added");
              }
            }
          }
        }
      }
    }
  }
}

GZipping all content served up by ASP.NET

2011-02-22T08:13:00.001Z

Update – I now realise this post is kind of pointless, there is a module for compression of dynamic content, called unsurprisingly DynamicCompressionModule… But the approach described may be useful for someone somewhere…

I couldn’t find anything that will GZip all the content returned by ASP.NET. There’s a module for compression of static files but nothing for dynamic content. There may be a good reason for this, perhaps the overhead of GZipping content on the fly can kill your server, but since my current project has no static content I thought it would be useful to give it a go. The solution is pretty simple, register the following module in web.config and you’re good to go.

using System;
using System.IO.Compression;
using System.Web;

namespace MyNamespace
{
  public class GzipModule : IHttpModule
  {
    public void Dispose()
    {
      
    }

    public void Init(HttpApplication context)
    {
      context.BeginRequest += new EventHandler(context_BeginRequest);
    }

    void context_BeginRequest(object sender, EventArgs e)
    {
      HttpApplication app = (HttpApplication)sender;
      if ((app.Request.Headers["Accept-Encoding"] != null) &&
            (app.Request.Headers["Accept-Encoding"].Contains("gzip")))
      {
        app.Response.Filter = new GZipStream(app.Response.Filter, CompressionMode.Compress);
        app.Response.AppendHeader("Content-encoding", "gzip");
        app.Response.Cache.VaryByHeaders["Accept-encoding"] = true;
      }
    }
  }
}

Registration is as follows

    <modules>
            <add name="GzipModule" type="MyNamespace.GzipModule" />
    </modules>