Development Diaries

PRTG as a Pachube/Cosm/Xively replacement?

2015-12-06T14:50:00.001+00:00

Developer/Maker friendly long term data storage can be hard to come across but I think I have found a pretty neat solution.

PRTG is a network monitoring tool produced by Paessler. It also allows ticket management and all sorts of very clever stuff that's totally over the top for a home user, but it does give you access to a really cool dashboard creating tool (see above), a load of built in sensors, the ability to add custom sensor it's totally free for up to 100 sensors. Before I jump ahead lets give your a brief overview of the terminology used in PRTG.

What is a channel

A channel is basically a single data point. Each channel can have a value (a decimal normally), but it can also be a lookup with text as shown below.

For each channel you can specify various options, but the most useful are Units (specifies the units for the thing you are measuring)

You can also specify limits that cause the channel to switch state automatically (in this example a value above 90 will cause the sensor to be in error state).

Finally you can do get some pretty nice graphs for each channel (this shows my energy usage for the last 2 days)

What is a sensor

A sensor is basically a collection of channels (up to 50 is recommended). It has an overall status of Ok, Warning or Error depending on the child channels. PRTG does a really nice job of summarizing the status of all your sensors.

The sensor screen acts a bit like a dashabord for all the channels the sensor contains.

What is a device

A device can have a collection of sensors that normally relate to the devices IP (but they don't have to). The device is clever in that it will automatically pause any sensors on it if some of the lighter weight sensors report an error condition (if for example it stops replying to ping)

The device can also show a nice summary of the status of it's sensors

Device Groups

You can also group devices together like "Windows Machines" etc. Again the status of all the sensors below get grouped up.

Notifications

You can out of the box configure PRTG to e-mail you (or push a notification to you) when a channel goes into a specific state.

Maps

This is perhaps my favourite feature! This lets you create really cool dashboards showing the status, and values of all the items above. Here for example is the map of my network

Mobile App

Paessler have created amazingly complete applications for iOS, Android and even Windows Phone. The apps are very fully featured and make use of the native platforms (for example you can add sensors as widgets on android). As a mobile developer I'm really happy to see this!

Ok, what sort of things can you monitor

Out of the box there are over 200 sensors. I have mainly used the following:-

Ping Pretty obvious, but lets you know if something has gone down quickly.
HTTP Gives you average load times for a web page (used to monitor your internet connection)
Windows Sensors You can monitor a lot on a windows machine (CPU & memory usage, disk free space, uptime, pagefile usage, network traffic, RDP availability, is a service running), I use most of these!
Network Flow If your router/switches support this (dd-wrt does!) then you can get a lot of information about the traffic flowing through your network. Really handy for IoT devices when you have no idea what they are doing!

What has any of this go to do with makers

Nothing, because I haven't mentioned the best sensor types yet! PRTG also supports several custom sensors allowing you to monitor just about anything you want:

EXE/Script You can write a console application that can output xml
Python Script Same as above, but in python
HTML XML/REST value This can poll a web api and do things like count the number of times an element appears. I use this to monitor how many people are streaming from my plex server
HTTP Push Data Here you can push data to the server whenever an event happens that you would like to track (HOW COOL IS THAT!)

Obviously the above scripts could be used to interface with things like an Arduino really easily! The Paessler blog has listed some cool project makes have done using PRTG - here is a project that counts how much mail you have received during the year, one that monitors fuel prices in Germany (and uses a nice dashboard to present the data). A hospital in South Australia is even using it to monitor blood bank levels and send alerts when they run low! Paessler have a great list showing some of the interesting things that are monitored using PRTG here.

It's also really easy to configure PRTG to have custom notifications too. There are notifications for Twitter, The terrorist favourite Telegram and a load of others, but it's really easy to add your own!

So you basically have your own hosted version of Pachube/Cosm/Xively or whatever it's called now, plus it has ifttt type functionality (in-fact using the ifttt maker channel you could easily make the two work together nicely!)

In the future I'm going to blog a bit about some custom sensors I've made, and a sensor host I'm working on to make it really easy to use the HTML XML/Rest sensor!

Ok, how much is it then

Ok, it's not cheap if you are going to be monitoring a lot of things, but it's totally free for the first 100 sensors! And each sensor can have up to 50 channels, so you could monitor up to 5,000 channels for free.
It used to be a lot less (10 sensors), but Paessler recently changed this in the hope that hobbyists/home network enthusiasts/geeks will love using this at home, and then tell the sys admins at work that they need to be using this in the office. I can easily see how that would work, because this is first and foremost an excellent network monitoring tool, it just so happens to also be an excellent *insert your project here* monitoring tool too :-)

Hacking a PacMan Lamp

2013-04-10T21:12:00.000+01:00

I recently got given a PacMan Lamp (which can be bought from Firebox) from microsoft because I published a windows store app (called Space Race for those who are interested - its a phonics game for young children learning English!)

The lamp comes with a little remote that lets you select any colour. I just had to figure out how to control this thing with an arduino!

I took the remote to bits, but quickly realized I was going to have to solder a wire per button at least, and this would hardly scale - so I decided the next best option would be to simulate the IR the remote produces.

Lady Ada has a brilliant guide I used to read the IR signals via the arduino. To do this you will need a IR Detector, which I found to hand

Well, I say to hand... No kids toys were (permanently) broken you will be happy to hear

After hooking it all up to the Arduino I was able to pull out the timings of the "On" periods vs the "Off" periods. As advised you ignore the first off, and then look at the time periods for on vs off. It became pretty apparent that most times were around 530 µs, or 1620 µs, and the on periods were always the short time.

Once I had profiled several of the buttons I turned the "Off" periods into binary with the short period representing 0s, and the long periods representing 1s (as shown by column J in the screenshot above.

With enough codes it became fairly obvious that the first 16 bits where always "0000000011110", and the final 3 bits were always "111". After scratching my head for a bit I realised that the first 5 bits, and the last 5 bits where the only things that changed (the middle bits where always "000", and once I had pulled the bits that changed out it was really obvious that they were just the inverse of each other

Now all I had to do was write an arduino script that would take 5 bits, and translate it into the code, and then correctly send it to the IR pins. Heavily borrowing from the sample code Lady Ada provided I was pretty easily able to implement the algorithm. Feel free to grab the code from my GitHub repository. Once you have uploaded your code you will need to add an IR Diode to your arduino as shown in this image (ignore the stuff to the right of the image - thats for hooking up the IR Detector)

(Image from Lady Ada's site)

Now with everything loaded I needed to create a little app that would fire some commands at it. I created the following little library class that wraps everything up.

To use it simply construct the object (passing in the com port the arduino is connected to, then call do command passing in a command you want) as shown below

I'd love to hear what you use this for - I have just hooked mine up to Lync (its like skype for business). It now flashes white and blue when I get a call and when I'm not getting called it's coloured the same as my lync presence. I'll get a video up asap.

A BBSB Online controller clone (for £26)

2011-12-22T20:44:00.000+00:00

I love home automation (as I'm sure any geek does!), so when I saw the BBSB online controller I knew I needed it in my life but could really justify the £90, plus I figured why not try and build one myself. I mean, how hard can it be?

First thing I needed was some programmable hardware and since I have done so many arduino based projects before, the choice was obvious. Or was it? I noticed a new arduino clone had popped onto the market called a Nanode. Acting much like an arduino but at an incredible £22 from SK Pang (a similar arduino is over £44!). Now I actually ended up getting the upgraded model (Nanode RF) which includes the following extra features:-

A Hope RF RFM12B (supplied with 868MHz module) transceiver for 2 way communications with other boards.
A microSD card for general datalogging storage, storing applications and webpages. microSD card not supplied.
A realtime clock IC with alarm function which also holds a unique ID. not supplied.
An 8 pin socket (under the H logo) to allow you to add non volatile RAM for program download. Supplied with 23K256 256kb SRAM and DIL socket.

This allows me to use it to talk to other devices, but more on that in a later blog post!

To simulate the bye bye standbye online controller I needed to know 2 things. 1 how does it talk to the sockets, and 2 how does it receive commands from app such as the brilliant remote for Bye Bye Standbye.

Well thankfully both problems have been solved, and well documented many times before!

Arduino actually have a playground for all of the HomeEasy integration code - so that was the easy bit. The only additional thing I needed was something that could "talk" over the 433 frequency. Thankfully Farnell sell one for a smidge of £4.

Once I proved I could get this working nicely (using code sample 3 from the playground) I moved on to problem 2 - how does the BBSB controller talk to other software?

Again this was (unusually) well published by bye bye standanby (hats off to you for that!). So all I had to do was get the nanode to "talk" UDP.

Well this was a tad harder! The Nanode does not use the same Ethernet chip as the Ardiuno Ethernet stuff - so you can't use the standard "Ethernet" Library. Arduino recently released a new version of their IDE (V1) this means a lot of the code samples online have changed so it's a tiny bit harder to get good samples, but with some great help from the nanode community (get involved on irc) I was able to pull apart the tftp sample and just handle the udp packets I needed.

So without further ado here is my code hosted on github (my first open source code!) https://github.com/rossdargan/Arduino-Projects/blob/master/BBSBControllerImpersonator/BBSBControllerImpersonator.ino

And here is the final solution:-

Testing

BBSB also provide a really handy application to test to see if everything works fine - you can grab that from the forums

Future changes

Well the Nanode RF already includes a transmitter so it's a bit daft not to be using it - unfortunately it's the wrong frequency - but you can easily by a compatible 433 transiever from farnell.

Announcing WHS Drobo Status!

2011-11-01T19:54:00.008+00:00

I have always loved Microsoft's Windows Home Server and sing its praises whenever possible; however with the most recent version Microsoft decided to remove what I always thought was one of the best features - the drive extender!

Drobo as a product is a perfect fit for this gap. The product allows you to add multiple hard drives and collates the space, making it easy for a home user to expand their storage space on demand.

I always wanted a Drobo so when I saw a competition running on the Home Server Show's website to win one I knew I had to enter!

The competition required the development of an add-in for Windows Home Server (something I have always wanted to do but couldn't quite think of anything to build!) So, having put sleep aside for a few days I'm very pleased to announce the public beta of my WHS Drobo plugin!

The plugin allows you to at-a-glance view how much space you have available, and the status of the drives as shown below:-

The screenshot below shows the drive information if for example you remove a disk:-

You can find the beta here: WHSDroboStatus.wssx. Simply install the add-in as normal from any WHS client computer.

Please note you must have the Drobo dashboard software installed on your Windows Home Server for the plugin to work (you probably have it installed already! If not you can find the right version from the drobo website).

Any request for new features/feedback can be added here: http://whsdrobo.uservoice.com

Please also leave a comment to let me know how you get on with it!

B&Qs Cubo Colour changing LED cube

2010-09-03T22:52:00.001+01:00

I was in b&q and I spotted one of these:

It was reduced down to £1.50 – but I can only find them in store not on the web, so no linky I’m afraid!

The cube light is pretty small at around 5cm each side, but produces a fairly nice light.

To remove the bottom part requires a bit of force as its glued in place but once you do pry it apart you are presented with a single LED with two legs connected to a switch, then a 22 ohm resister +/-5% followed by 3 * 1.5v AG13 batteries. It looks like this:

At this point I realised something clever must be happening– all of them colours via only 2 pins! If you look very closely at LED you will notice on the top of the anode leg inside the led a tiny black dot. That dot is what controls the colour, so hacking this is going to be extremely limited if you intend on using this LED.

Whilst it’s currently driven from the 4.5v provided by the batteries I did successfully drive the unit from the 3.3v from a neduino board , so you could very easily use it as some form of output. Powering it externally also means it will last a lot longer than the 6-8 hours the leaflet expects it to last from the batteries – and given the batteries cost £1.89 from Amazon (more than the device cost!) it’s a good job!

I think that the best bet if you were hacking this device would be to replace the current in place IC controlled LED with a RGB LED. The plastic Plug holding all the gubbings also comes with a hole which could easily fit any cables you needed to bring out of the device!

Happy Hacking!

Rainbowduino + Microsoft Office Communicator

2010-08-20T10:51:00.001+01:00

The Project

Create a Cube Grenade with a rainbowduino I had lying about. Cube grenades can be described as:

This, I believe, is where my cartoons work the best- “Cube Grenades”- small objects that you “throw” in there in order to cause some damage- to start a conversation, to spread an idea etc

http://gapingvoid.com/2009/05/13/cube-grenades/

Since I work at a small company (around 60 employees) I could use the 8*8 grid to display peoples communicator status.

The Parts

A rainbowduino is an LED driver platform which can control an RGB matrix. It looks like this:

Since it’s basically an arduino you can replace the default firmware with a custom one and make it do what you want.

The next item is the RGB matrix itself:

This slots into the rainbowduino perfectly

Finally I needed some way to flash/communicate with the rainbowduino. Effectively you need a USB to serial convertor.

I decided upon a seeedstudio UartSB to make it as easy as possible (this also allows you to create a wireless connection via Xbee should you want to!)

The pins also line up perfectly with the rainbowduino so it’s ideal for this project.

I also bought a harness because it came with a 2.1mm DC jack to pin plug adapter (which is what the rainbowduino needs for to power it), and I paid extra for an arduino power supply when I bough my arduino.

I got most of my parts from www.skpang.co.uk and at the time (March 09) they cost:

Rainbowduino LED driver platform= £15.99
60mm Square 8x8 RGB LED Matrix= £19.67
UartSB V2.1= £20.13 (No longer available)
Harness for arudino = £5.76
Arduino PSU = £9.99

Total: £71.54

Ok, so it’s an expensive toy, but I have used them for other projects (including writing minesweeper for the rainbowduino!)

Please note that you can use an arduino to flash and communicate with the rainbowduino instead of the UartSB as shown in this image:

http://www.rngtng.com/2009/06/25/rainbowduino-here-it-is-and-how-to-program-it/

The Idea

I had a play about with getting OCS statuses and decided that the easiest way would be to use COM to connected to a local Communicator client, get all the statuses and send them via the usb serial port to the rainbowduino.

I could have saved it to a database and used an ethernet shield to get the data, but I decided since a machine would have to be on to get the statuses anyway I might as well just use the easiest method.

The Rainbowduino Program

The first thing I had to get working was the rainbowduino. The firmware it comes with is pretty good, but you have to use I2C to send it instructions and the firmware doesn’t allow you to specify an individual pixels colour.

The firmware was a really good starting place for me, but I hade to modify it quite a bit. By default the rainbowduino starts with a pretty screen, but I wanted it to start with every light turned off. This can be achieved by modifying the multi dimensional array in the data.c file. The array is defined as follows:

unsigned char dots_color[5][3][8][4] = {…}

This array allows for a buffer (up to 5 different images can be buffered) the three colours, and the individual data for each row/column.

so to set the first two pixels to green you would set the following:

dots_color[Buffprt][0][0][0] = 0xff

A pointer to the screen currently being displayed is stored in the variable Buffprt. the next 0 refers to green, (1=red, 2=blue). The next 0 refers to the first row, and the final 0 refers to the first two LEDS on the first row.

The column gets complicated because it uses a byte (8 bits) to define two LEDs. This is a really clever way of saving memory, but it does get complicated!

To turn the first LED you would use the bits 00001111 (0Xf) and for the second LED you would use 11110000 (0xf0).

Once I understood this I could write a setpixel function with the following signature:

void SetPixel(byte red, byte green, byte blue, unsigned char rowToSet, unsigned char columnToSet)

I liked the idea of the buffer, so the first thing I do is copy the current buffer into the next frame – this means the rainbowduino will never forget current statues, and will only modify the led we want to change. It’s important to note that this really isn’t required and you could just modify the current frame.

  unsigned char color=0,row=0,dots=0;
  
   for(color=0;color<3;color++)
  {
    for (row=0;row<8;row++)
    {
      for (dots=0;dots<4;dots++)
      {                 
         dots_color[((Buffprt+1)&1)][color][row][dots] = dots_color[((Buffprt))][color][row][dots];  
      }
    }
  }

The next thing I had to do was find out which column we are actually setting – for example if the row we are trying to set is the last LED on a row we would pass in 7 (LEDs are 0 indexed). This would translate to column 3, and the last 4 bits on this byte.

Lets say the 6’th and 7th LED were currently 1/2 on, and we wanted to turn the 7th LED fully on. this would mean that the byte was currently:

00110011

What we need to do is create a mask that will blank out the first four bits, and then or it with the value we want to set.

So in our example the mask would be:

00001111

We would and the mask with the current value leaving:

00110011

00001111

00000011

We then need to add in the new value of the LED. We do this by first bit shifting our input (0xf) and then or’ing it with the above value.

e.g.

1111 << 4 (0xf bit shited 4 places)

11110000

We then or this with the original byte value after it has been and’ed with the mask.

00000011

11110000

11110011

This is exactly the result we want. This is all achieved with the following code:

 boolean firstBit = columnToSet % 2==0;
  columnToSet = columnToSet/2;
   
  Value = red;
  Mask = 0xf0;
  if(firstBit)
  {
    Value=red<<4; 
    Mask=0x0f;
  }
  
  color=1;
  Value = (dots_color[Buffprt][color][rowToSet][columnToSet] & Mask) | Value;    
  dots_color[((Buffprt+1)&1)][color][rowToSet][columnToSet] = Value;

This is repeated for the other two colours.

Finally all we have to do is tell the current buffer to move on by one.

Now that we have written the setpixels method we need to write some way to communicate with our rainbowduino via the serial port.

This is done via the following code:

void loop()
{
    unsigned char color=0,row=0,dots=0;
         
    if (Serial.available() > 0) {
      // read the incoming byte:
      byte  red = Serial.read();
      delay(2);
      byte blue = Serial.read();
      delay(2);
      byte green = Serial.read();
      delay(2);
      int row = Serial.read();
      delay(2);
      int column = Serial.read();
       
      // say what you got:
      Serial.print("I received: ");
      Serial.print(red);
      Serial.print(",");
      Serial.print(blue);
      Serial.print(",");
      Serial.print(green); 
      Serial.print(",");
      Serial.print(row); 
      Serial.print(",");
      Serial.println(column);
      SetPixel(red, blue, green,  row, column);
      Serial.println("sent");
    }  
}

There is a delay between reading the value in the Serial port to ensure the next value is in the buffer. But the code is pretty simple. It listens for 5 bytes of data then pushes the result into the SetPixel method we created before. As long as the data doesn’t get send to quickly it works fine, but this is pretty nasty code and I’m sure there are better ways of doing this.

The Office Communicator Software

The integration with OCS is quite easy. By downloading the SDK then adding a reference to CommunicatorAPI you can get all the statuses you need.The code below attaches allt eh events we will require, logs in, then does the initial load of the contacts

CommunicatorAPI.MessengerClass _messenger;
public OCSInterface()
{
    _messenger= new CommunicatorAPI.MessengerClass();
    _messenger.OnContactStatusChange += new DMessengerEvents_OnContactStatusChangeEventHandler(_messenger_OnContactStatusChange);
    _messenger.OnIMWindowContactAdded += new DMessengerEvents_OnIMWindowContactAddedEventHandler(_messenger_OnIMWindowContactAdded);
    _messenger.OnIMWindowContactRemoved += new DMessengerEvents_OnIMWindowContactRemovedEventHandler(_messenger_OnIMWindowContactRemoved);
    if (_messenger.MyStatus != MISTATUS.MISTATUS_ONLINE)
    {
        _messenger.AutoSignin();
    }
    LoadContacts();
}

Because we don’t really want to be dealing with iMessengerContacts in other bits of code I translate the objects into a Plain Old CLR Object defined below:

   public class Contact
    {
        public string FriendlyName { get; set; }
        public string SipAddress { get; set; }
        public Status Status { get; set; }
        public int Row { get; set; }
        public int Column { get; set; }
    }
    public enum Status
    {
        Offline =0,
        Online=1,
        Busy=2,
        Away=3,
        Idle,
        Phone
    }

This shows the different status I’m going to be using, and also shows the pixel the contact relates to.

The load contact method does this translation and is shown below:

private Dictionary<string, Contact> _contacts = new Dictionary<string, Contact>();
private void LoadContacts()
{        
    foreach (CommunicatorAPI.IMessengerContact messengerContact in (_messenger.MyContacts as CommunicatorAPI.IMessengerContacts))
    {
        Contact contact = new Contact();
        contact.FriendlyName = messengerContact.FriendlyName;
        contact.Status = Status.Busy;
        switch (messengerContact.Status)
        {
            case MISTATUS.MISTATUS_OFFLINE:
                contact.Status = Status.Offline;
                break;
            case MISTATUS.MISTATUS_ONLINE:
                contact.Status = Status.Online;
                break;
                            
            case MISTATUS.MISTATUS_AWAY:
                contact.Status = Status.Away;
                break;
            case MISTATUS.MISTATUS_IDLE:
                contact.Status = Status.Idle;
                break;
            default:
                contact.Status = Status.Busy;
                break;
        }
        contact.SipAddress = messengerContact.SigninName;
        _contacts.Add(contact.SipAddress, contact);
    }
}

As you can see the pixel location is not set here, this is done when we actually interface with the rainbowduino.

The most import event we attached to on the MessengerClass was the contactstatuschange event. This lets us know when we need to update the rainbowduino. The code simply identifies the Contact POCO object, and updates the status.

In a great example of shocking code I have duplicated the mapping between my POCO status and the messenger status – this should have been a private method – but this isn’t production code and I can’t be bothered to change it! At the end of the call we notify an event that the status has changed, and we should notify the rainbowduino.

public delegate void ContactStatusChanged(Contact changedContact);
public event ContactStatusChanged onStatusChanged;
private void _messenger_OnContactStatusChange(object pMContact, MISTATUS mStatus)
{
    Contact contact=  _contacts[((CommunicatorAPI.IMessengerContact)pMContact).SigninName];
    switch (mStatus)
        {
            case MISTATUS.MISTATUS_OFFLINE:
                contact.Status = Status.Offline;
                break;
            case MISTATUS.MISTATUS_ONLINE:
                contact.Status = Status.Online;
                break;
                            
            case MISTATUS.MISTATUS_AWAY:
                contact.Status = Status.Away;
                break;
        case MISTATUS.MISTATUS_IDLE:
        case MISTATUS.MISTATUS_OUT_OF_OFFICE:
                contact.Status = Status.Idle;
                break;
        case MISTATUS.MISTATUS_ON_THE_PHONE:
                contact.Status= Status.Phone;
            break;
            default:
                contact.Status = Status.Busy;
                break;
        }
    if (onStatusChanged != null)
    {
        onStatusChanged(contact);
    }
}

The other two events we attach to respond to the event “IMWindowContactAdded” – this basically means starting and IM chat with someone. I added this as I wanted a way to tell who related to which pixel and this was the easiest way of adding that.

void _messenger_OnIMWindowContactRemoved(object pContact, object pIMWindow)
{
    Contact contact = _contacts[((CommunicatorAPI.IMessengerContact)pContact).SigninName];

    _messenger_OnContactStatusChange(pContact, ((CommunicatorAPI.IMessengerContact)pContact).Status);
}

void _messenger_OnIMWindowContactAdded(object pContact, object pIMWindow)
{            
    _messenger_OnContactStatusChange(pContact,MISTATUS.MISTATUS_ON_THE_PHONE);

}

The class I use to interface with the rainbowduino simply opens a serial connection on a port, and sends the required colour and pixel location to the rainbowduino. Notice I have abstracted the colours from rgb to system.drawing.color – this just makes it a bit easier to call the code, however I haven’t implemented every colour, so if a color gets send which isn’t implement it gets translated into Black.

public class RainbowInterface :IDisposable
{
    string _portName = "";
    SerialPort port;// = new SerialPort(_portName);

    public RainbowInterface(string portName)
    {
            port = new SerialPort(portName);

        _portName = portName;
        port.BaudRate = 9600;
        port.DataReceived += new SerialDataReceivedEventHandler(port_DataReceived);

        port.Open();
    }
    public void SetPixel(System.Drawing.Color color, int row, int column)
    {
        SetPixel(color, row, column, true);
    }

    public void SetPixel(System.Drawing.Color color, int row, int column, bool showHighlight)
    {
            
        byte bRed = 0x0; 
        byte bBlue = 0x0;
        byte bGreen =0x0;
        if (color ==System.Drawing.Color.Blue)
        {
            bBlue = 0xf;
        }
        if (color == System.Drawing.Color.Red)
        {
            bRed = 0xf;
        }
        if (color == System.Drawing.Color.Green)
        {
            bGreen = 0xf;
        }
        if (color == System.Drawing.Color.Yellow)
        {
            bGreen = 0xf;
            bRed = 0xe;

        }
        if (color == System.Drawing.Color.Purple)
        {
            bRed = 0xf;
            bBlue = 0xf;
        }
        if (color == System.Drawing.Color.Aqua)
        {
            bGreen = 0x1;
            bBlue = 0x1;
        }
        if (color == System.Drawing.Color.LightBlue)
        {
            bBlue = 0x8;
        }
        if (color == System.Drawing.Color.Salmon)
        {
            bRed = 0x1;
        }
        if (color == System.Drawing.Color.LightGreen)
        {
            bGreen = 0x8;
        }
        if (color == System.Drawing.Color.Orange)
        {
            bGreen = 0x2;
            bRed = 0xe;

        }
        if (color == System.Drawing.Color.White)
        {
            bGreen = 0xf;
            bRed = 0xf;
            bBlue = 0xf;
             

        }
        if (showHighlight)
        {
            byte blueFade = 0xF;
            while (blueFade > 0 && blueFade <= 0xF)
            {
                byte[] bytesBlue = { 0x0, 0x0, blueFade, System.BitConverter.GetBytes(row)[0], System.BitConverter.GetBytes(column)[0] };

                port.Write(bytesBlue, 0, bytesBlue.Length);

                System.Threading.Thread.Sleep(40);
                blueFade -= 2;
            }
        }
        byte[] bytes = { bRed, bGreen, bBlue, System.BitConverter.GetBytes(row)[0], System.BitConverter.GetBytes(column)[0] };
        port.Write(bytes, 0, bytes.Length);
        System.Threading.Thread.Sleep(50);
        // port.Write(sRed+sGreen+sBlue + row + column);
    }

    void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
    {
        Console.WriteLine(port.ReadLine());
    }

    #region IDisposable Members

    public void Dispose()
    {
        port.Close();

    }

    #endregion
}

Notice I have also added a highlight animation. This just helps show which pixel has been updated.

The last bit of code is what ties everything together.

class Program
{
    private static PresencePoller.RainbowInterface _rainbowInterface;
    private static PresencePoller.OCSInterface _cinterface;// = new PresencePoller.OCSInterface();        
    static void Main(string[] args)
    {
        _rainbowInterface =  new PresencePoller.RainbowInterface(args[0]);            
        _cinterface= new PresencePoller.OCSInterface();

            _cinterface.onStatusChanged += new PresencePoller.OCSInterface.ContactStatusChanged(cinterface_onStatusChanged);
            IEnumerable<PresencePoller.Contact> contacts = _cinterface.GetContacts();

            var contactsOrdered = contacts.OrderBy(p => p.FriendlyName);

            int row = 0;
            int column = 0;
            foreach (var contact in contactsOrdered)
            {
                contact.Row = row;
                contact.Column = column;

                SetStatus(contact);

                column++;
                if (column == 8)
                {
                    column = 0;
                    row++;
                }
                Console.WriteLine((int)contact.Status);
            }
            while (row < 8)
            {
                _rainbowInterface.SetPixel(System.Drawing.Color.Aqua, row, column);
                column++;
                if (column == 8)
                {
                    column = 0;
                    row++;
                }
            }                
            Console.ReadLine();
            
    }

    static void cinterface_onStatusChanged(PresencePoller.Contact changedContact)
    {
        SetStatus(changedContact);
    }
    private static  void SetStatus(PresencePoller.Contact contact)
    {
        switch (contact.Status)
        {
            case PresencePoller.Status.Away:
                _rainbowInterface.SetPixel(System.Drawing.Color.Orange, contact.Row, contact.Column);
                break;
            case PresencePoller.Status.Busy:
                _rainbowInterface.SetPixel(System.Drawing.Color.Red, contact.Row, contact.Column);
                break;
            case PresencePoller.Status.Offline:
                _rainbowInterface.SetPixel(System.Drawing.Color.Salmon, contact.Row, contact.Column);
                break;
            case PresencePoller.Status.Idle:
                
                _rainbowInterface.SetPixel(System.Drawing.Color.Yellow, contact.Row, contact.Column);
                break;
            case PresencePoller.Status.Phone:
                _rainbowInterface.SetPixel(System.Drawing.Color.White, contact.Row, contact.Column);
                break;
            case PresencePoller.Status.Online:
                _rainbowInterface.SetPixel(System.Drawing.Color.Green, contact.Row, contact.Column);
                break;
        }
    }
}

All this code does is on load loop through every POCO contact, assign a pixel to the user, then notify the rainbowduino of its status. Then anytime the status changes we update the rainbowduino again.

The final Result

Please note the paper is used to make the LED colours a bit easier to see

Future Enhancements

Instead of passing the red, green, and blue separately I could have combined two of the colours by a similar bit shifting method as used in the buffer – I didn’t for readability, however it would have been 20% quicker to transfer data if I did make this improvement.

Conclusion

My aim was to build a cube grenade, not something useful, and I think I have succeeded in doing this. I have lost count of the number of people that have seen this and said “Why would you do that, its pointless – but you could do this with it…”

Job done

Multiple monitors & windows 7

2009-10-13T16:43:00.003+01:00

Windows 7 has a great new feature called "aero snap" that lets you drag a window to the left or right of the screen and have it resize to that position (see here for a video of this http://lifehacker.com/5077728/snap-any-window-to-half-the-screen-size-in-windows-7).

This is great feature except if you have multiple monitors. Areo Snap won't snap correctly to the part of the screen that goes across your monitors.

You can work round this by holding down the windows key and pressing the arrow key for the direction you would like to snap in (e.g. if your window is on monitor 1 then press the start key and right and will will snap to the right part of monitor 1)

Perfect!

P.S. Also don't forget you can press the start key up to maximize, down to minimize and you can use the number keys to select items from the start bar to maximize them (or bring them to the foreground)

jQuery and asp.net

2009-10-01T15:52:00.000+01:00

There are many ways to get asp.net control values from javascript. Here are a couple of ways with pro’s/cons

Normal Javascript:

document.getElementById('ctl00_ContentPlaceHolder1_TextBox1').value

This is massivly hard coded and bad. If you change the content place holder, or refactor this into a user control this will no longer work. But this is quick.

document.getElementById('<%=TextBox1.ClientID %>').value

This is less hard coded which is good, but the code can be a bit tricky to read.

With jQuery:

$('#ctl00_ContentPlaceHolder1_TextBox1')[0].value

Again, just as bad as the first normal Javascript example

$('#<%=TextBox1.ClientID %>')[0].value

This is ok compared to the javascript version

$('input[id$=TextBox1]')[0].value

This is probably my favorite method. This tells jquery to search for an input where the id ends with TextBox1 – be careful you have no other TextBox1 ids on the page!

Note the array indexer. This is because jquery returns an array of all the items it finds. This means if you have a grid with 12 rows you can access all of the textbox1’s using

$('input[id$=TextBox1]')[0].value -> $('input[id$=TextBox1]')[12].value

Now you are probably asking why would you just use the first javascript example. This is reasonable for a text box but say you are using a drop down list – which looks better:

var CustomerInputbox = document.getElementById('<%=ddlCustomer.ClientID %>');

var CustomerID = CustomerInputbox.options[CustomerInputbox.selectedIndex].value;

var CustomerID = $('select[id$=ddlCustomer] option:selected')[0].value

That is all.

Ross

P.S. Learn to love jQuery, resistance is futile.

What don't you know?

2009-05-20T09:28:00.003+01:00

As part of the review process at Waterstons we have to set objectives to try and achieve throughout the coming year. I want to orientate mine around stuff I didn't know but wanted to know - here is my list:-

Large scale architecture & design: Probably some kind of training course

Cloud Computing: Microsoft have a beta product called Asure which could offer us the ability to produce some very innovative and scalable solutions.

Microsoft enterprise library application blocks: these include code to make the following easier (and best practice): Caching, Cryptography, data access, logging and security.

Code generation: Learn more about the code generation techniques we are using

Sql: My sql skills are purely self taught and there are loads of advanced techniques I just don’t know (not really sure how I would self learn this, I do know I have very little knowledge in SSIS).

Visual Studio 2010 & .net 4.0: This will be with us soon and I couldn’t really tell you what we will be getting (the beta is out now)

ADO.Net data services: a colleague sent an email around about these – I think they could be quite a different way for us to consume sql information.

BI tools: I know the theory behind BI but not the tools

Test Driven Development: The tools could be useful, but I’m really not sold on this

ASP.Net Routing: This technology allows us to create .net web sites which are search engine optimised (SEO)

ASP.NET MVC: This is a different way of developing web applications that is gaining a lot of momentum – could easily become industry standard and Microsoft are putting a lot of money and resources into it.

ASP.Net data driven websites: Creates the scaffolding for data driven sites – should do things like code generation but this is from Microsoft

FX Cop/StyleCop: These tools perform analysis on source code to identify potential coding issues

It's really interesting to write a list like this - it defiantly helps organise your (lack of!) free time a bit better - what would be on your list?

Securing Anonymous web services with Certificates

2009-03-26T17:47:00.001+00:00

We are currently working with a client who requires two BizTalk web services to talk to each other, however because they are in different locations there is no common account we can use to secure the communication. Below shows an example of the setup:

BizTalk Server 1:

This orchestration has an input port (which accepts an xml file) – this message is then passed to a request reply port (which will call a webservice hosted on BizTalk server 2). The response is then passed to the output port (which is saved to an xml file).

BizTalk server 2:

This orchestration is a simple webservice which accepts a message, does something with the contents and returns the new message.

This posting assume you have two orchestrations already talking to each other using wcf without any authentication

Ok that’s the scenario defined – lets get onto the more interesting part!

Configuring BizTalk server 1 (The WCF consumer):

Since we are going to be securing the webservice with certificates lets start by generating the client certificate. Run the following code on the server which will be connecting to the webservice (BizTalk server 1 in my case) in a command prompt:

makecert -r -pe -n "CN=ClientCertName" -b 01/01/2005 -e 01/01/2100 -sky exchange -ss my

To view the certificate type mmc into command prompt, then go to file add/remove snap in, and select Certificates. You should be given an option to select select who you will be managing the certificates for – select “My user account”

Now if you look in the Personal folder you should see your certificate. We will need to install this same certificate in a few other places so right click on it and select export. Ensure you include the private key (you actually don't need the private key on the BizTalk server hosting the webservice, but for this demo I have included it).

Now we need to install this certificate in two different locations. On BizTalk server 1 we need to install it under the user account running the host instance. To do this open up a command prompt and type:

runas /user:DOMAINNAME\USERNAME mmc

then type the password for the user. In the management console again select the certificate add in. You can then import the certificate by right clicking on personal and selecting import.

Now we can configure this server to use the certificate you have added. Go into the send port you have generated and ensure you used the type wcf-wshttp – then click configure. For the address modify the url to use Https, then go to the security tab. Here you want to change the security mode to Transport with transport client type: certificate. Then in the client certificate panel click browse. Here you should be able to select the certificate you have just generated.

That’s the first server configured.

Configuring Server 2 (The WCF Provider):

On the second server go to the receive location and select your WCF port. Click configure and go to Security. Set the security mode to Transport, and the client credential type to Certificate.

Now we need to configure IIS to host the ssl web service.

Firstly we will need to get another certificate to allow ssl to occur – so right click on the web site and go to properties, then Directory Security and click server certificate. Click the “Create a new certificate” and next followed by “Prepare the request now, but send it later” (You may be able to click send the request immediately to an online certification authority depending on your setup) Fill in all the details ensuring the field named “Common Name” is the dns name used to connect to this machine from Server 1. This will generate a text file which you can then give to a certificate authority to get a valid ssl certificate (this will have to be a certificate authority trusted by Server one – so possible a public authority like VeriSign).

Once you have your certificate you should be able to add it by re-running the above wizard.

Now what we need to do is install the client certificate generated on server 1 – ensure this is installed into the Local Computer certificate store so IIS can access it. Remember for security this certificate should not really include the private key.

After you have added your certificate click the edit button shown below

Then select the following “Require Secure Channel”, “Require Client Certificates” and “Enable Certificate Trust List”, then click New...

This Wizard will allow you to specify the client certificates which can be used to access the webservice. Click Add from store then select the client certificate installed above. Click next, give it a friendly name, next and Finish. Then select the certificate list:

You will probably have to restart IIS now before this works correctly (and I had to restart IIS every time I changed the certificate trust list).

And thats it! Transport security with client certificates DONE.

Here are some error messages you might get and how to fix them:

The adapter failed to transmit message going to send port "SendPort1" with URL "https://ServerName/TestWebserviceCommunication/TestWebserviceCommunication_ProcessMessage_SendAndRecieve.svc". It will be retransmitted after the retry interval specified for this Send Port. Details:"System.InvalidOperationException: Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'CurrentUser', FindType 'FindByThumbprint', FindValue '23D68B9999C21FC93702D259736DD0CC1C752B61'.
   at System.ServiceModel.Security.SecurityUtils.GetCertificateFromStoreCore(StoreName storeName, StoreLocation storeLocation, X509FindType findType, Object findValue, EndpointAddress target, Boolean throwIfMultipleOrNoMatch)
   at System.ServiceModel.Security.SecurityUtils.GetCertificateFromStore(StoreName storeName, StoreLocation storeLocation, X509FindType findType, Object findValue, EndpointAddress target)
   at System.ServiceModel.Security.X509CertificateInitiatorClientCredential.SetCertificate(StoreLocation storeLocation, StoreName storeName, X509FindType findType, Object findValue)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.ApplyClientCertificate(ClientCredentials clientCredentials)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.CreateChannelFactory[TChannel](IBaseMessage bizTalkMessage)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.InitializeValues(IBaseMessage message)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2..ctor(IBaseMessage message, WcfTransmitter`2 transmitter)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfTransmitter`2.GetClientFromCache(String spid, IBaseMessage message)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfAsyncBatch`2.BatchWorker(List`1 messages)".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

This error is because your client certificate is stored in your cert store, but not the cert store used by the biztalk host instance. Look at the step above showing how to run mmc as another user to fix this problem.

The adapter failed to transmit message going to send port "SendPort1" with URL "https://ServerName/TestWebserviceCommunication/TestWebserviceCommunication_ProcessMessage_SendAndRecieve.svc". It will be retransmitted after the retry interval specified for this Send Port. Details:"System.ServiceModel.Security.MessageSecurityException: The HTTP request was forbidden with client authentication scheme 'Anonymous'. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult result)
   --- End of inner exception stack trace ---

Server stack trace: 
   at System.ServiceModel.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.EndRequest(IAsyncResult result)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at System.ServiceModel.Channels.IRequestChannel.EndRequest(IAsyncResult result)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.RequestCallback(IAsyncResult result)".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

This error is because the service you have used was not trusted on the Server 2, ensure you have the correct client certificate selected in IIS.

Shrink a transaction log without doing a backup

2009-03-15T22:27:00.001+00:00

If you use Microsoft SQL 2005 (or 2000, and probably 2008) then Sometimes you just have to shrink the transaction log in a hurry especially if you get this error:-

The transaction log for database 'DB NAME' is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases

I can never remember the command to truncate the log before you can use the inbuilt tools to shrink it.

In case you have the same problem here is the command:

BACKUP LOG [DBNAME] WITH Truncate_only

you should then be able to shrink the log without issue

P.S. There are reasons why you wouldn't want to do this (mainly around data recovery) so only do this on a test/dev environment.

Thanks

Ross

Useful Tools

2008-11-07T21:20:00.001+00:00

Here are my five most useful tools

Live Mesh

A great tool that you probably all know about. Lets you keep files and folders sync'd across platforms (PC, Mac and smartphone at the moment). What I love most about mesh is the direction its heading, the sync bit is pretty cool (but foldershare was sorta doing that anyway) as James blog shows there will be four types of 'things' which can interact with the mesh.

Creators - These are things like digital cameras (even standard digital cameras using eye-fi could use this! how cool).
Consumers - devices like mp3 players and digital picture frames
Rich end points - Pcs and Macs are the main things here, but may also include printers as shown below - this I REALLY like!
Processors - These are more applications that run on the mesh, things like backup to another cloud app.

You can watch the recent PDC talk here: http://channel9.msdn.com/pdc2008/BB35/ on it.

You can also use this to remotely control your pcs from anywhere - its great as it traverses firewalls really well, much more secure than opening your pc remote desktop port to the world!

This is wild speculation but I wonder how long before microsoft's Office Live (http://www.liveside.net/main/archive/2008/10/31/office-in-the-cloud.aspx) will be integrated into your mesh live desktop, and you could print to your home printer which is also in your mesh... very exciting Im sure you will agree!

IDrive

I love this tool. It is essentially an online backup tool which just works. You get 2GB for free or 150gb for $50 (annoyingly the set up recurring billing, and they didn't email me to remind me it would be coming out - even though they swear they did!). The backup and restore speeds are pretty decent, and the "continues backup" mode is a life saver - the first time you recover a file you deleted by accident you know it was worth the £30 ish quid!

IDrive also supports a history mode allowing you to roll back to a previously version - very handy (and this also doesnt come out of your space allowance). Im very impressed with the security, and cant recommend this tool enough!

Acronis True image

This tool is similar to N orton ghost but a lot more user friendly and cheap! For those who dont know Norton Ghost this allows you to take and restore an image of your pc. If you do this once a month your machine will run very quick.

This help prevent issues with activating software multiple times (starting to become more essential with the DRM included in just about everything).

If you use this with Live mesh then after you restore your pc, live mesh will automatically sync your latest files to your pc from the cloud - ideal!

Google Reader

Keep yourself up to date with all your favourite web sites everywhere you go (you can read it on your mobile to!)

My Sip Switch

Im just starting to get into using VOIP at home, and I have loads to learn still, but My Sip Switch is proving to be a very good starting place. I aim to go into a lot more detail soon, but currently Im able to use my sip switch for the following setup:-

I have two numbers from Sipgate, My Sip Switch is registered with both of them. One of the numbers (my personal number) will ring the house phone automatically, the second number will only ring the house phone during allowed hours(I will use this number for companies I dont trust!).

Sipgate Also provides a voicemail service, which get email to you. I am going to use the following tool to be alerted when I get an email:-

maybe a bit tacky, but its very useful to know when you have an email!

My sip gate also allows you to notify of calls via Google Talk, so I can tell when the house phone has called (and who called) when Im at work!

Using Fring I can also receive and make the voip calls via my smartphone, and Phonerlite to get the calls via my PC. For my home Im looking at a few different possible solutions for my home phone including a SPA3102 and a SPA1001. Both cost under £50.

Still havent quite decided how Im going to use for outbound calls, but the great thing is about My Sip Switch is that you are not tied into a particular provider. You can pick which provider you use for which calls, whether you sent your Caller ID or not, and implement redundancy (if one provider fails to call out then you can fail over to another).

Finally you dont need to worry about emergency calls anymore, Sipgate allows you to make 999 calls after you set a home location.

Hope you find some of these tools useful - if you do let me know!

Patterns of Enterprise Application Architecture

2008-09-11T21:39:00.002+01:00

I have just started reading this book and was fascinated. What I loved most about was that I could start identifying patterns I was already using - just didn't know what they were called.

The book starts of as a conversation about the different types of patterns (but always gives a page number where you can get more information) as you get towards the end of the book each pattern is discussed in much more detail (including examples in mostly Java and sometimes .net)

Unfortunately Jave appears to be Martin Fowlers language of choice - the Table Data Gateway pattern has this bit of code:-

public void LoadWhere(String whereClause){
   String commandString=
       String.Format("select * from {0} where {1}", TableName,WhereClause);
   Holder.FillData(commandString, TableName);
}

Now I suspect most people reading this will spot the issue with the above instantly, but if not read up on SQL injection!!! (if you are unsure how devastating a sql injection attack can be then watch this video from TechEd - scary! http://www.microsoft.com/emea/spotlight/sessionh.aspx?videoid=989)

Perhaps I'm being picky, this is definitely only sample code - but it worries me how many people might just use this code and not notice the potential issue.

Anyway - the book is very good and I do recommend reading it - as Martin Fowler him self says

"Since patterns are common solutions to recurring problems, there's a good chance that you have already come across some of them... I'm not claiming to present anything new in this book. Indeed, I claim the opposite, this is a book of (for our industry) old ideas.... An important part of patterns is trying to build a common vocabulary, so you can say that this class is a Remote Facade and other designers will know what you mean"

The common vocabulary is exactly what I have found most useful about this book.

You can find the book on amazon here:

http://tinyurl.com/3ntvlm (ISBN: 0-32101274200)

Getting ASP.NET menus to work with Chrome

2008-09-05T10:17:00.001+01:00

The asp.net menu does a check to see if the browser the client uses can support JavaScript or not. But the check is not a very good one!

You can force the menu to work by overwrite the Page_PreInit method on the page and telling the page that the client target is a modern “uplevel” browser:-

protected void Page_PreInit(object sender, EventArgs e) 
{ 
    if (Page.Request.ServerVariables["http_user_agent"].ToLower().Contains("safari")) 
    { 
        Page.ClientTarget = "uplevel"; 
    }
}

Please note that you have to do this in the page class file, and not in the master page class file. Obviously this means you have to do it for every page – which is not ideal.

To get round this limitation simply make a class which inherits from System.Web.UI.Page, and change all your pages to inherit from this class instead (a find and replace is ideal here!).

RegisterDataItem can only be called during an async postback Error

2008-06-26T13:26:00.001+01:00

I received the above error whilst using a ajax popup extender. Originally the popup extender was within an update panel and worked fine, the second I removed the update panel I started to get the issue.

The problem turned out to be this line:-

PopupStatusImage.Cancel();

which basically tells the popup extender to hide. You can only call .Cancel() or .Commit() if your popup extender is within an update panel!

You can simply check the Script manager to see if its a partial post back if you need to keep the cancel in for some pages etc.

Hope this helps someone.

Learn a new Language

2008-06-19T14:56:00.001+01:00

I started coding in Java at uni about 4 years ago and since then I have been very Microsoft centric. Recently I have been reading Chris Dalbys blog (http://www.yellowpark.net/cdalby/) and bought my self a current cost meter, and am very very tempted to get a SLUG.

However I dont know Linux and really cant code in anything other than C#. So should I learn another language? If so which one!

Java: Know a bit but haven't touched in in nearly 4 years.
PHP: Again learned a bit at uni but not in any depth
C++: Did about 10 minutes at uni of this! maybe something to learn!
Ruby: Quite liked the idea of this dynamic language but was not totally sold on it!
Python: No nothing about python other than its a scripting language!

Last year I had a very interesting conversation with Dave Verner about Ruby on Rails and he said I should aim to learn a new language every year - at the time I thought he was a bit nuts, and he probably thought I was a lost cause - but I can see sense in what he was saying now!

I suspect Im going to try and learn a bit more about Linux so hopefully a language which will run happily in Java makes sense (though Im pretty sure all the above run happily under Linux).

Any suggestions??

Geek Test

2008-06-13T23:53:00.001+01:00

If you find this funny, you are a geek!

DDD Scotland: Session Five

2008-05-10T17:00:00.001+01:00

Red, Green, Refactor!

Ben Hall

Test->Code -> Refactor

Need to change your mindset

Because the tests have to be written we need to stop and think about what we have to implement. This encourages better design.

Code is a bit less of a black box

Test code 101

[TestFixture]
public class ObjectUnderTestTests
{
    [Test]
    public void MethodUnderTest_State_ExpectedBehaviour()
    {
        string result=Convert.ToString(true)
        Assert.IsTrue(tesult);
    }
}

Look at the frame works xUnit and MbUnit both contain more assertions and more attributes.

The TDD Mindset

Huge shift in development approach

ALWAYS write a test before writing code

always run your test

Fix failing tests as soon as possible

What do we test?

Everything! In isolation

Express how we want to interact with the code

small, isolated section of functionality

Single test should not be testing too much

What not to do!

Showed a demo from Scott Gutheries blog showing a page load doing everything. Not possible to test.

Showed how cruise control can be used to automatically run all tests

You need to have a controller set on top of the repository.

Testing in Isolation: Test Doubles

More flexibility for testing the application.

Dummy Objects, Stubs, Mocks, Fakes.

A Stub acts as a stand in

Mocks are powerful normally development using a framework.

Normal: Business Logic --> Data Layer

Test Dobule: Business Logic -> IData

IData -> DataLayer

IData -> Double (Pretends to be the data layer)

Sample:

We now want to send an email after successfully insertion.

We have an interface which implements SendInsertNotification

The controller then takes a interface in the constructor - the test service then is used.

You can get a hold of a realistic stub email service which will open port 25 etc.

Would we be creating a stub for the database? Recommended not to. But we do still want to be testing in isolation. So in the Business Layer we use the mocked data layer, and in the data layer we use the real implementation.

These could fail for other reasons, like network going down etc.

When testing the database you can use a tear down method which can clean the database and reseed all ids.

You can also create [Setup] code which will initiate everything

Code Coverage

NCover will use the profiler API to monitor what lines of code are being called an those which aren't.

Resouces

TDD By Exampl - Kent Beck

www.codebetter.com

webblogs.asp.net/rosherove

RhinoMocks

TestDrivent.Net

www.NUnit.com

DDD Scotland: Fourth Session

2008-05-10T15:40:00.001+01:00

Making the leap into advanced SQL

Tony Rogerson

Sub Queries

Sub queries (in the select statement) execute once per each row

Derived tables

select *
from (
    select type, name 
    from sys.objects 
    where type='U'
) as Syst ( type, name)

This can be used to overcome issues with unions etc.

Nested Functions in sql 2000 causes the function to be called twice.

Logically derived tables are a virtual table.

There are no statistics held on them.

Its self contained.

Common Table Expressions

Can be used for recursion

;with cteName (name, type, rowpos)
as (
     select name,type, rowpos = RowNumber() over (partition by type order by name)
     from sys.objects
)

Unlike derived tables this does let you self join e.g.

declare @HowMany int
set @HowMay = 10
with loopIT(i, descr)
AS (
    select i=1, descr=CAST('Anchor' as varchar(100))
    union all
    select i=i+1, descr = CAST('recuresed' as varchar(100))
    from LOOPIT
    Where i<@howmany
    )

select i, descr
from LoopID

Like derived table there are no stats, and they are expanded - but they do allow self join unlike derived tables. By default you can only recurse 100 times - you can extend this with MAXRECURSION.

Temporary tables/Table Variables

#Tables hold stats - you can tell it to refresh them and then the plan is more accurate.

Table Variables are only meant for < 1000 rows - so you get no stats but you do get the option to recompile - option (recompile). However it is not very accurate

The Sql plan uses the order of the column to compute selects not the most optimal index as it would use on a #table.

Table variable is not in memory, even local variables are stored on disk.

Hash tables do cause a compile lock so this causes an overhead, where as table variables dont - so they are better!

Cumulative Totals

You can use sub queries for this

select *,
tytd = (select sum(accounts) where xxx),
mytd = (select sum(accounts) where xxx)
from accounts a

The one pass approach uses the case expression

select *,
tytp = sum(case when xxx then amount else 0),
mytp = sum(case when xxx then amount else 0)
from accounts a

You can use pivots tables also, and CTE tables

Performance

You shouldn't use Duration - it is has a high duration but a low number of reads that means there is a problem somewhere. Reads, CPU are the two columns you should read.

Sub query = 6.5k reads

1 pass = 417 reads

union all = 1200 reads

and the pivot 1201 reads

Running Totals

CTE can do it by adding a row number, then self join look for the previous row and sum as you go along. However the query gets copied into each query + it doesnt filter rows correctly so this costs a lot in reads

Better approach is to copy it into a temporary table, index it then make your CTE read from the temporary table. goes from 20k reads to 7k.

DDD Scotland: Third Session

2008-05-10T13:05:00.001+01:00

Integration Service in Sql 2008

Allan Mitchell www.SQLDTS.com\ www.SQLIS.com

Behind the scenes: Threading

Multicast Transform generates n copies of the input. This is the same in 2008 but all output are on the same execution Path instead of the same thread.

A raw file means that IIS doesn't have to translate information - this means that its ideal for staging data.

To work around issues with SSIS 2005 you use the union all component. This works because the union all works asynchronously. This causes another execution tree (Another thread). This is not needed in 2008.

Pipeline Limiter

This doesnt sound good because we are moving data! Limiter exists in 2005 so its not going to be slower in 2008 - just you didnt know about it. In 2008 they tell you about it so you now get performance counters.

Why would I want to restrict the pipeline?

Date = buffer = memory.
Memory is reused when the buffer terminates
push back from a component = no reuse of memory = run out of memory!

Dont use select * because it will need to allocate memory in the buffer for something we wont use.

To capture these event in the Feb CTP you need to check the log messages for an attribute starting with "The Component%" Expect this to change tho

You can now use C# in the script component to write/manipulate data.

Lookup Component

Caching Options

Full: Before its used everything gets cached - Slow startup
Partial: Lookup component will try for a match in the cache first, then check the file.
None - The last query is cached but that it. Try to order your data. This is useful for volatile data.

Full cache is fasted but might take longest to cache, and a large amount of memory.

In 2005 these cannot be passed around - so if you want to loop through a directory then it will be cached for every file! This is also not transferable.

In 2008 there is now something called a Cache Transform. This can be used as a destination.

You can cache to:

A file - Its a Raw File
In Memory

This allows the reference to be passed around.

Because you can doesnt mean you should - by default the 2005 way is used.

Row Redirection

If a lookup component gets no match then by default it fails. You have to configure a redirect down the error output to ignore it if this should not be an error.

Now in 2008 we have a no match output option.

Points to note:

Lookup transform is case sensitive when done in cache. Obviously this is not the case in SQL

Data Profiling Task

Profiles data in your database (can do quick setup or verbose). only SQl2000 or above can be profiled.

Side Note: If you are going to Sql server then use the OLE Connection manager or the sql connection manager

This is fantastic for seeing how rubbish the data is.

Useful for identifying distribution of values.

Microsoft have provided a date profile viewer to read the xml produced by the above component. Which is really powerful - you can drill into it quite easily.

This is useful for checking data sent from a provider.

Change Data Capture (CDC)

Billed as an ETL function.

Gives increased efficiency - incremental extractions.

You can identify changed rows and columns.

Essentially transaction replication. It uses the log reader.

A lot of people uses triggers - this doesnt!

Use a sp called sp_cdc_enable_table with parameters to define the table you want to do the change data capture on.

Supports net changes/all changes so you can decide how you need your data (and potentially save space)

DDD Scotland: Second Session

2008-05-10T11:46:00.001+01:00

[Apologies because my laptop blue screened and windows live writer doesn't do auto recovery:( I REALLY need a new laptop now! - everything up to abusing threads is from memory]

How to Write Crap Code in C#

Ben Lamb www.benlamb.com

Started the demo by showing an application which analyses all off Shakespeare's books to count every word and figure out how often the words are used. Initially he runs it and it takes 2 minutes 1 second which he then explains has a sleep.timer in there set to 2 minutes

Initial program: 1 second

Abusing Exceptions

Initially he checked a dictionary to see if a word existed before updating it. This was then changed to just try and update the dictionary, and if it doest exist handle the exception to add it in.

When he ran this he showed us the CPU maxing out, and performance monitor looking at the number of errors thrown a second (175000 was how many)

Exception flowing: 41 seconds

Ben 1 - Redmond 0

Abusing Threads
You can get a thread by:

Using async callbacks
Threadpool
manually using Thread()

to really screw up a program you have to use Lock()

modify the program so we have one thread for each text file, and then we lock on the dictionary.

Multi Threaded analysers 1 second

Ben 1- Redmond 1

Misuse of the heap (Fun with Strings)

Value objects get stored on the stack (ints etc), reference objects get stored on the head.

Avoid Boxing/UnBoxing
Use Generic classes
Cache Locality
Prefer code that works to fast code.

Garbage collection is designed to clean up code - it can use reference counters. This causes the counter to be locked which can cause performance problems. .net uses mark and sweep which walks through an object tree and looks and everything and object it comes across is marked because its being used. Anything not marked gets removed. Then it sweeps everything to group together memory (defraging basically).

Every time an object goes through a process like this its generation gets marked as either generation 0/1/2 - the ones which are old (gen 2) they get put to the bottom so the garbage collection doesn't worry about it too much. Pinned objects are objects which cant be moved because its unmanaged code - this confuses things a bit.

Garbage collection is not deterministic - but don't bother calling it yourself you just make things worse. Plus things like the virus checker could cause more issues.

Showed a normalise line which basically loops through an array of chars to check it is a letter or a space. String builder is then used to append the values together. He changes this to a string which concats itself with the char.

Ben explains that he uses system.diagnostics.stopwatch to show performance details.

Consider putting timer metrics into apps so you can track down issues.

There are some counters showing garbage collection information. % of time in GC is really what you are after. any more in 10% then you have issues.

String concatenation takes: 1 minute 24 seconds.

Ben 2 - Redmond 1.

DDD Scotland: First Session

2008-05-10T10:52:00.001+01:00

Web Services: We dont need no stinking web server

Barry Dorrans http://idunno.org/

Agenda

What is WCF
WCF Vs Remoting
SOA, Services, Interfaces
Proving it

Remoting is going, WCF is an open standard (WS*) so much better, Also transport neutral so we can quickly switch from HTTP to TCP/MSMQ etc.

Roman Kiss has a null transport module which allows you to just communicate in memory

WCF Vs Remoting

WCF requires less permissions, runs in medium trust (mostly).
Easier versioning with WCF.
Two way communications (callback contracts)

SOA approach. Write the contracts first.

ABCs of WCF: Address (where to talk), Binding (how to talk), Contract (what we are talking about)

A server endpoint is constructed of Address binding Channel

A service can have multiple bindings - defines the shape of the service and the security.

A contract is defined for the service, data and message - can also create one for errors etc.

MEP: Message Exchange Protocol

Request - Reply
One Way
Duplex

Service Contracts are decorated with the attribute [Service Contract] and the methods with [Operation Contract]

Data contracts are decorated with [DataContract] and properties/functions with [MemberContract] - this gives us control of the message body or header - but ordering allows us to create optional components or to order them.

Step by Step instructions will be published

Create a WCF service library

Visual studio 2008 includes a test client for the library project. This even lets you inspect your request/response header.

Shows a simple?! wcf configuration file. Right clicking on it bring up the WCF configuration editor. This allows you to really easily change settings like the bindings.

WCF doesn't require a webserver to host HTTP web services

With windows 2008 it gets better. In XP/Vista we have to write a host if we dont use IIS with WAS we can host a service under any protocol underneath IIS.

Running through a demo called Rumour. Talks about using a constant file to define the namespace.

Name spaces should include a date to support versioning to prevent breaking changes

showed how you can set isrequired=false but did this on an int... not sure how WCF can handle a value non nullable type not being passed in?: Barry said that this would result in the int being set to a 0.

Barry recommends using Visual Studio 2008 because 2005 doesn't have a test harness.

What have we learned

We need data contracts for versioning. WCF will not send items down unless members are marked as a data contract. Barry claimed that even if you mark data as serializable it wont send it down the link... Im not sure this is correct as im sure I have done this... so I will chase it up

[LIVE DEMO]

Showed the MEX endpoint exposing meta data information.

To stop the service you must do:-

internal static void StopService()

{

if(myServiceHost.Stae == CommuncationState.Faulted)

MyserviceHost.Abort();

else if (myServiceHost.State!= CommincationState.Closed)

MyServiceHost.Close();

}

The reason we need this is because you cant use the using statement. Disposing of the WCF service calls close but if it is in a faulted state then it will throw an exception.

Do try and host WCF in IIS.

Replacing ASMX - double decorate contracts what the action & message names, rename the service and only use basic http binding.

Demonstrated creating a service host by passing in the base address

AddressAccessDeniedException - the process does not have rights to the name space, so that needs to be registered... however you can just run as admin.

This command allows you to register the address:

netsh http add urlacl url=http:\\+:8080\RumourService user=builtin\users

This should be done at install time.

Getting service host information:

endpoint.Address

endpoint.Binding.name

endpoint.Contract.name

Add service reference and discover is clever enough to check local projects (if they have an app.config file). Does this require the mex data?

The app.config then needed to be renamed to stop an error from happening

[My battery ran out even after a full charge - need a new one! so had to resort to manual blogging (pen&paper) - apologies if if this not as accurate]

Error Handling

Need to turn on error propagation on in the web.config. This then send the error message through but still sends it as a fault exception.

You can declare a data context which is decorated with the fault contract which allows the correct error to be send to the client and handled.

This then marshals everything into a soap fault

Barry then did a very quick run through of duplex working.

NO SPECTATORS, ONLY PARTICIPANTS

2008-04-30T22:17:00.003+01:00

There are loads of great conferences going on in may

Where: Glasgow
How Much: Free!

First up on Saturday the 10th of May is DDD Scotland (mentioned here). This is a free community event with some great tracks and speakers including Barry Dorrans (covering WCF - cant wait), Colin Angus Mackay and whilst I havent heard of Gary Short before he perhaps has the best Bio I have ever read (Gary Short was employed as a canoeist, but was presumed dead after disappearing under mysterious circumstances, only to reappear 5 years later. Gary had used this time to secretly re-train as a software Jedi and now plies his trade throughout the known galaxies, where he sells his skills to the highest bidder. Outside of his dog, his laptop is his best friend; inside of his dog, it's too dark to use it.)

Where: Newcastle
How Much: £599 + VAT

Next up is Thinking Digital run by CodeWorks on the 21st to the 23rd... not so much development - although Steve Clayton will be there talking about Microsoft new Mesh technology - even though he didn't give me an invite!

Where: Newcastle
How Much: Free*

*=May cost a demo/presentation

And finally Bar Camp: North East straight after thinking digital on the 24th and 25th.

The Rules of ~~Fight Club~~ Bar Camp are pretty simple, you have to take part... and that worries me a bit! At work we hold regular knowledge sharing seminars and its a great way to pass on new skills and anyone who knows me will tell you how much I love going on about some new technology - but running a demo/session outside of work is a different kettle of fish!

So I don't really have any idea what will be covered here, but Im really looking forward to meeting other developers from the North East!

Looks like I am going to have a busy May!

DDD Scotland - registration open

2008-03-17T11:32:00.001+00:00

This looks like its going to be a very good community event with some really interesting tracks. You can see the agenda here and sign up here.

Really looking forward to this, just wish there where more events like this in the North East!

Idea for some software

2008-03-15T10:59:00.001+00:00

I have just seen Will Shipley's latest Blog posting (http://wilshipley.com/blog/2008/02/ted2008-part-1-gossip.html) and he says he has had his mac book stolen but he knows his MAC code, so he wants people to look out for it.

Well this got me thinking - what if everyone subscribes their laptop mac code to a website, and if it goes missing they flag it as stolen. Then working with people who provide WIFI services we have a little app which basically checks the list. If a stolen mac address is spotted then the police could be notified.

Just a thought - but I think this could be a really useful tool against laptop theft!

Ross