One of These Days

Facebook Users Need More Moustaches

2011-08-18T00:00:00+12:00

Drag the link below to your bookmarks bar, visit a friend’s Facebook profile, click the bookmark, and see how manly they look with a moustache.

Mustachify.fb

Many thanks to Aidan for creating Mustachify.me!

Instance Evil?

2011-06-15T00:00:00+12:00

Some playing with dynamic methods in a Ruby class led me into an interesting problem. ‘instance_eval’ was being used to allow for dynamic getting and setting of various attributes. This worked great, until the objects needed to be cached in Rails. An error was being thrown, “TypeError: singleton can’t be dumped”, which didn’t make immediate sense. Some digging and playing however led to the answer that Marshal cannot serialize objects that have had instance_eval used on them, which is probably fair enough as Marshall would have no way of knowing how to redo the eval when the object is loaded back again.

We can add an instance_eval block to add in an instance method, to demonstrate.

class A
  def initialize
    self.instance_eval do
      def unleash(what = nil)
         puts what 
      end
    end
  end
end

a = A.new
a.unleash("the hounds") # the hounds

However, we are not able to serialize the object ):

puts Marshal::dump(a) #> TypeError: singleton can't be dumped

The solution was to use the magical method_missing; we get the same result, but we’re also able to serialize the object.

class A
  def method_missing(m, *args, &block)
    puts "#{m}(#{args})"
  end  
end

a = A.new

a.unleash # unleash([])

a.unleash = 'the hounds' # unleash=(["the hounds"])

puts Marshal::dump(a) # o:A

Happily marshaled (:

(Reverse smilies inspired from Frank Chimero)

Phonegap With Jquery Mobile And Backbone Js

2011-04-18T00:00:00+12:00

There’s a lot of hype about mobile web apps being the future of mobile application development, but with my time spent developing for iOS, and seeing how enjoyable/powerful it was, I wasn’t convinced. So I took the plunge into PhoneGap, using jQuery Mobile and Backbone.js as my weapons of choice.

The app will be a fairly simple one, showing off the basic naviagtion features of jQuermy mobile, and some of the audio capabilities of PhoneGap. The content of the app will be driven by a single JSON file that we can easily update.

The post will go through each file, and the complete phonegap project can be found here on github.

script.js

Here’s our usual backbone namespace, with models, views and collections. We’ve got pages inside views to contain the different page types we will encounter.

var backbone = {
    models : {},
    views : {
        pages : {}
    },
    collections : {}

base is a global reference to our main DOM object. This was required to make jQuery mobile work a bit nicer, and we’ll get to it soon.

}, base;

Note: I would normally extend the backbone namespace with the name of the app I’m buidling, to ensure that multiple backbone apps in a page won’t interfere with each other.

$(function() {
    
    base = $("#base");

Launch the app, setting our view, and passing in page_data which contains our parsed JSON file.

    mainApp = new backbone.views.Application( {el : base, pages : page_data } );
    
});

views/Application.js

This is the first view that kicks off everything. If you’re coming from the iPhone world, consider this to be the (Home|Root)ViewController

$(function() {
    backbone.views.Application = Backbone.View.extend({
        
        initialize : function(args) {

Remember the pages variable we passed in during instantiation? We’re grabbing it here, and using one of underscore.js' enumerator methods to iterate over each page object. _.map is going to return an array with values equal to what we return in the closure, for each value in args.pages.

            this.pages = _.map(args.pages, function(page) {

We’re creating a new Page view, and passing in the page data that _.map has given us.

            return new backbone.views.Page({
                page : page
            });
        });

Now we have a backbone view for each page to be navigated, so we can call render to kick things off.

            this.render();
        },
        
        render : function() {

Just as iPhone apps have screen titles when you traverse content hierarchies, we want to be able to give each page a unique title. We need these titles to display on the main screen, so we can show the user what pages are available for them to navigate to. That’s why we’re collecting them here.

            page_titles = [];

_.each functions much the same as _.map, except it doesn’t return anything.

            _.each(this.pages, function(page) {

Remember that each page is a backbone view, so we’re calling render to get the HTML content that makes up the view, and appending it to our view element. As we set our el to be equal to base, we’re appending everything to the $(“#base”) element.

                this.el.append(page.render());

Grab the title and put it in the array, along with a unique id. The id is required for jQuery Mobile, which we will get to shortly.

               page_titles.push({ id : page.page_id(), title : page.page.title });
            }, this);

Using icanhazjs, we render the home template, passing in the array of page titles

            this.el.prepend(ich.home({page_titles : page_titles}));
        }
    });
});

New Blog

2011-04-11T00:00:00+12:00

Hello, world, again.

So, I’ve now completely switched blogging platforms, migrating off of Wordpress to a custom Jekyll setup. While wordpress was a seriously powerful platform, I was only using ~2% of its functionality, and that 2% wasn’t doing quite what I wanted. Now, with Jekyll, running on Sinatra on Heroku, I can hack things to work as needed. For example, posts can be pre-processed with Docco which creates the amazing styling on this page.

and the amazing styling of source code, like this –>

reasons.each do |reason|
    puts reason.explain
end

I figured it was worth the switch just for this alone, as most of the time I’m talking about code. I’m also hoping that the easiser method of posting (git push heroku master) will encourage me to write more frequently, so stay tuned!

(I will be gradually migrating posts from the old blog over to here, which hopefully won’t take too long)

Block Vs For In Objective C Enumeration

2011-04-01T00:00:00+13:00

Having now become used to, and in entirely in love with Ruby, I often find myself wishing other languages had blocks. It’s such an neat construct that leads to simplistic, elegant, and often more efficient code. During some work on an iPhone app, I had a need to iterate over the objects of a dictionary, getting both key and object for each pair. This is often done with a simple for loop, however the syntax was ugly and, in my opinion, wholly unnecessary:

    NSArray* keys = [data allKeys];
  for (int i = 0; i < objCount; i++) {
    [keys objectAtIndex:i];
    [data objectForKey:[keys objectAtIndex:i]];
  }

5 lines just to get keys and objects from a dictionary? Crazy! Pining for another way, I delved into the documentation and pulled out this gem of a method – “enumerateKeysAndObjectsUsingBlock”.

This does exactly what it sounds, and, oh what joy, it accepts a block.

    [data enumerateKeysAndObjectsUsingBlock:^(id key, id obj, BOOL *stop) {
    }];

The block is declared with a slightly odd, but manageable syntax: ^ (params) { code } In this case, the params are the key, object, and a boolean called stop. According to the documentation, if we set stop to true, the enumeration will stop, effectively acting as a ‘break’ function.

Now that we’ve got our code looking pretty, how does it match up in performance? Curious, I ran a simple experiment.

    int objCount = 100000;

  NSMutableDictionary* data = [[NSMutableDictionary alloc] init];
  for (int i = 0; i < objCount; i++) {
    [data setObject:@"test" forKey:[NSNumber numberWithInt:i]];
  }

  int iterations = 100;

  NSDate *methodStart = [NSDate date];

  for (int j = 0; j < iterations; j++) {
    NSArray* keys = [data allKeys];
    for (int i = 0; i < objCount; i++) {
      [keys objectAtIndex:i];
      [data objectForKey:[keys objectAtIndex:i]];
    }
  }

  NSDate *methodFinish = [NSDate date];
  NSTimeInterval executionTime = [methodFinish timeIntervalSinceDate:methodStart];
  float forTime = executionTime / iterations;
  NSLog(@"Standard For Loop : %f", forTime);




  methodStart = [NSDate date];

  for (int j = 0; j < iterations; j++) {
    [data enumerateKeysAndObjectsUsingBlock:^(id key, id obj, BOOL *stop) {

    }];		
  }

  methodFinish = [NSDate date];
  executionTime = [methodFinish timeIntervalSinceDate:methodStart];
  float enumerableTime = executionTime / iterations;
  NSLog(@"Standard Enumerable Loop : %f", enumerableTime);

  NSLog(@"Times Faster : %f", forTime / enumerableTime);

Here we’re simply creating a dictionary with 100,000 key/value pairs, and iterating over it grabbing both key and value, for the two different methods. To attempt to eliminate any time fluctuations due to unpredictable things like caching or context switching, we’re running each loop 100 times. (Note: I tried to run it 1000+ times, but it crashed probably as there are memory leaks galore in the code. Having said that, the memory leaks probably affect the run time as well, but I’ll leave that as an exercise for the reader to figure out.)

And the results? Surprising.

2011-04-01 18:42:49.163 Standard For Loop : 0.025023

2011-04-01 18:42:49.616 Standard Enumerable Loop : 0.004506

2011-04-01 18:42:49.616 Times Faster : 5.553028

Using the block is more than 5 (FIVE!) times faster than the ugly, verbose for loop version. I know what I’ll be using from now on.

Backbone Js Html5

2011-02-12T00:00:00+13:00

As stated previously, Backbone.js is buckets of fun to work with. In the spirit of this praise, I thought I’d have a play with it, and see how it could work with HTML5 animation development.

The end result is a cannon that shoots a small cannon ball, viewable here. While the example is entirely contrived, it demonstrates a number of aspects of Backbone.js, and well, is kind of fun. This first post will look at the Backbone.js portion of the app, and a second post will follow to briefly go over the canvas animation.

The first file we need is our main html page, which will include all of the necessary javascript files, a bit of styling, and the 2 HTML elements we need for our cannon. The cannon image is what we click to fire the cannon, and the firingRange canvas is where the cannon ball will be animated.

Next, we’ll look at the main.js file, as this is what kicks off the app.

$(function() {                
    myCannon = new CannonView();
});

That’s it. This will be executed when the DOM is ready, and will ensure our cannon is fully functioning. This file is a clear example of how backbone can help organise and DRY up your code. There’s no doubt as to where the execution flow of the app starts, and from here, it is easy to follow through. This makes testing easier, debugging easier, and allows other people to get up to speed with the code in a far shorter period of time (including yourself!).

window.CannonView = Backbone.View.extend({

    el : $('a.do'),

    model : new Cannon,

    events : {            
        'click' : 'lightFuse'            
    },

    initialize : function() {
        _.bindAll(this, 'fire');
        this.ball = new BallView;

        loadingView = new LoadingView;
        Backbone.beforeSync = loadingView.render;
        Backbone.afterSync = loadingView.remove;
    },

    lightFuse : function() {
        this.model.fire(this.fire);
    },

    fire : function() {
        this.ball.animate();
    }

});

One of three, the cannon view takes care of our event handling as well as setting up the other 2 views. It’s given the anchor tag as its element, allowing us to easily bind a click event. The model it’s given is Cannon, a simple model that will store the amount of ammo available, and persist this to a server.

When the cannon is clicked, lightFuse is called, which subsequently calls fire on the Cannon model. We’re passing in the fire function, which the model will treat as a callback and call it once firing has completed at the model level. This call will cause animate to be called on the ball, resulting in fancy animation. Note that in the initialize method we are using the underscore.js bindAll method to ensure that when the callback is passed into the model, it is still scoped to CannonView and not Cannon.

The last area of interest in this view is the 3 lines regarding the loadingView. The loading view is essentially our fuse animation. It seems appropriate to illustrate the loading of the cannon ball with such a depiction, and so we need a way of showing this when we click the cannon, and hiding it after it’s fired. While we could use the success callback in the Backbone.sync object, doing so would require us to put code related to the loading down in the cannon model. In keeping with a strict-as-possible separation of concerns view, it would be more appropriate to have this logic higher up, in the view class itself. As the fire method on the model is asynchronous, we cannot simply show the fuse before calling fire, and hide it afterwards. To achieve the functionality we want, I thought it would be an interesting exercise to edit the Backbone.sync object itself, in the backbone source. While this may not be the most appropriate way of doing this, it should at least be educational.

Two lines are added to the top of backbone.js to allow us to set before and after callbacks.

Backbone.beforeSync = function(){};
Backbone.afterSync = function(){};

The last edit needs to go in the Backbone.sync object.

    Backbone.sync = function(method, model, success, error) {            
        var params = {

here is where our after callbacks are run

          success: function() { success(); Backbone.afterSync(); },
          error: function() { error(); Backbone.afterSync(); }
          
        };

run our before callback

        Backbone.beforeSync();         $.ajax(params);
      };

We’re modifying the success and error callbacks to also execute the afterSync callback. It’s included in both success and error, as even if there is an error we want to stop the fuse from showing. Before we execute the ajax request, we call beforeSync. This will occur synchronously, and so $.ajax will be called afterwards. On completion of the request, afterSync will be called. As we have hooked loadingView.render to before, and loadingView.remove to after, we will now see the fuse light on click, and stop after firing. Note: if this way of doing things has made anyone cringe or question, I’d love to hear from you in the comments or email about an alternative way of achieving this. My justification is that the edits to the source are in no way connected semantically to anything in the app, and the functionality could fit a wide number of use cases.

While we’re on the topic of loading, let’s take a quick look at the loading view. It’s nothing fancy, just a view that on rendering, puts a span element containing and image into the DOM. We’ve bound render and remove to itself so that when inside the Backbone.sync callback, they still have the correct scope.

window.LoadingView = Backbone.View.extend({

   tagName : 'span',

   className : 'loading',

   initialize : function() {
       _.bindAll(this, "render", "remove");
   },

   render : function() {
       $('body').append($(this.el).html("<img src='images/fuse.gif'/>"));
   }

});

The final view for this app is the ball view. This is where our HTML5 canvas magic happens, and so for brevity, this will be covered in a separate post. The view has an animate method however, which when called by the cannon view, will show the cannon ball flying out of the cannon. This is a great example of how backbone can hide away implementation details. The main app knows nothing of the canvas animation, and we can easily swap this out for whatever we want. All the cannon view cares about is that there is an animate method it can call, and the rest will follow.

Finally, we come to the cannon model itself. In order to demonstrate the loading functionality, it was necessary to create some form of delay between calling fire on the model, and having the model state update. A simple php script was used that would randomly sleep for between 0 and 2 seconds.

sleep(rand(0,2));

and now our model:

window.Cannon = Backbone.Model.extend({

our delay script

   url : 'http://oneofthesedaysblog.com/server/backbone-canvas/sleep.php', 
   defaults : {

a new cannon has 100 cannon balls

ammo : 100 
   },

   fire : function(callback) {

only fire if we have positive ammo

       if (this.get('ammo') > 0) {
           this.save({ ammo : (this.get('ammo') - 1) }, {success : callback, error : callback });
       }
   }
});

We’re giving the cannon model a single attribute called ammo, which will keep track of how many times the cannon has been fired. This is being set an arbitrary value of 100, such that after 100 firings, it won’t fire again until the page is reloaded. The fire method is also here, and we can see that it takes in the callback we pass in from the view. If there is positive ammo, the model will try and save this to the server, with a value of ammo decreased by one. In the case of our simple script, we’re not really concerned whether or not the sync is successful, so I’ve set the callback to run on either success or error.

If we look back at the Backbone.sync object, we can see that this callback is being executed in the line we modified:

success:      function() { success(); Backbone.afterSync(); }

It’s running the callback we pass in to save(), as well as our afterSync callback.

That concludes the overview of the cannon app, with the exception of the ball view, which will be covered in another post. Hopefully this has been useful in, once again, highlighting the advantages of using simple MVC framework such as backbone. Our presentation and business logic is clearly separated, and the components are decoupled sufficiently such that we can easily make changes without breaking the rest of the app. The structure of the app is also useful for navigating the various areas of functionality, resulting in fewer WTFs per minute.

Ruby Day 14 Tty L

2011-01-22T00:00:00+13:00

I left the 30 day to Ruby series with a rather abrupt and unfinished ending. This post is the next in the hopeful continuation and completion of the series.

Without further ado then, Input/Output (IO)! For any operations involving sending or receiving data (be it to a file, or another computer), you’re going to need some way of telling the operating system what to do. Any modern OS has a set of system calls that are used to interact with the computer’s hardware, while ensuring nothing untoward happens to it. For writing to a file, we might call fwrite in the C standard library. By passing in a file name and some data, the library will instruct the OS to put the given data in the given file. We can also instruct the OS to send and receive data to things other than files however, a process, for example.

Let’s look at an example to make the explanations easier.

puts IO.popen('date').read # Sat Jan 22 22:01:35 NZDT 2011

We can see that the output of the above line was today’s date and time, exactly as if we were to open the terminal and type in the UNIX ‘date’ command. Note that we are calling the read command on the object we’ve created, exactly as if we were reading from a file. What’s going on here? popen, or process open, is a function in the Ruby IO library that instructs the OS to fire up a process with the given command (date, in this case). A process is similar to a file in that it can receive inputs and produce outputs, and so Ruby ensures theses two ends are available for use, just as if a file had been opened. The advantage of the IO library is that the API is fixed for whatever IO object you’re actually using. Be it a process, file, socket, pipe et al, the commands required for sending/receiving data are always the same. It can be seen then that after popen runs the date command, we are simply reading the output of this command, retrieving today’s date.

Let’s take another example, but to do so we must install a rather neat program called phonepipe. You could run a similar example without this, a notifo account and a smart phone with push capabilities, but it’s rather satisfying to do it this way. Keep reading once you’ve got it set up and working.

What we want to do is have Ruby send some text to our phone. A simple solution would be to simply have Ruby call ‘echo hello | phone’, which can be achieved using the shorthand %x syntax: This simply has Ruby drop out the commands contained into a shell, causing the word hello to be sent to your phone. This doesn’t fit nicely with our IO examples however so let’s try something else.

%x{ echo hello | phone} 

IO.popen('phone', 'w') do |f|
    f.puts 'Hello from process ' + Process.pid.to_s
end

Just as we opened up a process for date, we’re opening up a process for the phone command. The difference is that we’re specifying that we want to be able to write to it, just as we would write to a file. By passing it a block, we can do things with the returned process handle. Inside the block, we’re simply using puts to send our message to the phone process. You should see a message along the lines of “Hello from process ——” appear on your phone, where —— is any number. While this is purely a contrived example, you can see just how easy it is to read and write to processes in Ruby. To show the flexibility of these, take a look at this example which opens up an interactive Ruby prompt inside the process and has it execute Ruby code:

IO.popen('irb', 'w') do |f|
   f.puts "puts 'hello'"
end

We’re opening an irb process, and feeding it “puts ‘hello’” which gets run by irb and executed.
Note: if someone can turn this into a quine, it would be awesome to see in the comments! Here’s a start:

IO.popen('irb', 'w') do |f|
   f.puts " IO.popen('irb', 'w') do |f|
       f.puts \"puts 'hello'\"
    end"
end

Hopefully this brief foray into the world of Ruby::IO has piqued your interested, but before I TTYL we must look at the TTY. In times past, there was the Teletypewriter (TTY): A magical device that would take digital input and type output onto physical paper via a typewriter. This name, tty, has stuck around in UNIX, such that any terminal window you open is named something along the lines of ttyn where n is the index nth terminal window you’ve opened (the first window would be tty0, and so on). While nothing to do with typewriters, the idea of printing text to a screen is somewhat similar to this, so if you don’t think about it too much it makes sense. On a typical OSX or Linux system, the tty interfaces appear as simple files on the filesystem, allowing us to read and write to them. I happen to be running OSX, and I know that the first tty lives at /dev/ttys001, so if I open this up and write to it, the data will be displayed on the first terminal window I have open.

File.open("/dev/ttys001", 'w') do |f|
    f.puts 'Hello from Ruby!'
end

We’re using the File class in this case, not IO, to open up the tty file, but everything functions identically. If you have a terminal window up, you can check it’s tty file by viewing the terminal window’s properties. If it is indeed ttys001, you should see ‘Hello from Ruby!’ appear on it. If it’s something else, replace the file name and similar results should occur.

This is all well and good, but what can you do with it? Well, here’s a feeble attempt and what could be the beginnings of a game of pong:

width = 80
height = 20

length = 160
fps = 30

File.open("/dev/ttys001", 'w') do |f|
   row = 0
   increment = 1
   (1..length).each do |i|
      sleep(1.to_f/fps)
      f.puts "\e[2J\e[f" # clear screen character
      f.puts  "" + ("\n" * row) + (" " * (i % width)) + "o"
      
      row += increment
      if row > height
          increment = -1
      elsif row < 1 
          increment = 1
      end
   end
end

That’s all for now, but give it a whirl, and be sure to post any cool uses/examples you have in the comments!

2010 And Beyond

2010-12-30T00:00:00+13:00

The year being just about over, it’s time to write one of those summary posts. I’ll keep it brief!

From starting my first company, to finishing a degree in Software Engineering, to starting full time freelancing, the year has been an almost overwhelming learning curve. I wouldn’t, however, change a second of it. Thanks especially to the people I’ve met through study, work, meetups, twitter, and everywhere else; you’ve all been amazing, and helped make the year an absolute blast.

Slightly drained now, I’m looking forward to a bit of recuperation, and then will be right back into it next year, 2011.

To those of you who’ve visited me here I sincerely thank you, and I hope you’ll stick around for a hopefully more frequent and interesting year of posting to come!

Sam

A Simple Jquery Dropdown Menu

2010-12-20T00:00:00+13:00

A while ago I wrote about a bare bones CSS dropdown. Given the widespread adoption of jQuery since then, there’s little reason to go for the CSS only approach (although it is perhaps more elegant). @tedawebguy suggested I write a post about a more modern version, so let’s go!

(Note: This should work in all browsers, although I’ve only tested it in Firefox and Chrome on OSX. Any problems with it will be simple styling issues.)

Here’s the HTML we require:

Home
- Link 1
- Link 2
About
Blog
- Blog 1
- Blog 2
Contact

This is a simple nested list with 4 top level links : Home, About, Blog and Contact. Home and Blog have 2 sublists that we want hidden normally, but visible when their parent list element is hovered over. We can have an arbitrary number of links, but the current (read: lazy) version only allows nesting 1 level deep.

The styling we will need is fairly straight forward: The first style removes the bullets from our list, gives it a reasonable height and padding to ensure that the hover area is user-friendly enough.

    ul {
        list-style-type:none;
        height:30px;
        padding:10px;
    }

Next we’re styling the list elements, making them display inline and giving them a bit of padding and spacing. The border and cursor is not essential, but it makes it easier to see where to hover. Lastly, we’re giving it a relative position so that the child lists will be position relative to where this element is i.e. directly below the element you are hovering over.

    ul li {
        display:inline;
        padding:4px 50px;
        border:1px solid #555555;
        cursor:pointer;
        margin:2px;
        position:relative;
    
    }

To finish our CSS, we’re styling the second level list. It’s hidden to start with, and given an absolute position just below the top level list. The negative left position may cause some alignment problems in IE, but it’s there to offset the indent that browsers put on list elements.

    ul li ul {
        display:none;
        position:absolute;
        top: 26px;
        left:-15px;
        width: 600px;
    }

Now that we’ve structured and styled our navigation menu, we can sprinkle some javascript over it.

        $(function() {
            $("#nav li").hover(
                function() {
                    $(this).find("ul").fadeIn(100);
                }, 
                function() {
                    $(this).find("ul").fadeOut(100);
                });
            });

We’re including the Google hosted jQuery library as per best practices, and then kicking off with the usual document.ready shortcut that jQuery gives us. We’re then setting up a hover event listener on each list element in the top level list. This takes 2 callbacks, one for mouse over and one for mouse out. On mouse over, we’re finding any list that may reside inside of the current element and subsequently showing it. The reverse happens on mouse out. Fading was used to make it a bit easier on the eyes, while also allowing the mouse pointer to leave the list area for a few milliseconds. This increases the usability as the user does not have to be quite as accurate when moving the mouse from top-level to sub-list elements.

Plugging these 3 components together gives us the following 2 states:

If you want a more flexible solution, there’s a silly amount of libraries to be found on the web. Hopefully however, this post shows just how simple they are to construct, and may even be useful if you’re looking for a lightweight solution.

Backbone Js

2010-11-20T00:00:00+13:00

Backbone.js is a simple, very lightweight MVC framework built on jQuery and Underscore.js, a “utility belt for Javascript”. As with any MVC framework, the core premise of Backbone is to separate an application’s representation and storage of data, from the presentation of it. 2 main classes provide this functionality, Backbone.Model and Backbone.View. The neat thing is that when you associate a view with a model, any changes to the model will be reflected in the view without you having to write any linking code.

This sounds all well and good, but is it really that useful? Yes. Keen to get my teeth into it, I rewrote Vital Gifts app with it. The process took less time than the initial version, and the end result was a marginally larger Javascript file that executed faster while saving a large proportion of my sanity. The reason for this usefulness stems from the modularity it lends to your application; you can easily create self contained visual elements that have all of the data persistence and manipulation they need, and place them onto the page with one line. To illustrate this modularity, let’s create a simple friend selector that could be easily fitted out to use the Facebook Graph API.

This creates an empty model that we can use to store information about the selected friend. Note that we’re not specifying any attributes such as name or photo. This is because the Backbone model is just a JSON object, and so no schema is necessary – we can add and fetch attributes however we see fit.

window.User = Backbone.Model.extend({});

In a typical MVC setup, the view would contain a template with rudimentary data manipulations. Any events received by the view are sent to the controller for processing, and then results are sent back to the view. Backbone ‘breaks’ this model slightly by binding changes to the model directly with the view object. The results are still passable however, with the use of a template framework such as Mustache. We can use the Backbone view object for controller style logic, and keep the presentation in a template.

window.UserView = Backbone.View.extend({

create a new model to store the selected user

    model : new User,

the text field we’re using for the selector

    el : $("#friend-selector"),

we want to listen for 2 events on the text field

    events : {
        "focus"     : "selectInput",
        "keyup"  : "friendSelected",
    },

This first section is all we need to get our view up and running. We tell the view what model it represents, what DOM element it is going to use, and we define a set of event/callback pairs to handle user interaction. To do the actual autocompletion we will use the jQuery autocomplete library.

    initialize : function(friends) {

set up autocomplete

        this.el.autocomplete(friends, {
            minChars: 0,
            width: 310,
            matchContains: false,
            autoFill: false,
            formatItem: function (row, i, max) {
               img = "<img src='" + row.picture + "' width='50' height='50' alt='friend-picture' />";
               return img + row.name;
            },
            formatMatch: function (row, i, max) {
               return row.name;
            },
            formatResult: function (row) {
               return row.name;
            }
       });

bind this object to the form, so we can access it from the result event

        this.el.data('scope', this);
        this.el.result(function(e, data, formatted) {
            $(e.target).data('scope').autocompleteSelected(data);
        });
    },

I won’t go into detail about the autocomplete library, as the documentation on it is fairly comprehensive. We are using the initialize method to take in an array of friends that we want to be searchable in the element – this functions exactly as a typical object constructor. The caveat with this however is the last section, which is unfortunately an ugly solution. The callback for the autocomplete plugin is set with the ‘result’ method. From within this, we need to be able to call a method on the view object so that the model can be updated. The view won’t be in the scope of the autocomplete plugin however, so we are using the data property of the text field to store a reference to the view object. Both the autocomplete and the view model are attached to the same element so the data property acts like a form of shared memory.

    selectInput : function(e) {
        $(e.target).select();
    },

    autocompleteSelected : function(friend) {
        if (friend != null) {
            this.model.set({
                name : friend.name
            }); 
        }
    },

    friendSelected : function(e) {
        this.model.set({
            name : this.el.val(),
        });

see if they typed in a correct name

     this.el.search();  
    }
});

selectInput is purely a convenience for the user. It will result in the contents of the text field becoming highlighted when clicked on, making it easy to type in a new friend. It should save the user 1 click on average.

autocompleteSelected is called when the user selects a friend that has been autocompleted. It calls ‘set’ on the model and sets the name parameter to be the chosen friend’s name.

friendSelected is called on the keyUp event. It is required in case the user wants to type in a friend that is not registered in the autocomplete. If this were to happen, the autocomplete plugin would never match it, and so the result callback would never be fired. This would leave the user model empty, even though the user has typed in a name. When the event is triggered, the model is set to be the current value of the text field, and ‘search’ is called on the text field. Search is a method that comes from the autocomplete plugin, and it simply forces the plugin to see if the current value is in its list. If it can’t find it, we have already updated the model, keeping it consistent, and if it does find it, autocompleteSelected will be called and the model will still have the friend’s name.

The 2 objects above are all that we need to have a self contained friend selector. To render it on the page, we simply create a new UserView object and pass in a list of friends, assumed for this example to exist as a global variable called ‘friends’.

user = new UserView(window.friends);

That’s it!

Hosting Your Rails App

2010-08-27T00:00:00+12:00

So you’ve got this sweet rails app made, and it runs fine locally, but you’d like other people to be able to use it, right? For that, you can sign up for an account with a web hosting company, but with so many, how do you choose one?

So far, I’ve used 2 different companies to host a rails app, Rackspace and Heroku. I’ve settled with Heroku, and couldn’t be happier.

What they offer is essentially a Virtual Private Server (VPS) solution, along with your run of the mill cloud storage, much like Amazon’s ubiquitous S3. Having an entirely blank server that you can install any OS and software on is a real charm. Instead of having too little control, you have too much control. If you’re a sysadmin geek or bash guru, you will have a blast, making this a service that probably wouldn’t suit someone with limited server experience.

I installed a typical LAMP stack, with Ubuntu 8.x as my distribution of choice. There really was no difference from setting up this server than if I were doing it on a local development machine, with the exception of DNS. Alongside setting up your /etc/hosts file, you will need to use a simple control panel to add in your domain.

This excess of freedom however does create a large amount of setup work to get your server running, especially when it comes to getting rails going. Following this tutorial on setting up Passenger and Capistrano, it took around 30 minutes and a couple of headaches.

One downside with Rackspace is that it is down to the administrator of the account to ensure the server is secure. For an experience sysadmin this is probably a non-issue, but for those competent enough to get a server running but lacking any solid security experience, it can be a tad dangerous.

To sum up, if you’re looking for complete control, extremely reasonable pricing, on demand support, and don’t mind getting your hands dirty in the command line, then you will probably love Rackspace.

If you then navigate to the URL it gives you, you will see your rails app. The only caveat to this is that you must be in the root directory of your app, and it must be in a git repository. If you’re starting a brand new rails app, the setup would look like this:

heroku create MyAwesomeRailsApp
git push heroku

This will then have your new rails app running at http://skynet.heroku.com.

rails Skynet
cd Skynet
git init
git add .
git commit -m "Importing Skynet"
heroku create skynet
git push heroku

Heroku also comes with a swathe of plugins. From hosted MongoDB installations, to effortless DNS, to Cron jobs and auto backups, it’s like a Sith lord having the keys to the Padawan training room, and all the Jedis are away on holiday.

Essentially, Heroku has done to hosting what rails did for web development. Convention over configuration, excellent user experience, and having everything “just work” with the minimum amount of effort.

Another neat thing with Heroku is that all of the commands and tools it provides are accessible through a command line tool. You can add domain names, add more concurrency to your app, run rake commands, and just about every other feature they provide.

So if you’re looking for a no fuss, beautifully designed, effortless workflow, rubyesque, affordable and scalable solution, you might want to check out Heroku. (However, it’s built on Rails, which can’t actually scale, so you may want to take that into consideration.)

Ruby Day 13 Ruby On Rails

2010-07-19T00:00:00+12:00

Getting Started

Creating a weblog in 15 minutes
This was the video I started with, and it blew my mind (as it said it would). It really doesn't exaggerate when it says create a blog in 15 minutes. Start with this; you may not understand the intricacies and details, but you will be exposed to the features and power of rails, and after watching you will want to dive right in.
Getting Started with Rails
From the official rubyonrails.org website, this is the official quickstart. The site offers a wealth of information on all aspects of rails, with this particular guide taking you through the blog creation in the video above.
A List Apart: Getting Started with Ruby on Rails
If you're a little more cynical about this whole rails thing, A List Apart nicely outlines the who, where, whys and hows of rails. Comparing it to PHP, this is a good read if you come from that background.
Ruby Forums
When you start running in to strange errors and bugs, try doing a search at the Ruby Forums, or even posting a topic about it. There is an absolute wealth of knowledge, and a wonderful group of people.

Tools and Plugins

RubyGems User Guide
While not part of rails, gems will become an essential part of your work. There is a wealth of functionality that you can add to your app with only a few commands. This guide will explain a little more about this tool.
10 TextMate bundles/plugins to boost your Ruby on Rails development productivity
Hopefully by now you're getting the hang of rails. These Textmate plugins will make your time with rails even easier, and more enjoyable. (You are using Textmate, aren't you?)
Getting Started with User Authentication
Authlogic is an amazing plugin that lets you implement a fully functional user authentication system into your app with the absolute minimal amount of effort and code. This NETTUTS tutorial gives you a nice overview of how to implement it with an AJAX login form.
How-To Setup a Linux Server for Ruby on Rails
This excellent and in-depth guide shows you how to set up a linux server to host your rails apps. It uses Capistrano, Phusion Passenger and GitHub to manage deployment in a way that, after you've done it once, you'll wonder why you ever did it any other way.

Podcasts / Screencasts

Ruby on Rails Screencasts
With over 200 screencasts, Railscasts is a fantastic resource for new tools, best practices, tips, and more.
Peepcode Rails from Scratch Screencast
At just $9, this is fantastic value. The screencast shows you how to create a complete rails app, touching all of the important areas. Even though there is a lot of free information and tutorials available, the money spent on this is well worth it for the complete picture it provides on how to use rails. After deciding rails was too difficult for a period of time, this screencast got me right back into it and over the bump.
The Ruby Show
If you spend a lot of time on public transport, The Ruby Show is a fantastic podcast to have with you. It goes over the latest Ruby news in a humorous and insightful way. A great laugh, and a great resource.

That’s it for now, but I think these resources should definitely be sufficient to get you going, without completely overloading you with information.

Ruby Day 12 Unit Testing

2010-07-10T00:00:00+12:00

Before we get into unit testing in Ruby, I should apologise for the inconsistent posting of content. Amidst other work commitments, writing 1 post per day for 30 days is proving to be quite a challenge. I imagine at the end of this I will be able to write a post on “How Not to Write a Blog Post Series”. (Here’s a tip now: start writing at least a week before you start publishing)

We have a reasonable HTTP server working for us now. It can handle multiple simultaneous requests, log important information out to a file, and handle query parameters in the requested URL. While we can see that it works by running it, it would be good to have a more thorough test which we can run often and easily to ensure that any future changes do not break it. We will be using the Ruby Unit Testing Framework to achieve this.

Writing unit tests is a bit like using source control. Once you’ve set it up and are using it regularly, your level of sanity is slightly higher than if you weren’t using it. Similarly, once you’ve written some good unit tests, they will always be there for you to check that your code runs ok. An example of where a good set of unit tests would be handy is in a series of text manipulation operations. On a UNIX machine you may simply be checking for the existence of ‘\n’ for newlines, but if the same code is run on a Windows machine it may fail. By running the tests on each new machine, issues like this will become apparent with very little extra effort.

As the synchronised buffer is a crucial component of the multi-threaded web server, we will create a set of unit tests specifically to test this class.

A unit test class at its most basic looks like this:

class SyncBufferTest < Test::Unit::TestCase
  def test1
    assert(true)
  end
end

We are simply extending from the ‘Test::Unit::TestCase’ class and then defining our test in the method ‘test1’. There is a convention that must be followed for defining new tests, but it is a simple one: each method must begin with ‘test’. You can have whatever you like afterwards, but with ‘test’ at the front Ruby will know this is to be treated as a test case. The following are all valid test case names:

test1
test_my_test
test_if_exception_thrown
test123ABC

In order to examine more of the Ruby Unit Test API, we will need to make a quick addition to the SynchronisedBuffer class. Add the following line to the beginning of the initialize method. (i.e. before @capacity = capacity)

raise "Capacity must be greater than 0" unless capacity > 0

This does exactly what it says it will: raises an exception with the message “Capacity must be greater than 0” unless the capacity is greater than 0. This stops someone creating a new buffer with a capacity of 0 or -1, which simply does not make sense. We can then write test cases to check that this actually happens.

def test_zero_capacity
      begin
          SynchronisedBuffer.new(0)
      rescue 
        assert_equal("Capacity must be greater than 0", $!.message)
      end
  end

  def test_negative_capacity
      begin
          SynchronisedBuffer.new(-1)
      rescue 
        assert_equal("Capacity must be greater than 0", $!.message)
      end
  end

‘test_zero_capacity’ and ‘test_negative_capacity’ are testing this line we just added in above. An exception should be thrown when the supplied capacity is less than or equal to 0. By using the error handling begin/rescue syntax we can catch any exceptions thrown. In the rescue section, we are calling ‘assert_equal’ which is a method provided by Test::Unit::TestCase to ensure that 2 values are equal. It is by convention that the expected value comes before the actual value, thus we are seeing if “Capacity must be greater than 0” is equal to the message contained in the thrown exception. ‘$!’ is a global variable that contains the most recently thrown exception, and the message method returns the message that the exception contains.

If we now run this file we should see output like below

A dot, ‘.’, represents a test that has passed. We can see results in a sentence as well, ‘2 tests, 2 assertions, 0 failures, 0 errors’. We have 2 tests that asserted 1 statement each, and neither failed nor threw an error. If we remove the begin/rescue block from each test, and simply have ‘SynchronisedBuffer.new(0)’ then there would appear an ‘E’ for error instead of a ‘.’ as the thrown exception is not being rescued. Alternatively, if in our SynchronisedBuffer class we did not put in the line to raise the exception, we would see an ‘F’ in place of a ‘.’ as no exception is being thrown, and so no assertion is true. (This idea of writing tests that will fail before the function has actually been implemented is known as Test Driven Development).

The trick to unit testing is finding the right set of test cases that don’t overlap, but don’t miss anything out, either. We have tested the capacity of 0, which is neither a positive nor a negative number. It is in a class of it’s own and so we must include this test. -1, however, is a negative number just like -2 or -9,235,123. It is in a class with an infinite amount of other negative numbers. It would be impossible, and senseless, to test each negative number as we know that the condition only requires the number to be greater than 0. -1 is therefore sufficient to ensure that it will work with any negative number.

Let’s now test the empty? and full? methods. Here we are creating a new buffer with a valid capacity and then calling the ‘assert’ method. This simply checks if the value inside returns true. If it does, then the assertion is true. Without putting in any objects, empty? should return true. With a buffer of capacity 1, after putting in 1 item full? should return true.

def test_empty?
    buffer = SynchronisedBuffer.new(1)
    assert(buffer.empty?)    
end

def test_full
    buffer = SynchronisedBuffer.new(1)
    buffer.put("item 1")
    assert(buffer.full?)
end

These tests check that full? does not return true when it shouldn’t, and likewise with empty?.

def test_not_full
    buffer = SynchronisedBuffer.new(2)
    buffer.put("item 1")
    assert(!buffer.full?)
end

def test_not_empty
    buffer = SynchronisedBuffer.new(2)
    buffer.put("item 1")
    assert(!buffer.empty?)
end

We now have some simple test cases in place, but they only ensure a limited part of the functionality. What we really want to test are the put and get methods. As these methods can potentially put to sleep the thread that calls them, we have to be careful when testing them. The scenarios we must test for are

calling put while the buffer is empty
calling put while the buffer is full
calling get while the buffer is empty
calling get while the buffer is full
calling get while a thread is waiting on put
calling put while a thread is waiting on get

and we can translate them to the following test case names

def test_put_while_empty
end

def test_put_while_full
end

def test_get_while_empty
end  

def test_get_while_full
end

def test_get_waking_up_sleeping_thread
end

def test_get_waking_up_sleeping_thread
end

Our first test is rather simple We create a new buffer with a capacity of 1. After putting in 1 item, the buffer should be full. The buffer was empty when we called put, so the current thread should not be put to sleep while waiting for the buffer to empty. Thus we are asserting that the status of the current thread is equal to ‘run’. This concept of checking thread status will be very important for the rest of the test cases.

def test_put_while_empty
    buffer = SynchronisedBuffer.new(1)
    buffer.put("item 1")
    assert_equal('run', Thread.current.status)
end

As before, we are creating a new buffer and placing in an item so it becomes full. As the buffer is full, any subsequent calls of put will cause the calling thread to sleep. As we don’t want to put our main thread to sleep, we create a new thread and pass in the buffer. By calling ‘put’ in this thread, we expect the thread to go to sleep. The line that comes after,

def test_put_while_full
    buffer = SynchronisedBuffer.new(1)
    buffer.put("item 1")
    thread = Thread.new(buffer) { |buffer|
        buffer.put("item 2")
    }
    Thread.pass  # run 'thread' to ensure it sleeps
    assert_equal('sleep', thread.status, 'Thread should be asleep waiting to put')
    thread.kill!
end

Thread.pass is important as it without it, assert_equal might be called before the new thread has had a chance to actually call ‘put’. If this were the case, the thread would not yet be asleep, and so it’s status would be ‘run’, and the test would fail. ‘Thread.pass’ says to the Ruby thread scheduler “I’ve had enough, let another thread run for a while”. This would give the thread to actually call buffer.put and then we can test if it’s state is set to ‘sleep’. Lastly, while perhaps not necessary, we are destroying this new thread.

Having written this test case, the subsequent ones will follow much the same pattern.

def test_get_while_empty
    buffer = SynchronisedBuffer.new(1)
    thread = Thread.new(buffer) { |buffer|
        buffer.get
    }
    Thread.pass # run 'thread' to ensure it sleeps
    assert_equal('sleep', thread.status, 'Thread should be asleep waiting to get')
    thread.kill!
end

def test_get_while_full
    buffer = SynchronisedBuffer.new(1)
    buffer.put("item 1")
    buffer.get
    assert(buffer.empty?)
end

This last test case has a minor addition, ‘thread.join’. We are testing the case when we are trying to put an item into a full buffer. This call will make the thread sleep until someone else calls ‘get’. Once get is called, we want to ensure that the thread waiting actually completes calling ‘buffer.put’ and so we use the join method. Join makes the thread behave synchronously, pausing the current thread until the joined thread has finished executing. If thread.alive? returns false, we know that this thread has died. For it to have died it must have completed executing, and for that to have happened it must have been woken up. Thus if the assertion is true, calling ‘get’ successfully woke up our thread. The same applies for calling get on an empty buffer.

def test_get_waking_up_sleeping_thread
    buffer = SynchronisedBuffer.new(1)
    buffer.put("item 1")
    thread = Thread.new(buffer) { |buffer|
        buffer.put("item 2")
    }
    Thread.pass
    buffer.get
    thread.join
    assert(!thread.alive?)
end

def test_get_waking_up_sleeping_thread
    buffer = SynchronisedBuffer.new(1)
    thread = Thread.new(buffer) { |buffer|
        buffer.get
    }
    Thread.pass
    buffer.put("item 1")
    thread.join
    assert(!thread.alive?)
end

The results of all our test cases running should look something like the following:

With all these test cases in place, we can be fairly certain that our implementation is correct and the any future bugs that may be introduced will be caught. Note that each test case only makes 1 assertion. While this is not entirely necessary, it is good practice. If a test case fails to pass, then there is no doubt as to which assertion failed. By placing 10 assertions into a test case ‘test_buffer’ it becomes much more difficult to find the exact cause of the failure as the name ‘test_buffer’ failing tells us nothing and what was tested. If multiple assertions are made in the same test case, either one or more of them is redundant, or the test cases should be separated into multiple separate cases.

Hopefully these examples give you an understanding of how to use the Ruby Unit Testing Framework, as well as how to test synchronised and multi-threaded classes. As always, please leave a comment or send me an email with any feedback, criticisms, questions or comments.

Ruby Day 11 Zlib And Gzip

2010-07-05T00:00:00+12:00

While high speed internet is gradually working its way into most homes, it’s not absolutely everywhere, and when it is, your ISP may be limiting speeds during certain times of the day or for certain types of internet traffic. To make things easier on users, we can have our web servers compress the data they send. Instead of sending a 200kb Javascript file, it can first be compressed on the server to ~50kb, sent via TCP, and then uncompressed by the web browser. The amount of data being sent is ¼ of the size, and so the total time is reduced significantly (time taken to compress and uncompress is considered negligible).

Unfortunately, my attempts to wrangle Ruby’s Zlib compression library have not been successful. The documentation is patchy, at best, with comments such as “???”, “TODO: better comments”, and the occasional snippet of Japanese. Given today’s time constraints, I’m shipping an example that I feel should work, but does not. I will work on resolving the issue, and update this post when it works.

socket.puts "HTTP/1.1 200 OK\n"
socket.puts "Connection: close\n"
socket.puts "Content-Type: application/gzip\n"

File.open(file, 'r') { |f|  
    gz = Zlib::GzipWriter.new(socket, Zlib::BEST_COMPRESSION, Zlib::FINISH)  
    while (line = f.gets) 
        gz < < line
    end
    gz.close
}

I have modified the header to say we are sending the browser compressed data, and created a new Gzip writer object inside the response loop which spits out the contents of the requested file into the gzip writer, which has been given the socket. The documentation states that Gzipwriter can take an IO object, which socket happens to be, however when it runs an exception is thrown saying that the IO file has been closed. While possibly something simple, the answer has thus far eluded me.

If you have any suggestions or solutions, leave a comment or send a tweet/email. Stay tuned for updates.

Ruby Day 10 Uri

2010-07-04T00:00:00+12:00

Today we will take a brief look at Ruby’s Uniform Resource Identifier (URI) class in order to handle a wider range of requests in our web server. Currently, if we enter a URL such as ‘http://localhost:8080/test.html’ into our web browser we will receive a request that looks like ‘GET /index.html HTTP/1.1’. If we want to send any extra parameters such as an id (http://localhost:8080/test.html?id=123) then our server will naively assume that the file we are requesting is named ‘index.html?id=123’, and so it will not be able to find it.

By treating the request as an HTTP URI, we can interpret the request more intelligently, separating it into the filename and the query parameters.

We can modify our worker class like so

file = request.split(' ')[1]
file = '.' + file

uri =  URI.split(file)
file = uri[5]
query = uri[7]

The ‘split’ method will take a URL and split it up into 9 components which will be explained by using the URL ‘http://www.google.com/index.html?user=123’ as an example

Scheme – http Userinfo – nil Host – www.google.com Port – 80 Registry – nil Path – /index.html Opaque – nil Query = user=123 Fragment – nil

(Userinfo, registry, opaque and fragment are not likely to be used for an HTTP request so we can ignore them.)

The two components we are interested in are path and query, elements 5 and 7 respectively. By passing the middle part of our GET request to the split method, we can grab the 2 elements we need. ‘uri[5]’ contains the file we want, so that will be passed to the File class. ‘uri[7]’ contains the query parameters which can be dealt with later on.

Ruby Day 9 Logging

2010-07-02T00:00:00+12:00

With most applications, it is important to keep an accurate and comprehensive record of any problems or significant events that occur during its execution. This typically involves sending this information to a log file stored on the computer. The Ruby Logger class lets us easily take care of this.

After importing ‘logger’, we can create a new logger like so:

log = Logger.new('server.log')
log.level = Logger::DEBUG

We want to save all messages into the file ‘server.log’. Alternatively, we could have entered ‘STDOUT’ to send all logging information to standard out. The second line sets the types of messages we want to log, which can be any of FATAL: an unhandleable error that results in a program crash ERROR: a handleable error condition WARN: a warning INFO: generic (useful) information about system operation DEBUG: low-level information for developers (Taken from Ruby API)

The level you set log.level to will mean that any messages logged equal to, or above this level, will be logged.

log.level = Logger::WARN

This would result in FATAL, ERROR and WARN messages being logged, but INFO and DEBUG being ignored. In a production environment you may only want to log fatal errors, while in development mode you would most likely want to see DEBUG information.

To add a message to the log we can simply call a method that has the name of the level we are logging to.

log.fatal "Program will now crash"
log.error "An exception occurred, but we're onto it"
log.warn "Memory running low"
log.info "1000 customers have now signed up"
log.debug "The program is on line 25"

Let’s modify our multi-threaded server to include some logging.

log = Logger.new('server.log')
log.level = Logger::DEBUG
  
server = TCPServer.new('127.0.0.1', '8080')
log.info "Server started on 127.0.0.1:8080"

buffer = SynchronisedBuffer.new(100)

workers = []

for i in (1..40)
   workers[i] = Worker.new(buffer)
   workers[i][:name] = 'worker' + i.to_s  
   log.info "'Worker #{workers[i][:name]} created"
end

while socket = server.accept
    log.info "New connection from #{socket.peeraddr[2]}"
    buffer.put(socket)
end

In each case we are printing out some useful, but non-critical information, so it is logged as info. We can also add logging to the worker class – the file has been attached for brevity’s sake.

If we run the web server with the changes made above, we will see entries in our server.log file like the following: Logfile created on Fri Jul 02 19:57:47 +1200 2010 by logger.rb/22285

I, [2010-07-02T19:57:54.231591 #14068] INFO — : New connection from localhost

I, [2010-07-02T19:57:54.232061 #14068] INFO — : worker1 has received a new socket

As each message has a precise timestamp, it can useful for tracking down bugs and performance issues. While ‘puts’ may be a simpler and quicker method for printing out this kind of information to standard out, the extra information and structure that logger provides makes it well worth using.

Ruby Day 8 Multi Threading Synchronisation And Buffers

2010-07-02T00:00:00+12:00

Now that you’ve seen the HTTP protocol in action, we can move on to a more advanced version. We will turning the server into a multi-threaded one, allowing it to handle multiple requests simultaneously. Previously, once we had accepted a socket we had to finish processing it until we could accept another one. This meant that if another person tried to connect while it was processing an earlier request, it would have to wait. By sending each request to a separate thread, more than one can be processed at a time resulting in a more responsive server for everyone.

The approach we will take to implement this is to use so-called circular buffer. It is essentially a ring with a start and an end. Items go into the end, and come out the front. As we add items, we move the back along to an empty slot. When we take items out, we also move the front along to an empty slot. When the front is the same as the back, we know the buffer is either completely full or completely empty. Lastly, the buffer will have a fixed capacity so that when the front is moved to a position greater than the capacity, it wraps back around and starts from 0.

When we accept a connection the returned socket will be placed into the buffer. A group of threads will continually be checking the buffer for new sockets, and when one is found a thread will take it and process it. This process is useful because it allows us to completely separate our request handling and response generation. We will call these threads ‘Worker’ threads, and the collection of them will be called a thread pool. Thus, a new request will be placed into the buffer and processed by a worker from the thread pool.

There is a slight caveat to working with threads that use any form of shared data structure (each thread has access to the same circular buffer). A thread can be interrupted at any time by the scheduler and so a number of problems could ensue. For example, let’s say we have a method called ‘put’, which takes a given object and stores it, and then updates the count of the number of stored objects. If the thread is interrupted in between the storing of the object and the updating of the count, the data structure is no longer consistent with itself. If there were no elements added before ‘put’ was called, the buffer will still say it is empty even though one object has been added. What’s worse is that these problems are wholly unpredictable, so we must find a way to ensure these problems can’t occur. We need a way of guaranteeing that once ‘put’ is called, no one else will access the buffer until the method has finished.

Ruby provides a class called ‘Monitor’ that will allow us to achieve this by letting us create ‘synchronised’ blocks of code. A synchronised block is one that can only be accessed by one thread at a time. A thread can get a lock on the method, and no other thread will have access to it until the lock is released. Let’s go over the logic we will need to implement for our buffer.

We need to put items in, and get items out. The buffer will have a fixed size, and when it is full we should not be able to put anything more in. When it is empty, we should not be able to get anything out. If we want to get something while it is empty, it should pause until something has been put in, and then return it. If we want to put in something while it is full, it should pause until something has been removed.

Let’s look at the implementation of this, then go over it.

require 'monitor'

class SynchronisedBuffer < Monitor

We are extending the Monitor class, which gives us access to ‘synchronize’ and the wait/signal methods. Next, the buffer is initialised

   def initialize(capacity)
      @capacity = capacity
      @front = 0
      @back = 0
      @elements = Array.new(capacity)  
      @empty_cond = new_cond
      @full_cond = new_cond
      super()
   end

After setting capacity, front position, back position, and creating a new array to hold our elements, we come to our first bit of the Monitor class. ‘new_cond’ is a method that returns a ConditionVariable object. This condition variable is an important construct that ensures the code will only execute when the condition is satisfied. We have a condition that elements can’t be removed when the buffer is empty (empty_cond) and a condition that we can’t add new elements when the buffer is full (full_cond). To have a section of code that is synchronised, we call the ‘synchronize’ method and have the code inside a do-end block. Once inside, we call the ‘wait_while’ method on the empty condition. We are passing in the method ‘empty?’ as our condition, and so the line can be read as ‘have any thread that calls this method wait as long as the buffer is empty’. Alternatively, we could write ‘wait_unless { !empty? }’ which would read ‘let any thread call this method unless the buffer is empty, in which case, make them wait’.

 def get
       synchronize do
           @empty_cond.wait_while {empty?}   
           element = @elements[@front]
           @elements[@front] = nil
           @front = (@front + 1) % @capacity
           @full_cond.signal 
           return element 
        end               
   end

When the buffer is empty, any thread that calls get will be put to sleep. If the buffer isn’t empty, then the element at the front is removed, and the front position is moved along by 1. The line ‘@full_cond.signal’ will wake up a thread that was put to sleep waiting for the full buffer to have some space. We have just removed an item, therefore there is now space to put in a new item, so we can wake up, or ‘signal’ a sleeping thread. ‘put’ is the opposite of get. Threads must wait if the buffer is full, and when there is space, an element is placed at the back of the buffer. Once this is done, ‘@empty_cond.signal’ is called and a thread who was put to sleep waiting for the buffer to have some items placed in it, is woken up. As only one thread is woken up at a time, it functions on a first come first served basis.

def put(element)             
       synchronize do
           @full_cond.wait_while {full?} 
           @elements[@back] = element
           @back = (@back + 1) % @capacity           
           @empty_cond.signal    
        end        
   end

The final two methods we have already seen used above If the front is the same position as the back we know that the buffer is either empty or full. We know it will be empty if the element at this position is nil, and full if the element at this position is not nil. These methods must also be synchronised as otherwise they could be interrupted in-between the two conditions, and a corrupted buffer state could result.

   def full?
     synchronize do
       (@front == @back and @elements[@front] != nil)
     end
   end

   def empty?
       synchronize do
           (@front == @back and @elements[@front] == nil)
        end
   end

end

Sincere thanks must go to Robert Klemme from the Ruby Forum who helped me work out the bugs in this implementation, as well as Craig Taverner from his blog for introducing the monitor class.

So now we have a synchronised buffer which we can fill with incoming requests. What we need next is a thread pool full of workers who are ready to process the contents of the buffer. The class below should look fairly familiar to what was in the basic web server, with just a few additions.

require 'thread'

class Worker < Thread

    def initialize(buffer)
        super(buffer) { |buffer|
            begin
                loop do
                    socket = buffer.get

                    request = socket.readline

                    validGET = request.match(/GET .* HTTP\/1\.1/)

                    unless (validGET) 
                        socket.puts "HTTP/1.1 400 Bad Request"
                        socket.close
                        next
                    end

                    file = request.split(' ')[1]
                    file = '.' + file

                    unless ( File.exists?(file) ) 
                        socket.puts "HTTP/1.1 404 File Not Found"
                        socket.close
                        next
                    end

                    socket.puts "HTTP/1.1 200 OK\n"
                    socket.puts "Connection: close\n"
                    socket.puts "Content-Type: text/html\n"
                    File.open(file, 'r') { |f| 
                        while (line = f.gets) 
                            socket.puts line
                        end
                    }

                    socket.close
                end
            rescue Exception => e
                $stderr.puts $!.inspect
            end
        }          
    end    
end

The main difference is at the top, and the bottom

class Worker < Thread

    def initialize(buffer)
        super(buffer) { |buffer|
            begin
                loop do
                    socket = buffer.get
          ...
        end
      rescue Exception => e
   				$stderr.puts $!.inspect
      end
    }          
  end    
end

Extending the Thread class means we can treat Worker objects exactly as if they were threads. To do this however, we also need to implemenet the initialize method. When creating a regular thread, the instructions to execute are passed to it as a block

Thread.new { # do something }

Therefore, all we need to do is call ‘super’, as this will call the initialize method on the parent class, Thread. Placing the block after this will mean that it is executed by the Thread class. We want to pass in the buffer for it to use, so this is passed in as an argument and then into the block.

As each thread will be running outside of the main execution thread, we will not be informed of any errors that occur which can make bug finding difficult. If we wrap the code to be executed in Ruby’s equivalent of a try/catch block then we can grab any exceptions thrown and send them to standard error. ‘$stderr.puts $!.inspect’ is a neat shortcut that uses the $! global variable. This contains the most recently thrown exception, and so we are calling inspect on this and sending it to stderr.

Lastly, instead of executing the processing code once, we are looping infinitely. We can to continually check the buffer for new sockets with ‘buffer.get’. When the buffer is empty, ‘buffer.get’ will put the worker to sleep, and it will be woken up eventually when a new socket is placed into the buffer.

Last but not least, we need to create a number of workers, and set up our buffer.

server = TCPServer.new('127.0.0.1', '8080')

We are creating our buffer with a capacity of 100. This means that we can have 100 requests queued up until we have to start waiting for the buffer to be emptied. We are also creating 40 workers, and naming them so we can tell them apart. Lastly, we have our familiar ‘while socket = server.accept’ loop but instead of doing any processing, we simply put it into the buffer.

buffer = SynchronisedBuffer.new(100)

workers = []

for i in (1..40)
   workers[i] = Worker.new(buffer)
   workers[i][:name] = 'worker' + i.to_s  
end

while socket = server.accept
    buffer.put(socket)
end

All things going well, your web server should now be capable of handling many simultaneous requests. Leave a comment, email me, send me a tweet or a message on Facebook with any questions, criticisms or comments.

Ruby Day 7 Http Protocol

2010-06-29T00:00:00+12:00

We have created a simple server that responds with whatever text it received when we connect to it via TELNET. The uses for this are limited and so we want now to begin implementing the HyperText Transfer Protocol (HTTP) to enable us to show a web page. Being a standard, and universally adopted protocol, a web browser knows that if it uses HTTP to talk to a web server, they will both understand each other. HTTP defines a set of actions that a web server must carry out. These are:

OPTIONS - Returns information and options available for the server, or a specified resource. 'OPTIONS *' will tell you what the entire server is capable of, whereas 'OPTIONS filename' will tell you what it can do with that particular file.
GET - Returns the contents of the specified file or directory. 'GET index.html' will return the file index.html. The GET method is what we will be implementing.
HEAD - Works the same as GET, except the contents of the file or directory is not returned. Instead, meta-information is returned, such as the file-type or when it was last modified.
POST - Sending information along with a POST request will associate with the specified file. 'POST index.php?foo=bar' will send the value 'bar' to the file 'index.php', storing it under the name 'foo.'
PUT - PUT is the opposite of GET: it sends a file and a location, and the server will store that file in that location.
DELETE - You can remove a specified file with 'DELTE filename', however there is no guarantee that the server will actually do this.
TRACE - Lists the exact servers a request must go through to reach the destination server. This may be a proxy server, for example.
CONNECT - Not used in HTTP 1.1, CONNECT is reserved for future implementations that may wish to set up some form of secure communication.

(Paraphrased and adapted from W3C)

For a basic web server we will only need to implement the GET method, as that will at least allow us to send content to the browser. We’ll also want to send back a few status codes. Status codes let the browser determine what the server has done. The most prevalent example is 404 – File Not Found. Equally as important is 200 – OK, which is returned when the server can successfully carry out the request. Along with these 2, we will send back 400 – Bad Request if something other than a GET request is sent. The entire server code is below, and each step is explained in detail afterwards.

require 'socket'

The first 3 lines should look familiar. We’re setting the program to listen on TCP port 8080, and once a connection is made, we’re storing the received information in the ‘request’ variable.

server = TCPServer.new('127.0.0.1', '8080')

while socket = server.accept
    request = socket.readline

After this we’re checking if the request is a valid HTTP 1.1 GET request. We are using the regular expression ‘/GET . HTTP\/1.1/’ to check if the request fits the format ‘GET file HTTP/1.1’. ‘.’ represents any sequence of characters, of any length. While this isn’t perfect, it should suffice for now.

validGET = request.match(/GET .* HTTP\/1\.1/)

‘unless’ functions identically to ‘if not’, so the code inside will only be executed if validGET is not equal to something other than nil. validGET will be nil if the request string did not match the regular expression. Hence, if the request doesn’t match we are sending back ‘HTTP/1.1 400 Bad Request’, and closing the socket. ‘next’ stops Ruby from executing the remaining contents of the loop, and forces it to start the loop again.

    unless (validGET) 
        socket.puts "HTTP/1.1 400 Bad Request"
        socket.close
        next
    end

Here we are simply finding the name of the file they requested the server to load. The GET request consists of 3 parts, with a space in between each: ‘GET’ ‘filename’ ‘HTTP/1.1’. Thus, the second element in the array returned by split will be the filename. We prepend the filename with a ‘.’ to ensure that the resulting file path is relative to the current directory. If they request index.html, the request will come through as /index.html. By prepending ‘.’ the file becomes ./index.html which is relative to the current directory.

    file = request.split(' ')[1]
    file = '.' + file

Just as we checked for a correct GET request, we need to check if the requested file can actually be sent. File.exists? will return true if the given file exists on the computer. If not, we send back the famous 404 error.

    unless ( File.exists?(file) ) 
        socket.puts "HTTP/1.1 404 File Not Found"
        socket.close
        next
    end

Having got to this point in the code, we can say that we have received a correctly formed request for a file that we are able to send back. What we do next is part of the HTTP protocol, and is us telling the client that the request can indeed be processed. This is the 200 OK response. Following this, we send a ‘Connection: close’ to indicate that we have finished sending back this message. ‘Content-Type: text/html’ is the beginning of a new message, and it is telling the client that they can expect to receive an html file. If we want our web server to handle images, and other file types, we will have to add in logic to send back the correct content type. Today we will just stick with text/html.

    socket.puts "HTTP/1.1 200 OK\n"
    socket.puts "Connection: close\n"
    socket.puts "Content-Type: text/html\n"

Finally, we can send the contents of the html to the client. We simply open up the requested file and loop over each line in it. For each line, we send it back through the socket.

    File.open(file, 'r') { |f| 
        while (line = f.gets) 
            socket.puts line
        end
    }

    socket.close
end

To test the server, create an html file called test.html in the same directory as the Ruby file containing the above code. Run the program, and point your browser to ‘http://127.0.0.1:8080/test.html’. Hopefully you should see the contents of the html file rendered by the browser.

There you have it. A web server that handles HTTP 1.1 GET requests to successfully send back an HTML file, while also handling malformed requests, and requests for non-existent files. Granted, there is a large amount of functionality that we didn’t implement, including some important error checking, but hopefully it highlights the overall mechanisms of HTTP and web servers.

Ruby Day 6 Sockets And Tcp

2010-06-28T00:00:00+12:00

To kick off this web-server series we’ll create a server which, at it’s simplest, consists of just 3 lines of Ruby code. It can be called an echo server, and it does just that; it sends back to you whatever you send to it. Find a large, empty room, yell out “Ruby is awesome” and you’ll hear it repeated (this also works in a small room filled with Ruby developers). If you’re not fond of going outside (or socialising) however, you can use an echo server to achieve the same effect. Send the text “Ruby is awesome” to this echo server, and it will reply with the same message.

This simple example expresses a concept that is fundamental to how any server works. They listen for requests, do something, and (sometimes) send something back. A web server is listening to requests to load a particular page, and in response, it sends back the contents of that page. A mail server listens for requests to download mail messages, and in response it sends the latest emails. As most servers follow this basic pattern, standards have been developed and adopted that describe in what format the requests and responses should be. Warning: You are now entering the world of acronyms.

TCP, UDP, HTTP, STMP, NTS, FTP, ARP, SSH, IMAP, DNS, DHCP, IRC, to name but a few, are all protocols that describe requests and responses. Today we will be using the Transmission Control Protocol (TCP) to implement our simple echo server. To describe TCP, we can look at it’s simpler cousin: the Universal Datagram Packet (UDP). UDP is a protocol that specifies how data should be broken up into chunks, in order to send over a network. It also specifies how that data knows where it has to go. By pairing up a packet with an Internet Protocol (IP) address, network hardware will (all things going well) be able to send a packet it receives to the correct destination. The packet also contains the IP address of the sending machine, so that the receiver knows where to send its response. TCP does exactly what UDP does, except it does a few extra things to ensure that packets sent do actually arrive correctly. This includes error checking and correction, as well as ensuring that the sender isn’t sending more packets that the receiver can handle at once.

If we want to create an ‘echo’ server, we will need to listen for packets that are sent to us via the TCP protocol. When we receive one, we need to extract the message, and send it back again. Here’s the server:

require 'socket'
server = TCPServer.new('127.0.0.1', '8080')
socket = server.accept
socket.puts socket.readline

Firstly, we’re creating a new TCPServer object that will listen for requests at the IP address 127.0.0.1 and on port 8080. 127.0.0.1 is an IP address reserved to represent the local machine. If the operating system is asked to send a packet to this address, it realises that this is actually the same machine and so it is a good way of testing without requiring 2 computers. Port 8080 is a common port used for alternative web servers, which we will be creating eventually, so we may as well use it now. As the network card receives packets for the entire operating system, and not specific programs (mail client, chat client, etc.), a program must associate itself with a port number. The operating system can then look at the port number in the received packet and determine which program it should go to. Packets with port number 21 will be sent to your FTP client, port 110 packets will go to your mail client. We want packets with port 8080 to be sent to our Ruby program. To ensure there is no confusion, only one program can take one port number at a time, so if you receive an error about port 8080 being use, you may have another program already attached to it.

When you run this program, nothing will happen. The program will continue running forever and nothing will be displayed to the screen. Why doesn’t the program end? The answer is in the second line, server.accept. Accept is a method which causes the program to pause until the TCPServer object receives a packet. As soon as it does, it returns a Socket object which contains the received packet and information on the machine who sent it. This socket acts as our end of the connection created between us and them, a tube of the internet, if you will allow the analogy. Whatever is sent down this tube, we will eventually receive, and vice versa. Therefore, to create our echo, we come to the third line.

socket.puts socket.readline

We read the socket, which will contain the message they sent, and we use ‘puts’ to sent it straight back. To see this in action, we can use telnet to connect to our running server and send it a message.

telnet 127.0.0.1 8080

The screencast below will show it working.

A Simple TCP Echo Server in Ruby from Sam Dalton on Vimeo.

Voila! Hopefully you managed to get it working, and understood why it works as well. While this is a server in just 3 lines, it’s not particularly useful. Once someone connects, the programs ends and the server closes. To make it run indefinitely we will need some kind of infinite loop.

while socket = server.accept
    socket.puts socket.readline
    socket.close
end

In Ruby, every operation returns something, even variable assignment. The returned value will be equal to whatever value was set. So in this case, when server.accept detects a connection and returns a socket object, the while loop will see that an object was returned. An object is not regarded as false, and so the loops is entered. When the loop ends, server.accept is called once more, and the process repeats indefinitely. The program will never end because server.accept will always wait until it gets a connection, and hence will always return a non-nil value to the while loop condition. Just as with files, we also have to close the socket. We didn’t do this in the first example because the program exited straight afterwards so there wasn’t much need.

So there you have it, a 3 (or 4) line echo server that uses the TCP protocol.

Ruby Day 5 Building A Web Server

2010-06-27T00:00:00+12:00

Looking through the upcoming topics that I plan to cover, I noticed a connection: they all cover important aspects of a web server. Threads let us handle multiple requests simultaneously, queues let us store pending requests and a mutex ensures that it won’t become corrupted from multiple threads accessing it. We also need a way of logging requests, interpreting URLs, and certainly, we’d like to test it to be sure it all works. So instead of covering the remaining topics in my proposed order, I will instead create sections of the web server that each focus on a different part of the Ruby API. Not only will this provide a more hands-on and relevant example, but it will also serve as a basic tutorial on how the HTTP protocol and web servers work. The resulting server will by no means be any competition to existing Ruby servers such as Mongrel or Thin, but it will serve us well for the purposes of teaching.

First up will be sockets, as with out that we have no hope of communicating with anyone.

Ruby Day 4 Forwardable

2010-06-26T00:00:00+12:00

Today’s topic is a favourite of upper management: delegation. Delegation is when you make someone else do something for you, but you still get the credit for it. The Forwardable class in Ruby does much the same thing. It lets an object call methods on another object, but appearing to the user like the first object is actually doing the work. To see the advantage of forwardable, we can look at an example where it’s not used. We will create two classes, one called MarsExplorer and a second called MissionControl. We want to be able to control the mars robot in a more suitable, oxygen filled environment, which would be Mission Control.

class   MarsExplorer

   def forward
       @speed = 1
       puts @speed
   end 

   def backwards
       @speed = -1
       puts @speed
   end

   def stop
       @speed = 0
       puts @speed
   end
end

Our robot can move forwards, backwards, and it can stop. After each action we ask it to print out the speed, so we can keep track of when the methods are called.

class MissionControl

    def initialize
        @robot = MarsExplorer.new
    end

    def forward
        @robot.forward
    end 

    def backwards
        @robot.backwards
    end

    def stop
        @robot.stop
    end

end

Phew, that was a lot of typing! Too much, in my opinion. For each method in MarsExplorer, we have had to write the same method definition in MissionControl. If we ever change our robot class to have a turn method, or we change ‘stop’ to ‘brake’, then we will have quite a few changes to make in our MissionControl class. In addition, MissionControl is now very cluttered with method definitions that aren’t directly related to the class. It would be better if we could simply let MissionControl respond to the methods of MarsExplorer, in a simple and unobtrusive way. We can achieve this by using Forwardable, as you can see in our updated MissionControl.

require 'forwardable'

class MissionControl
    extend Forwardable

    def_delegators :@robot, :forward, :backwards, :stop
    def_delegator :@robot, :stop, :brake

    def initialize
        @robot = MarsExplorer.new
    end      
end

Notice that all of the method definitions are gone, with the exception of initialize, which hasn’t changed. Firstly, we have said that we want to make MissionControl extend Forwardable

extend Forwardable

This gives us access to the methods in Forwardable that follow, def_delegators and def_delegator. When we want to provide access to multiple methods, and we don’t mind calling them by the same name, we can use def_delegators. The syntax is explained below.

def_delegators :theObject, :method1, :method2, :method3

def_delegators :@robot, :forward, :backwards, :stop

We are saying that MissionControl should respond to the methods forward, backwards, and stop, and it should respond by calling the method of the same name on the robot object. Instead of 3 method definitions, we only need one line stating which object to delegate to, and what methods it should allow delegation of. If we want to change the apparent name of the method, we can use def_delegator.

def_delegator :@theObject, :methodName, :desiredMethodName

def_delegator :@robot, :stop, :brake

Now we can call the method ‘brake’ from MissionControl which will call the ‘stop’ method on our MarsExplorer. Neat! Let’s take it for a spin.

control = MissionControl.new
control.forward # 1
control.backwards # -1
control.stop # 0
control.brake # 0

Now, not all of us are engineers at NASA, so where might you use Forwardable in a real life? It is useful anywhere you find yourself writing methods similar to the ones in the first MissionControl implementation, simple wrapping of another object’s methods. It can also be used as a form of adapter. By defining your own class that delegates to another object, and only calling the methods on this class, the delegate object can be set to anything. For example, if you are writing an application that will be using a database, but you’re not sure if it will be MySQL, MSSQL, or PostgreSQL, you can create a wrapper class that has def_delegator set for each method you want to be able to call. Your implementation will use this wrapper class for all database access, with the method calls being delegated to whichever object you have set. If the underlying database object has different method names, you only need to update your def_delegator definitions, and all of your implementation code will remain the same. Here’s a contrived example to demonstrate this.

class MyDatabase
  extend Forwardabble
  
  def_delegator :@db, :newMySQL, :new
  
  def initialize
    @db = MySQL.new
  end
end	

db = MyDatabase.new
db.new

Suppose that there exists a ‘MySQL’ class, which provides the method ‘newMySQL’ to create a new database. If we build our implementation on the MyDatabase class instead of the MySQL class, we will be able to switch to PostgreSQL in the future by simply changing the def_delegator

def_delegator :@db :newPostgreSQL, :new

Our implementation code remains untouched, and we have only made one small change. While this example is entirely fictional, it shows how you can create a level of abstraction with Forwardable.

Ruby Day 3 Files

2010-06-25T00:00:00+12:00

The importance of files goes without saying, so let’s get straight into it.

Creating a File

This command will first look for a file in the current directory called myFile, creating it if it can’t be found. The ‘w’ tells ruby that we want to write to it. We can then write to the file by using the ‘write’ method.

myFile = File.open('myFile', 'w')
myFile.write('my awesome story')
myFile.close

When a file is opened the operating system records which program opened it. If we don’t tell the operating system that we have finished with the file, it will keep this information until the program exits. While this may not cause a problem in a simple example, it could lead to errors with a large number of files, or multiple programs accessing the same file. Calling ‘close’ on the file will take care of this. Whenever you open a file, you must close it when you are finished.

You can probably guess how to read from the file

myFile = File.open('myFile', 'r')
puts myFile.read
myFile.close# my awesome story

Identical to the first example, but instead of a ‘w’ for writing to the file, we use an ‘r’ for reading from the file. The letter ‘a’ can be used if we want to open a file, and append content to the end of it.

Reading From a File

For those of you coming from a PHP background, this code below should look familiar

$fp = fopen('myFile', 'r');
while (!eof($fp))
  fread($fp)
fclose($fp)

which simply opens a file and reads each line of it until it reaches the end. This is rather ugly, to be honest. We have to create a file pointer, check that we haven’t reached the end, read from it, and then close it. That’s 4 Steps, but we can do it in just 1 line in Ruby.

File.open('myFile', 'r') { |f| f.read }

By placing a block at the end, we can catch the opened file, and loop over each line in the file. When the block ends, Ruby will automatically close the file for us. The block method of file handling is preferred as you won’t run the risk of forgetting to close a file.

Peculiarities

If you try writing a new line to a file with the typical \n character, you may find that it does not work.

File.open('test', 'w') { |f| f.write('line 1 \n line 2') }
File.open('test', 'r') { |f| puts f.read }# line 1 \n line 2

The \n character is stored as a \ character followed by the letter n. Not quite what we wanted. So how can you create a new line in your file? If we use ‘puts’ instead of ‘write’, then Ruby will automatically place a real newline character at the end, once for each time we call puts.

File.open('test', 'w') do |f|
    f.puts('line 1')
    f.puts('line 2')
end
File.open('test', 'r') { |f| puts f.read }

line 1 line 2 The file clearly has ‘line 1’ and ‘line 2’ on separate lines so we have succeeded.

File Methods

If you are familiar with UNIX, bash in particular, then you will feel right at home with the File class. It offers just about every bash file operation, calling them by the same names and accepting the same arguments. For example, you can use ‘chmod’ and ‘chown’ to change permissions, or set who owns the file. Copy, move, delete, rename, stat, and more, are all available. Have a look at this UNIX tutorial if you want to learn about file operations.

End

Hopefully you now have some familiarity with the way Ruby handles files.

Update I clearly was not familiar enough with blocks at the time of writing this post, as David pointed out in the comments. I had an incorrect statement stating that a file handler passed into a block can only be used once. The post has been updated to reflect this.

Ruby Day 2 Array Enumerable

2010-06-24T00:00:00+12:00

Without the array, programming would be terribly difficult. To store 10 items, you would need 10 different variable names, and any more than this makes things hopelessly inefficient. The array lets us store a number of items under a single variable name. We can reference a specific item simply by saying which variable it’s stored in, and in what position it lies. Like in C and PHP, Ruby starts indexing at 0. myArray[0] will give you the first element. In C, if you tried to access myArray[-1] you would meet a nasty runtime error. In Ruby, however, a negative index is treated as an index relative to the end of the array. Thus -1 becomes the last element, -2 the second to last, and so on.

Ruby’s Array class has a lot of methods, so let’s get right in to it and explore some of what it has to offer.

Creation

puts [1, 2, 3] # 1 2 3

That’s all it takes! The square brackets let Ruby know that we want to create a new array, with the elements set to be the contents of the brackets. This syntax is an alias of the actual method, [].

puts Array.[](1,2,3) # 1 2 3

If we want to create an array, but we don’t have anything to store in it yet, we can use the ‘new’ method.

Array.new

If we know that we want to store n items, we can create the array with a capacity.

Array.new(10).size # 10

Size is a method of array which returns the number of elements. We created an empty array with space for 10 elements, and that is the size of the array.

Generation

The Array class gives us a powerful way of generating elements using a block.

even = Array.new(5) { |i| i + i }
puts even # 0 2 4 6 8

We have created an array called ‘even’ that contains the first 5 even numbers. Firstly, we stated that we wanted a new array of size 5

Array.new(5)

Secondly, we put a block at the end of it.

{ |i| i + i }

This could also be written as

do |i| i + i end

What is this voodoo magic, do I hear you ask? Well, the block is taking a number, i, and returning the sum of i with itself. The value of i that is passed is the index into each element created. Thus for an array of size 5, we will have the indices 0, 1, 2, 3, and 4. Working this out the long way gives us

even = Array.new(5)
even[0] = 0 + 0
even[1] = 1 + 1
even[2] = 2 + 2
even[3] = 3 + 3
even[4] = 4 + 4

Using a block lets us define each element programatically, based on the the index of that element. Nifty!

Interesting Methods

There are a lot of methods in the API, and most of them have some sample code to go with them. Therefore, I will look at some of the more interesting ones.

*

While this method can be used to duplicate an array n times, like so

['echo'] * 3 # echo echo echo

the more interesting usage is as a join operator. PHP offers the join() method which, given an array and a character will return a string consisting of each array element, followed by the given character. Ruby also has a join method, but the * operator is a great shortcut.

['c','s','v'] * ','# c,s,v

abbrev

The API states that this method ‘calculates the set of unambiguous abbreviations for the strings in self’. The example given, while sensible, doesn’t relate to a particularly useful application.

require 'abbrev'
%w{ car cone }.abbrev   
{ "ca" => "car", "car" => "car", "co" => "cone", "con" => "cone", "cone" => "cone" }

Note that this method isn’t part of the core API, and so it needs to be explicitly included

So why is this useful? The main application I can see for this is auto-completion. Given a set of names or commands, what are the fewest characters I can type to identify one in particular? Consider a lazy person using twitter. If I want to send a message someone, I don’t really want to type out their whole user id. Given an array of people I follow, abbrev can generate a list of possible abbreviations such that each abbreviation will only reference one person.

tweeps = ['jeff', 'steve', 'frank', 'james', 'stephen']
p tweeps.abbrev
{"frank"=>"frank",
 "stephe"=>"stephen",
 "jam"=>"james",
 "jef"=>"jeff",
 "steve"=>"steve",
 "step"=>"stephen",
 "ja"=>"james",
 "fr"=>"frank",
 "je"=>"jeff",
 "jame"=>"james",
 "f"=>"frank",
 "stephen"=>"stephen",
 "jeff"=>"jeff",
 "fran"=>"frank",
 "stev"=>"steve",
 "james"=>"james",
 "steph"=>"stephen",
 "fra"=>"frank"}

Steve and Stephen have the same first 3 characters, and so you can see that the shortest abbreviation for each is stev and step. You may have noticed that I used ‘p’ and not ‘puts’. This is because abbrev doesn’t return an array, but a hash of ‘abbreviation’ => ‘originalString’ pairs.

collect

This method is similar to the element generation described above. Given an array, place a block at the end which takes each element of the array. In this case, instead of the index being passed in, the actual element is passed in. The resulting value can either be stored back in the array, or placed into a new array.

numbers = [1,2,3,4,5,6,7,8,9]
p numbers.collect { |x| if (x % 2 == 0); x; end }.compact# [2, 4, 6, 8]

By checking if each number is divisible by 2, we can create a new array containing only the even numbers. ‘compact’ is used to remove elements that are ‘nil’. As collect will create a new array of the same size, when the number is odd no value will be returned and so that element will be empty. To overwrite the array instead of creating a new one, use the exclamation mark.

Array.collect!

uniq

By using uniq you can quickly make your array behave like a set.

p [1,1,2,2,3,4].uniq# [1, 2, 3, 4]

Enumerable

Now that we have a basic understanding of Array, let’s take a look at it’s cousin class: Enumerable. Enumerable is all about iterating. The collect method that we have just seen is an example of an Enumerable function. Collect lets you perform some operation on each element of an array. Sorting is another example of what you can do with Enumerable. The key difference between Array and Enumerable however, is that while you may create an array, you don’t create an enumerable. Rather, by defining certain methods, a class can behave like an enumerable object. Before we look at how to do that, let’s see some of the methods available.

inject

The inject method is similar to collect, but an extra value is sent to the block at each iteration containing the result of the previous iteration. This lets us compute the factorial of a number in a neat way:

p (1..4).inject { |sum, n| sum + n }# 10
4! = 10.

sort

When we have an array of elements, we quite often want to sort them. It they are names, we’d like to sort them alphabetically; if they are numbers, well, numerically would make sense. The sort method gives us great flexibility when it comes to sorting by letting us define the sorting comparator as a block.

p %w{a c b}.sort { |a, b| a < => b }# ["a", "b", "c"]

%w{a c b} is a short way of saying [‘a’, ‘c’, ‘b’]. If each element of your array is a single word, then the shorthand will be quicker as you don’t have to worry about typing in quotes. So, given our array, [‘a’, ‘c’, ‘b’] and our block, { |a,b| a < => b }, we see that they have become alphabetically sorted. The block operates on a pair of values, and compares them with the < => operator. < => returns -1 if the left value is lesser, 0 if they are equal, and +1 if the left value is greater. If we want to sort the array in reverse order, we simply swap the order of the values in our block.

p %w{a c b}.sort { |b, a| a < => b }# ["c", "b", "a"]

Custom Enumerable Classes

If you have your own class, you may want to do things with it as if it were an array. For example, let’s say we have a BookShelf class that maintains records of various books we own. One thing we may like to do is print out our books sorted by their author. If we include Enumerable into our BookShelf class, and define the method ‘each’, we will be able to write code that might look like the following.

p bookShelf.sort { |a,b| a['author'] < => b['author'] }
    [{"title"=>"Great Expectations", "author"=>"Dickens, C."},
    {"title"=>"Linchpin", "author"=>"Godin, S."},
    {"title"=>"Frankenstien", "author"=>"Shelly, M."}]

As you can see, we have an array of hashes, containing an author and a title. They are ordered by the author’s name. Here’s the BookShelf class that lets us do this.

class BookShelf
    include Enumerable

    def initialize
        @titles = []
        @authors = []
    end

    def add bookTitle, bookAuthor
        @titles.push bookTitle
        @authors.push bookAuthor
    end

   def each
       @authors.each_with_index do |author, index| yield 'author' => author, 'title' => @titles.fetch(index)  end
   end    
end

We are including the Enumerable class, which means that if we define the method, ‘each’, we can call any Enumerable method on the BookShelf class.

The each method is defined as

@authors.each_with_index do |author, index| yield 'author' => author, 'title' => @titles.fetch(index)  end

In English, this says ‘For each author, get me the name of the author and the index. Then I’m going to stop (yield) and return the name of the author, and the title of the book that appears in the same position in the titles array’. As you can see, it is ‘yield’ that is doing the magic here. It essentially stops the loop (each_with_index) and spits out the result of the statements that follow it.

If you want to try it out yourself, you could add some books to the BookShelf like so

bookShelf = BookShelf.new
bookShelf.add 'Great Expectations', 'Dickens, C.'
bookShelf.add 'Frankenstien', 'Shelly, M.'
bookShelf.add 'Linchpin', 'Godin, S.'

End

Phew! That was a lot to take in. Hopefully you’ve seen some of the neat methods that the Array class provides, particularly ‘abbrev’, as well as how we can make our own classes behave like Enumerable objects. Let me know if you have any questions, comments, and especially criticisms.

Ruby Day 1 Date

2010-06-23T00:00:00+12:00

Welcome to the first instalment of 0 to Ruby in 30 days. Today will cover a simple, but useful class: Date.

Most web applications that let users contribute content will likely require the use of date and time functionality. Just consider how useful would a website be that helps people coordinate meetings if there was no way of specifying the time? Also consider how difficult it would be to coordinate a global event, with people in different time zones, if everyone saw the time in GMT +0. Confusion (and possibly hilarity) would undoubtably ensue. Clearly, we need a way to handle dates and times.

Slightly strange motivation aside, let’s get started. The format of this tutorial is as follows: I have opened the API reference for the Date class and will be going through it, testing methods of interest. I have not used the date class before and so at each step, I am guessing the usage based on my limited experience of Ruby gained from _Why’s (Poignant) Guide. You can consider this tutorial, then, to be an extra reference to the API page, with examples of usage and results of various important methods.

One of the major differences between the Ruby date library and the PHP date library is the epoch used. PHP uses the date ‘January 1 1970 00:00:00 GMT’, the beginning of UNIX, as a reference point. Time is defined as the number of seconds after or before this time. Ruby uses January 1, 4713 BC as its reference point. This is known as the Julian Date, with any date defined by the number of days before or after this reference point. But this is a programming tutorial, not a history lesson, so let’s see some code.

Great! We can create a new date object by providing the year, month and day. (Date.new and Date::new seems to function identically). By printing it to the console with puts, we can see that, indeed, this Date object contains the date we specified.

d = Date.new(2010, 6, 22)   
puts d                      # 2010-06-22

Lets get specific parts of the date

puts d.month    # 6
puts d.day      # 22
puts d.year     # 2010

cwday The method ‘cwday’ gives the Commercial day of the week Today is Tuesday, the 2nd day of the working week. Sunday is 7, the last day of the week.

puts d.cwday    # 2

What happens when we create an invalid date?

d = Date.new(0325, 14, 45)  # ArgumentError: invalid date

We have a basic understanding, so let’s start looking at some of the class methods

day_fraction_to_time

puts Date.day_fraction_to_time(0.75) # 18003.72423446248771e-17

This is equal to 18 hours, 00 minutes, 03 seconds, 0.72.. milliseconds

time_to_day_fraction

puts Date.time_to_day_fraction(18,00,03)    # 21601/28800

21601/28800 turns out to be 0.75, the fraction we passed in to day_fraction_to_time.

today

puts Date.today # 2010-06-22

valid_time

puts Date.valid_time?(25,0,0)   # nil

25 hours is not a valid time, hence the result is nil.

– The – operator should return a date that is n days earlier than the current one

d = Date.today
puts d - 3  # 2010-03-19

+ adds days in the same way

<< The << operator should return a date that is n months earlier than the current one

d = Date.today
puts d < < 3    # 2010-03-22

>> adds months in the same way

leap? Is this a leap year?

d = Date.today
puts d.leap?    # false

go back 2 years

d = d < < 24    
puts d.leap?    # true

next The next day.

d = Date.today
puts d.next     # 2010-06-23

So, that’s the Date class. Pretty nifty, and very intuitive. But what good is the date without the time? Enter: DateTime

t = DateTime.new
puts t  # -4712-01-01T00:00:00+00:00

Looks like that wasn’t sufficient, but note the term ‘-4712’. This was the year Ruby set to be the reference time.

t = DateTime.now
puts t  # 2010-06-22T13:56:13+12:00

There we go 2010-06-22 is the date T demarcates the time 13:56:13 is the current time +12:00 is the time zone. I live in New Zealand which is GMT +12

Let’s just check that the timezone is correct

puts t.zone # +12:00

Now we’ve seen some of the various methods, lets put it to use in a practical example. If you’re developing a website, chances are you’ll be dealing with people from all over the world. This means you’ll be dealing with different time zones, and so every time you work with a date and time you must ensure you have the timezone correct. Let’s imagine you’re developing ‘Qwitter’, a social networking service designed for people who are trying to ween themselves off Twitter. When someone updates their status, their followers want to see the time in their own timezone, so lets define a function to convert a time from a certain timezone, into another

def time_relative_to_timezone (oldTime, newTimeZone)
    oldTimeZone = time.offset    
    oldTimeZone *= 24

    timeZoneDifference = oldTimeZone - newTimeZone

    time - (timeZoneDifference / 24)
end

We can call it like so:

puts time_relative_to_timezone(DateTime.now, -5)

So, what’s going on there? Firstly, we are taking the given time and finding it’s timezone. The ‘offset’ method returns the timezone as a fraction of 24 hours. 0.5 would be 12 out of 24, for a timezone difference of +12. Thus, if we multiply this fraction by 24, we get the timezone offset in hours.

We then find the difference between this timezone, and the supplied one. We will assume for arguments sake (no pun intended) that the supplied timezone is already an offset given in hours. By subtracting them, we find how different they are.

Lastly, recall that when used with a DateTime object, the – operator subtracts a number of days. If we subtract our time zone difference as a fraction of 24 hours (1 day), we will end up with a new DateTime object with a time that has been adjusted.

Obviously we would need to include error handling for invalid arguments, but this has been ocluded for clarity.

End Hopefully this tutorial has shed some light on the Ruby Date, and DateTime classes. If you have any questions, comments, criticisms, praise, please leave a comment or send me an email.

0 To Ruby In 30 Days

2010-06-22T00:00:00+12:00

In an effort to become competent with the Ruby on Rails framework, I’ve decided to write up 1 tutorial per day for the next 30 days. Coming from a background in PHP, I will also try to add commentary on similarities/diffe:rences between the two languages. While not all of the topics I intend to cover are relevant to the Rails framework, and web development in general, the experience gained from learning them will hopefully be beneficial by increasing familiarity with the language and its paradigms. I hope that these tutorials become useful to anyone else who is wanting to learn Ruby/Ruby on Rails.

The topics I will cover are:

Array Enumerable

pipe
tty

Thread

Mutex
Process

Marshall Logger Observable Queue Singleton Zlib Test URI Yaml ERB Rails

ActiveRecord
    Migrations
    Validations
    Associations
ActionMailer
ActionView
    Layouts
    View Helpers
ActionController
    Routing </pre>

(Some topics, such as Array and Enumerable, may be combined into a single tutorial as they are quite similar.)

These tutorials are designed for someone with reasonable programming experience. If you’ve been using a language such as PHP for web development, but you feel like something is missing, then this is for you. Before starting the tutorials here, I strongly suggest you sit down in a comfortable chair, make a nice drink, and read through . It is unlike any programming guide you have ever read, and probably will ever read.

Subversion For Code Management

2010-06-18T00:00:00+12:00

Gone are the days of a handful of simple html pages to make up a website. Projects can consist of many, many files and, especially with multiple people involved, this can become difficult to manage. From graphics to documentation to source code, a version control system (VCS) is an easy way to manage your project and ensure that nothing important is ever lost. This tutorial will take you through the basics of version control, setting up Subversion server on your PC, and lastly we’ll look at how you keep track of your files.

What is a Version Control System (VCS)?

If you’ve ever asked yourself the question “how can I get back this change that I’ve overwritten”, “this program used to work better before we added that feature” or, “can you send me the latest copy of your code before I make changes”, then a VCS is your answer to them.

A VCS consists of a central repository which stores all of your project files. It keeps track of the files that are added and will store of all the changes that are made to them. This means that when you update a file, any previously saved work can be recovered by undoing particular updates. Multiple people can save to this same repository and so the VCS also takes care (with some user help, as we shall see) of managing files to ensure that no one can overwrite anyone elses work. Essentially, a VCS is turbo-charged undo/redo history for your computer.

Before we get into setting up a VCS for ourselves, let’s take a look at a common problem with VCS and how it’s solved. Called the “Lost Update” problem, it occurs when 2 people are editing the same file at the same time and both want to save their changes. The image below is from the Subversion Book, which is the user guide for a common VCS.

Obviously we don’t want something like this happening and fortunately, VCS provide us with 2 methods of ensuring that it doesn’t. The first method is called “Locking” (also known as “Reserved Checkouts”) and involves ‘locking’ a file before editing it. ‘Locking’ the file means that no one else is able to edit the file until it is released again. This ensures that changes cannot be overwritten as there is only ever one person editing the file at a time. While this can cause problems, it offers complete assurance that work will not be overwritten.

The second method is called “Merging” (also known as “Unreserved Checkouts”) and involves merging copies of work when the same file has been edited. If the parts of the file that were edited are in different locations then the two files can automatically be merged by the system. If ‘overlap’ between edited parts occurs however then it is up to the user to manually combine the files. The code examples below illustrates two files that are to be merged both with, and without overlap.

Edit without overlap

Edit 1:

$edit1 = "The cat sat";

rest of the code

Edit 2:

rest of the code

$edit2 = "on the mat";

As both edits are in different areas of the code, they can be merged with no conflicts.

$edit1 = "The cat sat";

rest of the code

$edit2 = "on the mat";

Edit with overlap

Edit 1:

$edit1 = "The cat sat";

Edit 2:

$edit1 =  "on the mat";

In this case, the same line has been changed in both edits and so they cannot be automatically merged. Differencing tools can be used to find out what exactly is different between the files and so the desired parts of each can be chosen.

Now that you have a basic understanding of what a VCS is and how they can help, let’s look at setting one up and using it

Installing Subversion

There are 2 big version control systems (VCS) in use today, Concurrent Versions System (CVS) and Subversion (SVN). We’ll be using SVN because CVS seems to be going out of fashion. SVN itself does not have a graphical user interface and so to make things more familiar, we’ll be using a program called TortoiseSVN to interact with it.

If you have Apache 2.0 or greater installed then you can grab this release, here If you don’t have Apache then you can get a standalone version from Slik SVN, here. This is the version that we’ll be using in this tutorial.

Also download TortoiseSVN from here

Once your chosen version of SVN and TortoiseSVN have been downloaded, double click to start installing. Choose the typical install for both programs.

That’s all there is to it! You now have a working VCS on your system and a great graphical interface to go with it. There is no program to start up to use SVN as TortoiseSVN integrates straight into the context (right-click) menu in windows explorer. Now let’s get right into using it.

Using SVN

First of all we need to create a repository. This is where all of the saved project files and version information is kept. Create a new folder on your system and call it ‘MyProject’. Right click on this folder and click TortoiseSVN –> Create repository here.

Now we need to do what is called “checking out” the repository. This will create a working directory for us to add files and make changes. Think of this as downloding files from your FTP server in order to work on them locally. Right click on your desktop and click TortoiseSVN –> Checkout repository. A dialog box will open asking for the location of your repository. Enter in the path to where you created this. You can also specify what you would like to call your checked out folder. Click OK when you’re done. You should now see a newly created folder with a green tick on it. This green tick is a TortoiseSVN icon that means the contents of your folder have the latest versions from the repository.

You may be wondering why we have to create our repository in one place, and check it out in another. The repository is not meant to be edited manually. It contains a number of files and folders which SVN uses to keep track of the changes. You could store this repository anywhere (including on another computer) and check it out in the same way.

Now let’s create a file. We’ll deal with a very basic example to demonstrate the key actions so create a text file called numbers.txt in the checked out folder, and paste in the following text

Now that we’ve made some changes, we need to let SVN know about it. Right click in the folder and click on SVN Commit.

A dialog box will open asking you for a message to save along with the file. It is good practice to always comment on each commit you make. This will allow you to easily see what changes you’ve made and is especially helpful when you have multiple people working from the same repository. Enter in a quick sentance about what the file is (TortoiseSVN will give you hints for filenames as you type), tick the check box next to the file and then click OK.

Your file has now been saved to the repository and a dialog box will open confirming this. Note that it says “Completed at revision 1”, each time you commit the revision number will update.

Now we’re able to keep track of our own changes, but what if we want to bring someone else into the project?. Right click on the desktop and checkout the repository again giving it the name ‘DifferentUser’. Notice how the icon is not a green tick this time but a red exclamation mark. This means that our folder is not in sync with the repository as we have added a file from the first checkout. Go into this new folder, right click and click SVN Update.

You should now see the numbers.txt file in this folder. Let’s make a change to this file. Open it up and replace the number 5 with the number 6. Let’s also change the file in the first checkout we created, MyProject. Open the file and add the number 6 after the number 5 on a new line. We’ve now created a situation where 2 users have made different changes to the same file at the same time. Commit the changes in MyProject and the folder icon should have a green tick. Now go into the second folder and try to commit. You will get a dialog box pop up informing you that there is a conflict.

We can now use TortoiseSVN to help us resolve this conflict. Right click in the folder and click TortoiseSVN –> Resolve conflicts. This brings up the TortoiseMerge window with which we can do this. The file on the left labelled ‘Theirs’ is what is in the repository. This should contain the file that we committed from MyProject. The file on the right labelled ‘Mine’ is what we’re trying to commit. The area below labeled ‘Merged’ is what will be committed once we’ve made our changes. There are various options such as appending the files together, taking just one file or manually edit changes. We will manually edit the file in the ‘Merged’ window and take this to be our new version.

In the ‘Merged’ section, edit it so that it looks like the code below

Now we can save our changes. We have successfully resolved the conflict and both folders should have green ticks.

While this example was rather trivial, you should now have a grasp on what version control is and how you can use SVN to help manage your projects.

Juicy Fresh Marketing

2010-06-16T00:00:00+12:00

It occurred to me today why Jucy Car Rentals works. They have a brand that’s cool, saucy, new, that people will want to show off. No one cares if you hired your car from Avis, or Budget, or another boring company. But if it’s jucy, someone’s going to ask about it. (If you haven’t seen, they put suggestively posing women on the side of their campervans.)

This means that they can put advertising on the cars they rent out. It lets the renter tell a story, so why would they mind?

They can spend less on traditional (read: boring) advertising because they’re not renting cars, they’re renting mobile billboards, and so they can charge you less. Win-win.

Just like you can’t out Amazon Amazon, you can’t out Avis Avis. But if you’re a backpacker, both the price point and the image is appealing, and so the choice is obvious.

They’ve found their remarkable.

Linchpin Meetup Day

2010-06-15T00:00:00+12:00

June 14th was (and still is, for most of the world at the time of writing) World Linchpin Day.

This was an event started by Seth Godin on meetup.com for people who are interested in his latest book, Linchpin. It’s a book that asks the question “are you indispensable?”. It challenges you to think about what you do each day, and see how it adds value or meaning to your life. Manual labour can be difficult, but ‘emotional labour’ is even harder. Godin talks about how it is essential to put emotional labour into the work we do, to create art and give it away, and to give with no expectation of reciprocation.

The meetup page for Auckland, New Zealand, showed 57 people who were interested. As we could link our Facebook accounts, people were able to read about (stalk) each other before hand. This meant that when I arrived, not only did I not have to give my name, but even my job and background were common knowledge! This really shows the power of social networking, and also how careful you have to be with what you share. I digress, however.

I had come with expectations about discussing the book itself, and ideas related to it, but it turns out the many people hadn’t read it and were there for the networking, meeting people. That turned out to be pretty amazing. All of the people there were operating on a similar page. Entrepreneurship, social ventures, making connections, all things to do with people. This made for lot of interesting and stimulating conversation, sharing our ideas and backgrounds, and getting to know people.

After the swapping of many business cards, it was eventually time to leave however. Fortunately, being a meetup for Linchpins, the initiative had been taken to put together some resources to keep and grow the connections made. Thanks Alan!

Being a global event, there were many meetups happening simultaneously. One was a meetup in Japan that was running on Junto. It was great to jump in there and meet some great people, and share my experience from the meetup here.

There are many Linchpins in the world, and many who are but just don’t know it yet. I think, if anything, this event has highlighted just how prolific the desire to do meaningful work is. It also shows that sometimes all people need is a word to describe something, and then everything moves forward from there.

Getting Things Done The Linchpin Way

2010-06-11T00:00:00+12:00

The old kind of linchpin I wrote previously about how competitions are a great way of getting things done. There’s a new method that’s come up recently, and it’s from Seth Godin’s book, Linchpin. (If you haven’t read it yet, stop reading this, buy it, start reading it, and hold on tight).

All you need is the desire to make a difference, and then the shipping will follow. Yukie is starting this on her blog, We Are The Linchpins by encouraging people to show themselves. By creating an environment for people to make a difference, it’s easy to get things done. You feel compelled to work because the work you’re doing actually means something.

By joining the Linchpin community you become surrounded by people who strive to make a difference, who live to ship, and who love what they do. This keeps you inspired and motivated, and helps you aspire to be one as well.

If an excuse is what you need to get started, then have a look for a Linchpin meet up near you for Linchpin Meetup Day on June 14.

960 By 640

2010-06-09T00:00:00+12:00

That’s the resolution of the newly announced iPhone 4

5 years ago we were complaining that it was difficult to design websites that would fit old monitors. The old standard of 800x600 was simply too small to do anything useful with. The table layout, and then the fluid CSS layout helped us cope, but it was still hard to imagine that most people were seeing it through that size a window. Even today it’s still hard to imagine people viewing content through such a resolution, because now they’re doing it on a phone.

Multi-tasking, video calling, and HD video are all amazing features, but this new screen is what really amazes me.

We’ve gone from 480,000 pixels on a 15 inch, multiple kilogram weighing, ungainly behemoth of a screen, to over 600,000 pixels on something the size and weight of a pack of cards. Wow.

Bring on the 1600 x 1050 smartphone.

Dropshorty File Sharing

2010-06-07T00:00:00+12:00

Here’s a quick demo of an awesome OSX app, DropShorty

Once you’ve downloaded it, you need to enter you Dropbox user ID. You can find this on your Dropbox folder online by going in to your shared folder, right clicking on a file and selecting “Copy public link”. You then need to copy the number in the url, as shown below:

Once you’ve set this (in Preferences ? + ,) , you can then connect it to to one of your URL shortening accounts.

Note: The link didn’t load first time round because Dropbox was trying to sync 5GB of files I had moved recently.

The Limits Of Social Networks

2010-06-04T00:00:00+12:00

When we talk one-on-one and face-to-face with someone, there’s a chance they might not like what you say. They could disagree, they could be offended, they could misunderstand you. There’s a risk of embarrassment. They know what you said, and you know that they know. There’s no escaping it.

With social networks, there is less risk. If you say something then many people will see it. Some will still disagree, but it’s also likely that someone will agree and like what you have to say. From behind your computer, you won’t know if someone disagrees until they say so, and so it becomes easier to say things. (Besides, if you’re on Facebook, you can always delete their comments).

There’s also less meaning. We lose the raw nature of conversations, and we lose the feedback that drives us to improve. It is also open to all kinds of misinterpretation. There’s a limited sense of sarcasm, and other attitudes, that we effortlessly portray and interpret through intonation when we speak.

The person who can fit body language, vocal pitch, intonation, facial expression, appearance, pace of speaking, volume, as well as the words we say, into 140 characters will well and truly change the world.

NB just look at how many more text comments there are on youtube than video responses. Showing yourself is scary.

10 Reasons To Buy Things

2010-06-04T00:00:00+12:00

(By ‘Things’ I’m mainly referring to software, music, movies and books.) Pirate Bay For Buying

You're rewarding the creator of the content for their invested time and effort.
You're likely to get some physical packaging, especially if it's music. This is great if you like collecting things.
It's the right thing to do, really.
You might get freebies, like free upgrades or bonus tracks, or an author's signature.
You'll likely get a support service if you're buying software.
If you have a problem with what you've bought, you can ask the creator about it and get help.
You can get credit card rewards points.
It won't come with any nasty spyware or malicious parts.
Some things simply aren't available anywhere other than from the original seller.
It's not against the law.

For Pirating

If it's music, chances are the artist gets next to nothing. You're feeding blood sucking, soulless record companies.
Packaging just takes up space. Digital copies are the way to go.
It's not quite the right thing to do, but who's going to notice?
The entire thing is a freebie.
The given trial period or trial set of features might not be enough to convince you to buy it. After all, you don't want to buy something that's not quite right.
If you have a problem with what you downloaded, you can just find another version elsewhere.
I don't need a credit card.. the interest is outrageous!
It won't come with any nasty DRM.
Some things simply aren't available anywhere other than on torrent sites.
It's not against the law in Russia.

Sam Likes This Blog Post

2010-06-02T00:00:00+12:00

A look down my Facebook news feed shows me things like “Steve likes ‘I wish i was little again, when the hardest choice was picking a crayon’ ”. Other than liking people’s business pages, or events, or photos, people are starting to like sentences.

It turns out there are websites dedicated to listing sentences and phrases that people can ‘like’, with the release of Facebook’s new Open Graph API.

One example is superlike. Another is chelick

Instead of updating my Facebook status, I can ‘like’ a series of phrases that other people have said. Due to Facebook’s new settings, these likes will be publicly available.

We are so used to relying on content on the web, that the act of liking even everyday phrases and comments is becoming popular. While it existed before with the many Facebook pages that were available, now it is even simpler. This is similar to re-tweeting on twitter, in a way, except the phrases you ‘like’ are listed anonymously so you are not celebrating what a person has said, just the content of it.

Will the next step be personality archetypes that we subscribe to, which automatically publish content in line with what we would say? It’s an extreme example, and not likely, I simply find it interesting to watch the trends that appear on social networks.

Getting Things Done The Microsoft Way

2010-06-02T00:00:00+12:00

I recently took park in the world’s largest technology competition, the Microsoft Imagine Cup. It’s a competition that challenges students to use technology to solve the biggest problems of the world today. Quite a daunting and seemingly unsurmountble task, but clearly it seemed possible to the 300,000 students who entered worldwide.

Throughout the 6 months I was involved in the competition I learned many things, how to present well, how to gather feedback on ideas, how to work in a new team, etcetera. But above all else, the most important thing it taught me was how to ship.

(If any of you follow Seth Godin, you’ll know what I’m talking about.)

The imagine cup was an excuse to do something. We all have ideas, and we’d love to seem them come to fruition. But doesn’t that take time, effort (and super powers)? Of course it does! But that’s the easy part.

We put hours and dollars into writing that work report, or doing our tax returns. Boring things that we don’t want to do, but have to. Clearly we’re good at putting in time and effort when we must.

So the hard part is getting the effort to go where we want it to. If we take that first step, and step onto the right platform however, it’s easy. A competition was my (and my team’s) step and platform, and it was fantastic.

As a result, the idea made it to the top 4 in New Zealand, has now transformed into a real business, and has opened up many opportunities for us to invest time and effort into something worthwhile.

So if you’re stuck with an idea and don’t know how or where to start, find a competition that is related to it (there’s bound to be one). This will give you the motivation, the environment, and an excuse, to start doing what you really want to do.

Besides, not only will your idea be coming to life, but you may even win something for it.

Japanese Learning Program

2010-05-28T00:00:00+12:00

If anyone is trying to study Japanese, and wants a list of english:hiragana:kanji readings of common words from which to make a language training program, this may help: Japanese Words

\ denotes a new definition the english, hiragana and kanji readings are separated with a colon :

\to be born:?????:?????
\to die:?????:????

I wrote it for final exams for a high school Japanese course. I’d like to think that it helped but really it was just a way to justify writing code while I should have been studying.

A New Kind Of Labrat

2010-05-28T00:00:00+12:00

The study of robotics is moving from exorbitantly funded research centre’s and organizations to the humble garage workshop. Hobbyist robotics is a growing field and is being adopted all over the world. LabRat is just one of the many platforms that is allowing this to happen.

Designed to fit inside the body of a typical computer mouse, LabRat gives students, researchers and hobbyists a low-cost, but in depth look into some of the core fundamentals of robotics. From between $120 and $250 (USD) the LabRat provides an 8MHz 8bit AVR processor, optical sensor, touch sensors, Infrared (IR) sensors, IEEE 802.15.4 wireless extensibility, basic IO pins, Gumstix processor extensibility and Pulse Width Modulated (PWM) driving motors that fit neatly on to an open-sourced Printed Circuit Board (PCB). The development environment for the robot is an open-sourced C based library that gives the user exact control over the robot, allowing complete freedom of creativity.

The open-source nature of the device has accelerated its own design as researchers and students were able to identify bugs and flaws which LabRat could take on board straight away. It has also led to some interesting projects with the device including autonomous charging and wireless control through a playstation controller.

The majority of this work has come from the Missouri University of Science and Technology, where the LabRat was used as the basis of an introductory robotics course. The device was given to the university before its final release, giving the company time to act on any feedback they received.

The robot has a unique selling point in its low price and open sourced nature. As it is the only hobby level robot with open-sourced hardware, it is the obvious choice for anyone interested in extending the basic functionalities of the device, which cleverly allies with the market they are targeting. Researchers want a device they can customise to suit their own research areas or courses; students want a device they can take apart and see precisely how it works; hobbyists want a device that they can extend, hack and use in other projects; and certainly all groups want a device that doesn’t cost a lot of money.

The report that was released at the 2009 International Conference on Intelligent Robots and Systems portrays precisely these attributes, providing a perfect sales pitch for those who read it. The background on robotics it provides gives the reader a very clear context in which to read the remainder of the report, and serves to highlight its key benefits and features.

(note: this was an article review written for a robotics class)

The Influence Wall

2010-05-24T00:00:00+12:00

It used to be, that if you wanted information on something or someone, you would go to a website about it. Once you arrived, a large number of influencing factors fought for your attention to try and make you stay on the page. The website’s appearance, colour scheme, spacing, vibe; its quality of content, user interface experience, even the domain name, had an affect on the site’s credibility.

Then RSS came along and people started syndicating their content into a single stream. The interface and other usability aspects were handled by your RSS reader, leaving only the content to influence you. This placed the focus on the content. All you had to judge someone by was their words.

Fortunately, however, a person who was publishing content had a lot of words to use. With the advent of twitter, people have just 140 characters to influence someone.

Profile pictures and twitter backgrounds can influence to an extent, but with so many users using a twitter client, the 140 characters is all there is. (perhaps the number of followers you have is also a factor)

How do we write such compelling content in such a confined space? Well, we must write snippets. Lots of them. Single thoughts that, over time, build up to a personality. We can then recommend these personalities to people. Find one person who will listen to you, make them happy, and they will talk about you.

What else can one do?

Reflection Based Data Layer On Zf 1 10

2010-05-19T00:00:00+12:00

Working on a new project, I came up with this simple data layer. It uses Zend_Db_Tables to do most of the heavy lifting, but the added layer makes persisting objects a breeze.

Here’s how you use it

$store = new Application_Model_Persistence();
$user = $store->fetch('User', 'id', 5);
echo $user->name;

Not much going on, really. The fetch method takes 3 arguments:

<ul>
  <li>The object you want</li>

  <li>The key you want to search on</li>

  <li>The value to search for the specified key</li>

</ul>

The object name matches the table name in the database. The fetch method is looking for a model class of the name Application_Model_User, instantiating it, and then querying the database with a query select * where id = 5, from the table Zend_Db_DbTable_User.

The Application_Model_User class is then populated with the returned database data, and subsequently returned to the user.

A ‘persist’ method does the opposite. Given an object, it either creates a new entry in the database or updates an existing one.

At the moment it can only handle primary key selects – multiple rowsets will probably cause it to crash.

It does really simplify the actual implementation however, as you only need a single object to fetch, update, or create all of your data objects.

Calling the fetch method 1000 times takes roughly 6 times longer than a raw sql query, but it’s vastly quicker and simpler to implement with, and more convenient as the data is returned as an object.

Here’s the method For each type of model that is fetched, the dbTable object is stored here for quick access. Speeds up subsequent accesses in a single request by a factor of 6 or more.

  
protected $tableCache = array();

Create an Application_Model_Template object of the given type, by selecting from the database querying on the given key with the given value. For example fetch me a location which has an id of 13 fetch(‘Location’, ‘id’, 13); @return Application_Model_Template @param string $type @param string $key @param mixed $value @throws Exception – Class name does not match DbTable name

public function fetch($type, $key, $value) {
    $dbTable = $this->getDbTable($type);

    if (!$dbTable instanceof Zend_Db_Table_Abstract) {
  throw new Exception('Class name does not match DbTable name');
    }		
    
    $select = $dbTable->select();
    $select->where($key . ' = ?', $value);
    $row = $dbTable->fetchRow($select);
    
    try {
        $class = 'Application_Model_' . $type;
  $obj = new $class($row->toArray());
    } catch (Exception $e) {
  echo $e;
  return null;
    }	
  
    return $obj;
}

Return an Application_Model_DbTable_dbTableName object for the given dbTableName. The tableCache is checked first using dbTableName as the key; if not found it is created, and stored in the cache. @return Application_Model_DbTable @param string $dbTableName

protected function getDbTable($dbTableName) {
    $dbTable = 'Application_Model_DbTable_' . $dbTableName;
    
    if ( array_key_exists($dbTableName, $this->tableCache) ) {

cache hit

  return $this->tableCache[$dbTableName];
    } else {

cache miss

        $this->tableCache[$dbTableName] = new $dbTable();
        return $this->tableCache[$dbTableName];
    }
}

Mistakes, flaws, and potential security threats would be greatly appreciated! It’s in no way complete, mainly proof of concept at present. I’d like to expand it with some real error checking, let it handle multiple rowsets, etc.

Meeting People

2010-05-19T00:00:00+12:00

Meeting people is good to do. People love being recognised, and also recognising other people. It makes us human. Meeting people leads to great things. That new job you were looking for, sponsorship for your latest project, a new friend even.

It’s simply amazing where you meet people. I was sitting in a meeting today (funny, that) and someone came in who I had met a few weeks ago at an event. Neither of us knew that we had anything to do with the group we were both meeting with, but it just so happened that we did. I looked good, he looked good, and the group in the middle looked good as well.

All because of a chance meeting, which only occurred because of a previous chance meeting, which only occurred.. you get the idea.

So next time someone wants to have coffee with you, or invites you to some event, just go! Who knows who you will meet there.

Cross Browser Css Dropdown

2009-02-28T00:00:00+13:00

The CSS dropdown from the last post works in all browsers except IE6! I’ve made a few minor changes to ensure it displays correctly in IE7 however.

.Navigation li ul { 
    	background:#FFFFFF none repeat scroll 0 0;
    border:1px solid #000000;
    padding:4px;
    position:absolute;
    width:720px;
    top:30px;
    overflow:visible;
    height:42px;
    left:-135px;
    z-index:10;
  }

A z-index was applied to keep it above any content that might be below, and a left position was added to move the dropdown over slightly.

Now as for IE6, we can get it to work with some simple javascript. Not the most accessible solution but really, if you’re using IE6 and you have javascript disabled, you deserve it.

Using jQuery we can add the hover event to our list with a few lines */

$(function() {
  $(".Navigation ul li").hover(
  function () {
    $(this).css("overflow", "visible");
  },
  function () {
    $(this).css("overflow", "hidden");
  });
});

and voila! Simply setting the overflow property when each list element is hovered is enough to make the dropdown work.

Now if you don’t want the extra overhead of the jquery library you can use conditional comments to only grab it if they’re using IE.

<!--[if IE 6]>
<script src='http://jqueryjs.googlecode.com/files/jquery-1.3.2.min.js' type='text/javascript'></script>
<script type='text/javascript'>
$(function() {
  $('.Navigation ul li').hover(
  function () {
    $(this).css('overflow', 'visible');
  },
  function () {
    $(this).css('overflow', 'hidden');
  });
});
</script>
<![endif]-->

Pure Css Dropdown For Firefox

2009-02-27T00:00:00+13:00

Here’s a quick and dirty pure CSS dropdown that at present only works in Firefox.

.Navigation li {
  position:relative; display:block;
  float:left;
  height:26px;
  overflow:hidden; 
  display:inline; 
  padding:0px 10px;
}
.Navigation li:hover { color:orange; cursor:pointer; overflow:visible;}

.Navigation li ul { 
  background:#FFFFFF none repeat scroll 0 0;
  border:1px solid #000000;
  padding:4px;
  position:absolute;
  width:600px;
  top:30px;
  overflow:visible;
  height:42px;
}
.Navigation li ul li { display:inline; color:#000; }

That’s all the CSS needed, along with this HTML:

<div class="Navigation">
    <ul>
        <li><a>Home</a></li>
        <li>
            <a>About Us</a>
            <ul>
                <li>Sub Link 1</li>
                <li>Sub Link 2</li>
                <li>Sub Link 3</li>
            </ul>
        </li>
        <li><a>Products</a></li>
        <li><a>Contact</a></li>
    </ul>
</div>

By floating each list element as a block element, we can hide the sub menu by hiding the overflow content of the div. Hovering over an element sets overflow to visible, and so the sub menu appears.

Now if only it worked in IE..

Running Yui Rich Text Editor Inside Thickbox

2009-02-04T00:00:00+13:00

//The latest project at work required me to get a rich text editor (RTE) running inside jquery thickbox. After trying out a few different RTEs I finally settled on the Yahoo YUI editor as it looks the nicest, and is easily customized.

//After 5 lines, YUI editor was up and running inside thickbox. Sweet, no problems there. However the insert image and link boxes failed completely, in all browsers. 

//The boxes were created and inserted alongside thickbox, instead of inside. Thankfully thanks to all of YUI editor's events this was an easy fix.
editor.on('windowInsertImageRender', function() {
     document.getElementById('dd-editor').appendChild(this.get('panel').element);
});

//Here, "dd-editor" is the ID of the div surrounding by YUI editor and 'editor' is the name of my YUI editor object. We simply set it to listen to the 'windowInsertImageRender' event and, when it occurs, attach it to the same div that the editor is in.

//That fixed, it seemed to be working very smoothly until, surprise, it fails in IE.

//![YUI Thickbox Bug](/uploads/2009/02/yui-thickbox-scroll-bug.png)

//When the thickbox is too small to hold all of the content and scrollbars appear, scrolling down creates all manners of positioning problems with the toolbar. Drilling down with the IE developer toolbar revealed that scrolling works down to the list element with a class of "yui-toolbar-groupitem". It is the elements inside that bug out.

//After an hour of trying various css positionings and styles, I managed to fix the content area and title bar in place with a bit of jquery.
$("#editor_editor").attr("position", "static")

//A fix for the buttons was eventually found as well, however it isn't so much a fix as it is a sluggish, ungainly hack.

$("#TB_ajaxContent").scroll(function () { editor.toolbar.collapse(true); editor.toolbar.collapse(false); } );

//Hooking in to the scroll event on the thickbox window, each time it moves we collapse and then redisplay the toolbar. This causes the buttons to jump into the correct place at the cost of a very slow scrolling time. 

//Note: the above 2 lines need to be called after the editor has fulled rendered and only in IE. This can be done by using the windowRender event and a bit of jquery, like so:
if ($.browser.msie) {
     editor.on('windowRender', function() {
         $("#editor_editor").attr("position", "static");
         $("#TB_ajaxContent").scroll(function () { editor.toolbar.collapse(true); editor.toolbar.collapse(false); } );
     });
}

Checking Your Account Data With Zend_http_client

2009-01-26T00:00:00+13:00

I live in a flat with 2 other avid internet users. Some months our bandwidth gets used up within a week yet my computer tells me I’ve only used a few hundred megabytes. In order to get accurate accountability for my internet usage, in comparrison to my flatmates, I wrote a PHP script to give me my usage as a percentage of the total usage for each day.

Our ISP provides a web portal which gives statistics such as daily and monthly usage. To get the statistics from my own usage, I installed vnstat.

Two combine the 2 I decided I would use PHP as an excuse to try out the Zend_Http_Client component.

First off I needed to grab the stats from my computer. This is quite straightforward with the system() function.

ob_start();
system('vnstat -d -s');
$vnstat = ob_get_contents();
ob_end_clean();

Output buffering is used to capture the information that the system command normally sends to the browser. As the system command only returns the last line of output from the command, we can get the entire output by using ob_get_contents().

The options for vnstat make it return statistics for just today (-d) and in short form (-s), looking like this

          rx       /         tx      /        total    /  estimated
 wlan0:
           yesterday    162.76 MB  /   78.25 MB  /  241.00 MB
         today    138.14 MB  /   89.97 MB  /  228.11 MB  /     523 MB

Now we need to get just the total for today: make an array of the output

$vnstatComponents = split("/", $vnstat);

get today’s usage (2nd to last element of the array) and remove whitespace

$netUsage = trim($vnstatComponents[count($vnstatComponents) - 2]);

remove ‘MB’, giving us just the number

$myUsage = substr($netUsage, 0, strlen($netUsage) - 3);

We’re now ready to start using Zend_Http_Client. open a link to isp website

$client = new Zend_Http_Client("https:// the.vodafone.co.nz/acnts/myaccounts.pl/login");
$client->setMethod(Zend_Http_Client::POST);
$client->setParameterPost(array(
    'login'  => 'myusername',
    'password' => 'mypassword'
));
$response = $client->request();

grab our cookies

$cookiejar = Zend_Http_CookieJar::fromResponse($response, $client->getUri());

The link we open a connection to is the link that the login form will submit to. Firebug can be handy to quickly find this out. You’ll also need to set the array keys in the setParameterPost method to match the names of the username and password fields on the login form.

The last part of the code is what will enable us to persist our login credentials. Zend_Http_CookieJar is fairly self explanatory in it’s name. We pass in the request object and the uri of the site we’re posting to and it will store the cookies that the browser would receive if we were actually visiting the site.

Now let’s try and log in to the usage page with our new cookies. if we’re logged in ok, open account page

if ($response->getMessage() == "OK") {
  $client = new Zend_Http_Client("https:# the.vodafone.co.nz/acnts/myaccount-int.pl/usage-data");
  $client->setCookieJar($cookiejar);
  $usageResponse = $client->request();	
}

And that’s it! We’ve now got an object that contains the entire content for the page we specified in the Zend_Http_Client constructor. Now this data might be a little unwieldy to dissect so we can use another handy Zend component, Zend_Dom_Query.

$dom = new Zend_Dom_Query($usageResponse->getBody());

If you have any experience with CSS then the usage of this object for getting data will seem quite natural.

$dom->query('.tdDataRight');

This will return an array of all elements with the class ‘tdDataRight’.

While this example is rather specific, it shows how to use Zend_Http_Client in a way that will no doubt apply to many other situations.

New On The Blog Scene

2009-01-24T00:00:00+13:00

After reading many blogs for many months I have finally managed to enter the blogosphere.

First, a little about myself. I’m currently in my third year of studying Software Engineering at the University of Auckland, New Zealand. I’ve also been doing freelance web development for the better part of 4 years. I also have a job as a developer/IT support guy at a company that deals with online learning solutions.

I love to work with PHP; it’s a very no fuss language that gives you a lot of flexibility, while still providing the strength required to make complex applications. As far as frameworks go, I’m all for them and use the Zend Framework.

I’ve also worked with the .NET framework with C# for both web and windows development and I have to say that I really enjoy using it. It’s a very powerful system that makes coding easy and fun. Unfortunately it’s not so great to develop from a Linux machine, which is my primary computer, and the licensing costs are only viable when you have a company paying for everything.

If you’re interested in some of my work, check out my portfolio at www.samdalton.co.nz.

Now that work is out of the way, some more about my personal self.

I’m an absolute music nut and spend my days listening to the likes of Pink Floyd, Led Zeppelin, Beethoven, Bach, Miles Davis, BB King, Tool, The Doors, Don McLean and other such blues/classic rock/classical/jazz bands, artists and composers. I’ve been learning Japanese for about 8 years now and really enjoy doing so. I play guitar in a band too.

What’s my reason for blogging? Many times I’ve found a solution to a problem that I simply was not able to find on the internet. I hope that by posting about problems and solutions that I come across, it will help others who find themselves in the same spot. As well as this, it’s a place for me to collect my thoughts and write about things that interest me, and hopefully others.