Ben Emson

Installing MariaDB using Homebrew

2012-01-26T00:00:00+00:00

Installing MariaDB using Homebrew

MariaDB is a binary compatible and improved version of MySQL. This means that MariaDB is just a drop in replacement for MySQL. Why would you use MariaDB over MySQL?

Well it's faster, and the speed in which the team fixes bugs and provides new features is also faster. Additionally there is talk that there has been a developmental exodus from the MySQL development team to MariaDB.

Anyway it is likely to expect MariaDB to overtake MySQL in the next few years and become the default web database choice and for MySQL to gradually step back from the limelight.

So this post briefly describes how you can install MariaDB on a Mac using Homebrew.

Installation - MariaDB

Clean up any old MariaDB/MySQL versions

Firstly ensure that you have removed all traces of any previous MariaDB or MySQL installations. This includes the log directories - so go ahead and check to see if directories like the one below still exists:

/usr/local/var/mysql/

Especially if you are getting error messages like this:
mysqld: File '/usr/local/var/mysql/aria_log_control' not found (Errcode: 13)
This is most likely due to your permissions not being set up correctly.

Steps to install MariaDB

Brew install
From your terminal execute the following commands:
```
  brew install mariadb
```

Build the default databases

 unset TMPDIR
 mysql_install_db --user=mysql --basedir=$(brew --prefix mariadb)

Path settings
Make sure that you have the following settings in your .bash_profile file:
```
  PATH="/usr/local/bin:/usr/local/sbin:$PATH" # if not already present
```
Start the server
You should now be able to start the MariaDB server:
```
  mysql.server start
```
And stop it with:
```
  mysql.server stop
```
Note you can also start it in safe mode with:
```
  sudo mysqld_safe
```

Nice Little Extras

Building the MySQL2 Ruby Gem

If you wish to run MariaDB with Ruby on Rails then you will want to compile the standard MySQL2 Gem. In order to do this it is best to set your ARCHFLAGS for your Mac's processor architecture.
In this case I am using 64 bit processor.:

    export ARCHFLAGS="-arch x86_64"

Now for the MariaDB specific commands:

    sudo gem install --no-rdoc --no-ri mysql2 -- --with-mysql-dir=$(brew --prefix mariadb) --with-mysql-config=$(brew --prefix mariadb)/bin/mysql_config

With any luck this should all install fine.

Configuring for Speed

If you don't tune MySQL and leave default settings, performance will be terrible with production data. Make sure to apply this cnf before importing any data or creating a database.

Here is a sample /etc/my.cnf (good if you have 3GB+ RAM)

[mysqld]
thread_concurrency = 16
log-bin=mysql-bin
binlog_format=mixed
innodb_data_file_path = ibdata1:100M:autoextend
innodb_buffer_pool_size = 1024M
innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 256M
innodb_log_buffer_size = 32M
innodb_flush_log_at_trx_commit = 2

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

References

Ruby Quine Example

2010-06-24T00:00:00+01:00

Ruby Quine Example

I was recently challenged to write a quine in the language of my choice. If you are unsure what a quine is you can find out from http://en.wikipedia.org/wiki/Quine_(computing). However basically in our case it is a piece of code that when executed creates a copy of itself. This copy can then be executed and ad infinitum.

There are some official rules about how a quine should be implemented:

A quine must not take any input, as this would allow the source code to be fed in from the keyboard.
Following on from the previous rule a quine, should not open it's own file and display it's contents.

Anyway here is my solution in Ruby...

lambda{|x| puts x + x.inspect}.call "lambda{|x| puts x + x.inspect}.call"

Here is a brief description of what is happening. The lambda creates an anonymous function that takes one parameter and outputs it's value

Officially I think passing a value into the lambda x attribute violates one of the rules - but non-the-less it is quite a clear example and may be useful to get an idea of how a quine works.

What is the Difference Between a Proc and a Lambda in Ruby?

2010-05-11T00:00:00+01:00

What is the Difference Between a Proc and a Lambda in Ruby?

The concepts of a Proc and a lambda in Ruby are subtly different and this post aims to try and explain the differences.

Procs and Lambdas

Firstly why do we need Procs and lambdas? In order to answer this you need to understand what a block is in Ruby.

Understanding what a block is?

A block is simply a piece of code, which is usually attached to a method, Proc or lambda. It is important to note that this block of code is NOT an object (yet...), it is code that is defined either within braces {} or within a do end container.

The code in a block is just logic that hasn't been wrapped into an object yet. When this block is passed to a method and that method has a yield statement then Ruby will wrap that block into a special type of object. That type of object is a Proc object. Once the block becomes a Proc Ruby can use it and the block code 'comes alive'.

Therefore we need a Proc object in order to handle blocks. The next question is how does a lambda fit in.

A lambda is a type of Proc object, it is almost completely identical to a Proc apart from two differences. This little code snippet shows that a Proc and a lambda are derived from the same superclass, Proc.

def what_are_lambdas_and_procs
  puts "--- #{__method__} ---"
  lam = lambda { puts 'lam variable assigned a lambda' }
  lam.call
  puts "lam is a class of: #{lam.class.name}"
  puts "lam method count: #{lam.methods.size}"
  puts
  prc = Proc.new { puts 'prc variable assigned a Proc' }
  prc.call
  puts "prc is a class of: #{prc.class.name}"
  puts "prc method count: #{prc.methods.size}"
  ""
end

This will output:

--- what_are_lambdas_and_procs ---
lam variable assigned a lambda
lam is a class of: Proc
lam method count: 55

prc variable assigned a Proc
prc is a class of: Proc
prc method count: 55

So what are the subtle differences between Proc and lambda?

As mentioned there are two differences, and these differences are :

in how a Proc/lambda assigns arguments, and
in what happens, when a Proc/lambda calls return within a caller method.

Proc, lambda argument assignment

When a lambda is called with more or less than the required number of arguments Ruby will throw an ArgumentError.

However when a Proc is called with more arguments, no error is thrown and the extra values are simply thrown away. When it is called with less arguments these parameters are simply assigned a value of nil.

These two code snippets demonstrate this:

def lambda_args
  puts "--- #{__method__} ---"
  lam = lambda { |a, b| puts "lambda with 2 arguments a:#{a}, b:#{b}" }
  lam.call( "first", "second")
  puts "lambda called with three arguments..."
  begin
    lam.call("first", "second", "third")
  rescue
    puts "   ArgumentError: wrong number of arguments (3 for 2)"
  end
  puts "lambda called with no arguments..."
  begin
    lam.call()
  rescue
    puts "   ArgumentError: wrong number of arguments (0 for 2)"
  end
end

def proc_args
  puts "--- #{__method__} ---"
  prc = Proc.new { |a, b| puts "Proc with 2 arguments a:#{(a.nil?) ? "nil" : a }, b:#{(b.nil?) ? "nil" : b }" }
  prc.call( "first", "second")
  puts "Proc called with three arguments..."
  prc.call("first", "second", "third")
  puts "Proc called with no arguments..."
  prc.call()
end

This will output:

--- lambda_args ---
lambda with 2 arguments a:first, b:second
lambda called with three arguments...
   ArgumentError: wrong number of arguments (3 for 2)
lambda called with no arguments...
   ArgumentError: wrong number of arguments (0 for 2)

--- proc_args ---
Proc with 2 arguments a:first, b:second
Proc called with three arguments...
Proc with 2 arguments a:first, b:second
Proc called with no arguments...
Proc with 2 arguments a:nil, b:nil

Proc and lambda return behaviour

When a lambda returns a value and this lambda is called within a method then method will simply continue it's execution until it itself returns a value.

However when a Proc returns a value and this Proc is called within a method this containing method will be exited WITHOUT any further execution. This is slightly surprising behaviour, and consequently a Proc should be handled with care.

These two code snippets demonstrate this behaviour:

def lam_return
  puts "--- #{__method__} ---"
  puts "simple function that calls a lambda and then returns it's own string."
  my_lam = lambda { return 'Lambda has returned' }
  my_lam.call 
  return 'lam_return method string returned'
end

def proc_return
  puts "--- #{__method__} ---"
  puts "simple function that calls a Proc, "
  puts "UT instead of returning it's own string returns the Proc's string."
  my_proc = Proc.new { return 'Proc string returned' }
  my_proc.call # this function returns and finishes here
  return 'proc_return method string returned' # the return is never called
end

This will output the following:

--- lam_return ---
simple function that calls a lambda and then returns it's own string.
lam_return method string returned

--- proc_return ---
simple function that calls a Proc, 
BUT instead of returning it's own string returns the Proc's string.
Proc string returned

Proc return work around

There is however a very simple work around to avoid a Proc from returning out of it's caller method. Simply removed the return statement, because Ruby always returns the last line this last line will be returned automatically.

def proc_return_work_around
  puts "--- #{__method__} ---"
  puts "simple function that calls a Proc, \nhowever the method string still returns."
  my_proc = Proc.new { 'Proc string returned' }
  my_proc.call # this function returns and finishes here
  return "#{__method__} method string returned" # the return is never called
end

This will output:

--- proc_return_work_around ---
simple function that calls a Proc, 
however the method string still returns.
proc_return_work_around method string returned

Conclusion

As you have seen there are only two small differences between a Proc and a lambda but these differences have quite surprising behaviours. Therefore following the Ruby principle of "...least surprise", it is best practice to use lambdas over Procs unless you absolutely need the Proc behaviour.

For more information please see this article: http://en.wikibooks.org/wiki/Ruby_Programming/Syntax/Method_Calls

Code Kata Four Exercise - A Ruby Solution

2010-05-09T00:00:00+01:00

Code Kata Four Exercise - A Ruby Solution

I recently did code kata 4 (data munging), and here is my solution to the problem. Like anything it can be improved but I think it is relatively simple to understand and yet flexible enough to work for a number of different formatted .dat files.

The premiss is to write a program that will work out the deltas ( difference ) between two columns in .dat files and return the lowest values.

These .dat files have different numbers of columns and these columns can be in different order. The two .dat files used for this program supplied weather temperature ranges and football (soccer) result scores.

Anyway here is my solution in the Ruby language and a description of how it works:

# data_processor.rb

module DataProcessor

  def process_dat(filename, regex=/^\s*\d+\.?\s+/)
    items = []
    IO.foreach(filename) do |line|
      if line =~ regex
        num, max, min = yield line
        delta = max.to_i - min.to_i
        items << [num, delta] if items.size == 0
        current_items = items[0][1]
        if delta <= current_items
          items.clear if delta < current_items
          items << [num, delta]
        end
      end
    end  
    items
  end

end

if __FILE__ == $0

  include DataProcessor

  puts "-------- Weather  --------"
  list = process_dat('weather.dat') { |line| tokens = line.split(' ') }
  list.each { |item| puts "Lowest Day:#{item[0]} Delta:#{item[1]}"}

  puts "\n-------- Football --------"
  list = process_dat('football.dat') do |line| 
    tokens = line.split(' ')
    [tokens[1], tokens[6], tokens[8]]
  end
  list.each { |item| puts "Team:#{item[0]} Delta:#{item[1]}"}
end

The code above was created as a module as this will allow the process_dat method to be mixed-in to other objects, allowing these objects to execute the code without having to instantiate another class to get this functionality.

The process_dat method takes a filename and an optional regular expression. The regular expression (regex) is used to find the data in the text file for processing. In this case any line that starts with 0 or more spaces followed by at least one number and then an optional '.' after that number followed by at least one space. This regular expression could have been used to extract the data from each row as well - but it actually became much easier to simply split each row and tokenise it around a space, as we are processing space separated files.

Using the IO.foreach method will iterate over each line of the .dat file allowing the logic to process that line. This is much more memory efficient, as only one line is loaded into memory at each iteration of the loop - thus allowing this method to process very large data files.

Next the method checks to see if the line matches the regex (if line =~ regex) if it does it can be processed.

Processing each line is handled by a Ruby block, (as a yield statement is used), in the "weather.dat" case the block simply splits the line around a white space. In the "football.dat" case the block splits the line around the white space but re orders the returned columns. Incidentally it is useful to know that if the number of tokens in the array exceeds the number of possible assignments (num, max, min = yield line, where yield line returns a 10 item array for instance) Ruby conveniently ignores the other values after assigning the first lot.

The next couple of lines simply calculate the delta and assigns it to a temporary array (items) if items is empty. The items array simply holds the lowest values as a collection of lowest values with the same value. Note that the items array is an array of arrays. The item in the items array consists of a number and a delta.

Finally we get the lowest item from the items array and check to see if the current line delta is lower if it is, clear the items array and add this new item. If it equals the same value simply add this item to the items array. Thus the items array will contain the same lowest values.

I hope you find this helpful - if you have an alternative solution, why don't you add it to the comments at the bottom.

JavaScript The Good Parts Best Practices

2010-04-09T00:00:00+01:00

JavaScript The Good Parts Best Practices

Part I

The best practice suggestions listed here are summaries from Douglas Crockford's book: JavaScript: The Good Parts

By following these principles you will write better JavaScript code.

JavaScript Best Practices

1. Commenting

Avoid using /* */ for commenting out your code instead use the // method. The reason being that */ can occur in regular expressions and if you were to comment out a regular expression containing this, you would get a syntax error.

2. Object Value Retrieval

For an object such as var myObj = {key:"value"}; An object value can retrieved in two ways:

                              
myObj["key"];   // "value"
myObj.key;      // "value"

However the second version using '.' notation is preferred as it reads better.

3. Avoid Global Variables

Global variables will reduce the resiliency of your JavaScript programs so try to keep these to a minimum. A best practice strategy for this is to use a single global variable for your program and simply assign objects to it. e.g.

var MYAPP = {};

MYAPP.myObj = {
  key:"value"
};

4. Capitalize Constructor Functions

Functions that are created as constructor functions should be capitalized. Think of a constructor functions as a factory creating cloned functions. Strictly this is not classical inheritance but prototypical inheritance. This is essentially where a function is cloned and contains all the properties of it's parent.

var Shape = function (name) {
  this.name = name;
};

var square = new Shape("square");

This style of constructor functions is not recommended as if you forget to use new then the this object gets bound to the global variable.

5. Avoid Recursion

JavaScript does not offer tail recursion optimization, therefore functions that recurse too deeply will eventually fail.

6. Use Function Scope To Protect Against Namespace Collisions

JavaScript does not have block scope, i.e. variables declared in a block statement (e.g. with in curly braces) are accessible from outside of that block. However functions do have function scope, variables declared within a function cannot be accessed from outside of that function.

Best practice is to declare the variables used in a function at the top of the function.

7. Use Functions To Create Modules

A Module is a function or object that presents an interface but hides its state and implementation. This therefore avoids the use of global variables.

var myModule = function() {
  var call = '';
  
  return {
    setCall: function(myCall) {
      call = String(myCall);
    },
    doCall: function() {
      return 'Hello ' + call;
    }
  };
}();  // the () will immediately invoke this function

var myCaller = myModule();
myCaller.setCall = 'Ben';
alert( myCaller.doCall() );

8. Use Cascades

If you have a method that changes state, be sure to return this as opposed to undefined. The reason is that you can then chain these methods together. This is how JQuery works.

getElement('myDiv').changeColour('red').setFont('Times').setStyle('italic');

9. For Constructor That Take Many Parameters Use Object Specifiers

If an object takes many parameters it can be tricky ensuring that these parameters are in the correct order. Therefore create a single object instead that holds all the parameters as properties:

// Tricky
var myObj = someObject(a, b, c, d);

// Better
var myObj = someObject({ first: a, second: b, third: c, fourth: d});

General JavaScript Notes

Strings are immutable, in that once a string has been created that string object cannot be changed. However note that you can easily append and modify a string, the outcome just generates a new string.

JavaScript provides a prototype linkage function, which when used allows one object to 'inherit' the properties of the linked object.

A try catch statement only allows a single catch block, to catch all exceptions. If you need to differentiate between exceptions then add code to inspect the name of the exception and handle it appropriately.

Objects

Object creation methods:

var myObj = {};

Set a default value if an object property doesn't exist:

var myObj = {one:"1"};
myObj.two || "2" // returns "2"

Assigning a value to an object property that is undefined will simply augment (or create) that property in that object and assign it.

Objects are passed by reference, they are never copied.

Every object is linked to a prototype object from which it can inherit properties. This object is Object.prototype

The prototype link is only used in the retrieval of properties from objects, if the first object does not have the property then JavaScript will use this prototype link to get the next object up the chain and try and retrieve the property from this object. If no property exists then undefined is returned.

When using for in for enumerating over properties in an object, be aware that it will also enumerate the properties of objects further up the prototype chain. If you don't want this simply just use for,

Functions

As Douglas Crockford says:

Functions are the fundamental modular unit of JavaScript. Use functions for code reuse, information hiding and composition. Functions are used to specify the behaviour of objects.

Functions are objects, but they have a prototype link to Function.prototype, and then this root function object in turn links to Object.prototype.

A function can take 0 or more specified parameters, however functions get two extra hidden parameters this and arguments.

A method is simply a function assigned (bound) to an objects property. e.g.

var myObj = {
  myMethod: function (name) {
    alert("Hello " + name);
  };
};

A function always returns a value. If no return value is defined then the function will return undefined. However if a function is constructed with the new prefix and the return value is not an object the new object is returned.

JavaScript Gotcha's

typeof Returns incorrect types

Using the typeof method will return six results:

number
string
boolean
undefined
function
object

The problem occurs with typeof ['1'] or typeof null which both return object which is incorrect.

Function Invocation Binds 'this' To The Global Object

When a function is created by assigning it to a global variable, and the contents of that function contains a this statement then the this is bound to the Global Object

When a function, contains an internal function the this of the internal function is bound to the Global Object and not the this of the parent function. To fix this simply assign this to a that variable and use that within your internal function.

var myObj = {
  value: 0;
  
  double: function() {
    var that = this;
    var helper = function() {
      that.value = that.value + that.value;
    };
    helper();
  };
};

arguments Is An Array Like Object Not An Array

All functions have a parameter arguments which gives a function access to all the arguments supplied with it's invocation - including excess arguments that are not assigned to a parameter. However note that arguments is in fact a special object with a length method and not an Array object. So arguments will lack methods that an Array object will have.

How To Set Up and Build a Django Web Site

2010-04-01T00:00:00+01:00

How To Set Up and Build a Django Web Site

This article outlines how to set up your new Django site from scratch. I tend to use Dreamhost Web Site Hosting so there will be a few notes specific to them.

If you are unsure how to set up Python and Django using Dreamhost then checkout my previous article.

I hope you find it useful and drop me a comment if you have any questions.

Create the new site

Create a project directory

$ mkdir -p mysite-project/mysite.com
$ cd mysite-project/mysite.com

Create you new site:

$ django-admin.py startproject mysite

Now create the following directories:

$ mkdir -p mysite-project/settings
$ mkdir -p mysite-project/mysite.com/tmp/
$ mkdir -p mysite-project/mysite.com/public/appmedia
$ mkdir -p mysite-project/mysite.com/mysite/scripts
$ mkdir -p mysite-project/mysite.com/mysite/templates

Initialise git

$ cd mysite-project/mysite.com
$ git init
$ touch .gitignore

Add the following:

.DS_Store
*.pyc
packages/
work/

Add WSGI file

The following should be carried out from this directory:

$ cd mysite-project/mysite.com

Add the passenger_wsgi.py file to mysite-project/mysite.com directory:

import sys, os
if sys.hexversion < 0x2060000: os.execl("/home/myuser/opt/python261/bin/python2.6", "python2.6", *sys.argv)

sys.path.append(os.getcwd())
# START
# required if you are using VirtualEnv, to get Passenger to use local Python environment.
# see: http://wiki.dreamhost.com/Passenger_WSGI
INTERP = "/home/myuser/virtual/bin/python2.6"
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)
# END
os.environ['DJANGO_SETTINGS_MODULE'] = "mysite.settings"
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Create the public/appmedia directory in this directory to. Create tmp/restart.txt

Set up the VirtualEnv Environment

Move to the mysite-project directory. Ensure that you have virtualenv installed on your local system:

$ cd mysite-project
$ pip install virtualenv

Create the virtual/ directories using virtualenv:

$ virtualenv ./virtual

Create a symbolic link to the activate executable.

$ ln -s ./virtual/bin/activate ./activate

NB: if you rename your mysite-project directory you may well need to delete the virtual directory and activate symbolic link and recreate them using the previous virtualenv steps.

You can check which Python site packages being used with this command:

$ python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Create Local Dev Database

Change directory to the scripts directory in mysite-project/mysite.com/mysite:

$ cd mysite-project/mysite.com/mysite/scripts
$ touch scripts/create-db.sql
$ touch scripts/drop-db.sql

create-db.sql

create database mysite_com_dev;
grant all on mysite_com_dev.* to 'root'@'localhost';

drop-db.sql

drop database mysite_com_dev;

Now create the database

$ mysql -u root < scripts/create-db.sql

Set up the Django Project Files

Make the following changes to the settings.py file.

import sys
import os
import os.path
import socket

PROJECT_ROOT = os.path.realpath(os.path.dirname(__file__))
sys.path.insert(0, os.path.join(PROJECT_ROOT)) # needed because of shared hosting PYTHONPATH problems

# MEDIA_ROOT = os.path.join(PROJECT_ROOT, 'public/appmedia/')
MEDIA_ROOT = '/full/local/path/to/appmedia/'
# MEDIA_URL = '/appmedia/'
MEDIA_URL = 'http://media.mysite.com/appmedia/'
ADMIN_MEDIA_PREFIX = 'http://media.mysite.com/media/'

TEMPLATE_DIRS = ( os.path.join(PROJECT_ROOT, 'templates').replace('\\','/'), )

INSTALLED_APPS = (
    'django.contrib.admin', # add this
    'django.contrib.admindocs', # and this
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
)

# Add this to the bottom to import your local settings
try:
    from settings_local import *
except ImportError:
    pass

Now we need to create a settings_local.py file in the same directory as the main settings.py file:

$ touch settings_local.py

Then add your default settings (most probably your dev settings):

DATABASE_ENGINE = 'mysql'     
DATABASE_NAME = 'mysite_com_dev'
DATABASE_USER = 'root'         
DATABASE_PASSWORD = ''         
DATABASE_HOST = 'localhost'    
DATABASE_PORT = ''

As we will be using multiple environments we need to have different configurations for each environment. There are a number of ways to do this, however the DRYist and easiest is to have a main settings file that imports different settings according to the environment.

Change directory to the mysite-project/settings directory, and create the appropriate settings files - we will have a dev, staging and production file as well as the main one:

$ cd mysite-project/settings
$ touch settings_dev.py; touch settings_staging.py; touch settings_production.py

Now add the appropriate settings to these files database settings. Note that these settings will overwrite the settings in the main settings file.

Adding South to Your Application

For detailed installation instructions see http://south.aeracode.org/wiki/Download.

Note, for MySQL databases, you should install South to a new instance of your MySQL database and not an existing or legacy database. This is because South won't update it's self when you run: python manage.py migrate

Quick installation.

Add South (Note the capital 'S') to your pip requirements.txt file, and pip install it:

$ pip install -r requirements.txt

Or just run:

$ pip install South

Install the south (Note lowercase 's') into your project settings.py file. e.g.:

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.admindocs',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'south',
)

Now syncdb with your database so that your App knows about South.

$ python manage.py syncdb

You should get an output similar to this:

Syncing...
Creating table south_migrationhistory

Synced:
 > django.contrib.admin
 > django.contrib.auth
 > django.contrib.contenttypes
 > django.contrib.sessions
 > django.contrib.sites
 > django.contrib.markup
 > south
 > tagging

Not synced (use migrations):
 - 
(use ./manage.py migrate to migrate these)

Finally you need to tell South what Apps you want it to manage (in terms of migrations). So for example say you have an App called 'firepages', then run:

$ python manage.py startmigration firepages --initial

This will create a migrations directory and will allow South to manage all the models in that App:

  Creating migrations directory at '/mypath/mysite-project/mysite.com/mysite/firepages/migrations'...
  Creating __init__.py in '/mypath/mysite-project/mysite.com/mysite/firepages/migrations'...
   + Added model 'firepages.Group'
   + Added model 'firepages.Category'
   + Added model 'firepages.Page'
  Created 0001_initial.py.

Note that because you will be uploading this new migrations directory you shouldn't need to run this command on the server.

You will now need to execute the following command to migrate forward:

$ python manage.py migrate

Now if you want to change your database do the following:

Modify your model. e.g. add / remove fields
Tell South to create a migration: python manage.py startmigration myapp name-of-my-migration --auto
Execute the migration: python manage.py migrate

Change urls.py

Use the following imports:

from django.conf import settings
from django.contrib import admin

admin.autodiscover()

At the bottom add a new URL path so that the site can get access to the appmedia, NB don't forget to add a staging and production version to:

# simple trick to use local css files for development.
# (Just make sure DEBUG=False in production!!!)
if settings.DEBUG:
    urlpatterns += patterns('', 
        url(r'^appmedia/(?P<path>.*)$', 'django.views.static.serve', { 
            'document_root': settings.MEDIA_ROOT }),
        )

Now add the following URLs:

urlpatterns = patterns('',
    #Admin URLS
    # (r'^admin/doc/', include('django.contrib.admindocs.urls')),
    (r'^admin/', include(admin.site.urls)),
    
    #Home URL
    (r'^$', include('mysite.foo.urls'),
)

You may want to change the admin URL pattern to be something else. This will make the admin area more obscure thus helping to prevent hacking. e.g. mylogin

Add a views.py file

In the mysite-production/mysite.com/mysite/ directory add a views.py file with the following:

from django.http import HttpResponse

import settings

def hello(request, path='none'):
    """quick hello test"""
    return HttpResponse("Quick test, PROJECT_ROOT: %s<br/>URL path: %s" % (settings.PROJECT_ROOT, path))

Sync the DB and Start the Server

Navigate to the mysite-project/mysite.com/mysite directory

Start the virtual environment:

$ . ../../activate
(virtual)$ python manage.py syncdb
** create a user **
(virtual)$ python manage.py runserver

Now open a browser and point it at this URL:

$ open http://localhost:8000

It should display your hello view results.

Requirements

Add a requirements.txt file to the following directory:

$ touch mysite-project/mysite/requirements.txt

Add in any requirements e.g. django-tagging

$ echo mysite-project/mysite/requirements.txt << django-tagging

Execute and add requirements:

$ pip install -r requirements.txt

Local Apps Directory

Create an apps directory in the root of your project folder. Then tell your settings.py file about it:

  sys.path.insert(0, os.path.join(PROJECT_ROOT, "apps"))

Now any apps that you want in here and not in your "pip install" directory just add them and they will be found.

RequestContext on All Views

All your views should implement a RequestContext variable, but first you will need to add the following CONTEXT_PROCESSORS to the settings.py file:

TEMPLATE_CONTEXT_PROCESSORS = (
    'django.core.context_processors.auth',
    'django.core.context_processors.debug', # passes the DEBUG variable to the template
    'django.core.context_processors.i18n',
    'django.core.context_processors.media', # passes the MEDIA_URL to the template
    'django.core.context_processors.request', # passes the request object to the template
)

Now Make sure that you pass the RequestContext to the render.

def home_page(request):
    """display home page"""
    template = "page_home.html" 
    return render_to_response(template, { 'foo':'bar' }, context_instance=RequestContext(request))

This will allow you to use generic templates, and also get a handle on certain variables in the template.

Page Specific Menus

See this article for more information: http://gnuvince.wordpress.com/2007/09/14/a-django-template-tag-for-the-current-active-page/

Django doesn't out of the box provide a current_page variable like Ruby on Rails - but it is easy enough to implement.

You will need to have the 'django.core.context_processors.request', setting set in the template context processors as specified above.

You will also need to create a module called templatetags with a init.py file in it. Now add a file called active_menus.py

from django import template

register = template.Library()

@register.simple_tag
def link_to(request, pattern, url, name):
    import re
    if re.search(pattern, request.path):
        return '<span class="active">%s</span>' % name
    return '<a href="%s">%s</a>' % (url, name)

Now in your template load this template tag and call it like this:

{% load active_menus %}
<li>{% link_to request "^/$" "/" "home" %}</li>
<li>{% link_to request "^/software/$" "/software/" "software" %}</li>
<li>{% link_to request "^/books/$" "/books/" "books" %}</li>

Template Includes

When creating an include template file give it a name starting with an underscore e.g. _nav.html

Add a Sitemap to Your Site

Add the following to your settings.py file in the INSTALLED_APPS section:

    django.contrib.sitemaps

Now check that you have the following installed also.

This should be in the TEMPLATE_LOADERS:

  'django.template.loaders.app_directories.load_template_source',

Make sure youâ€™ve installed the sites framework. Also make sure that you have set the site domain name and description correctly in the sites project in the Admin Panel. It should not be example.com, but something like: myproject.com

Create a sitemaps.py file in your project. This file should contain something like:

from django.contrib.sitemaps import Sitemap
from myproject.models import ProductPage, ProductCategory

class ProductPageSitemap(Sitemap):
    changefreq = "never"
    priority = 0.5

    def items(self):
        return ProductPage.objects.all().filter(published=True)


class ProductCategorySitemap(Sitemap):
    changefreq = "never"
    priority = 0.5

    def items(self):
        return ProductCategory.objects.all()

Now add the following to your urls.py file:

        
from myproject.sitemaps import *
# from django.contrib.sitemaps import FlatPageSitemap, GenericSitemap

sitemaps = {
    'product_pages': ProductPageSitemap,
    'product_categories': ProductCategorySitemap,
}

urlpatterns = patterns('',
  (r'^sitemap\.xml$', 'django.contrib.sitemaps.views.sitemap', {'sitemaps': sitemaps})
)

Adding a new App

$ python manage.py startapp myappname

Add myappname to your settings.py file INSTALLED_APPS Now create some models in your myappname/models.py file.

Then run a syncdb:

$ python manage.py syncdb 
OR to see the SQL
$ python manage.py sql myappname

Now if you want to add South migrations:

$ python manage.py startmigration myappname --initial

Installing PIL

PIL should not be installed as part of the requirements.txt file, as you will need to compile libjpeg first.

How to Install Python on Dreamhost Shared Hosting

2010-04-01T00:00:00+01:00

How to Install Python on Dreamhost Shared Hosting

This article describes how to install and set up Python and VirtualEnv on Dreamhost web site hosting.

Some of these notes are based on this article:

URL: http://blog.localkinegrinds.com/2007/08/20/custom-python-installation-for-django-on-dreamhost/

Set up

Log into your shared server and do the following:

$ mkdir -p opt/python261
$ mkdir downloads
$ cd downloads
$ wget http://www.python.org/ftp/python/2.6.1/Python-2.6.1.tgz
$ tar xvzf Python-2.6.1.tgz

Note that: According to the Filesystem Hierarchy Standard, the /opt dir "is reserved for the installation of add-on application software packages."

Now configure where your new Python instance will be installed:

$ cd Python-2.6.1
$ ./configure --prefix=$HOME/opt/python261
$ make
$ make install > install.txt

NB: The install.txt file is useful to locate files if you wish to uninstall Python later. NB: you might wish to use:

$ ./configure --prefix=$HOME/opt/python261  --enable-unicode=ucs4

But Dreamhost does uses ucs2 by default (Python's default to), and there can be issues if you try to load packages that are compiled using a different Unicode setting.

Now edit your ~/.bash_profile file to specify the new Python install located in: ~/opt/

export PATH=$PATH:$HOME/opt/python261/bin

You might also want to add an alias so that instead of typing python2.6 to access the python shell, you can just type python in the command line.

alias python='python2.6'

Don't forget to tell bash to load this file after you have made your changes:

$ source ~/.bash_profile
$ python -V  # will display your new version 2.6.1

Installing VirtualEnv

$ cd downloads/
$ curl -O http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.4.5.tar.gz
$ tar -xvzf virtualenv-1.4.5.tar.gz
$ python virtualenv-1.4.5/virtualenv.py $HOME/virtual --no-site-packages

NB: This binds your new install with virtualenv and creates a virtual directory in your user home directory. You will need to use the --no-site-packages option so that virtualenv doesn't use any of the original Dreamhost Python install packages.

Update your ~/.bash_profile file so that it puts the virtualenv packages onto the Python path, and activate(s) your virtual environment:

export PATH="$PATH:$HOME/opt/python261/bin:$HOME/virtual/bin"
source $HOME/virtual/bin/activate

Check that the packages directory is in your new virtual site_packages directory:

$ python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Now activate your virtual environment, by executing the .bash_profile file:

$ source ~/.bash_profile

Or manually activate it by:

$ . ~/virtual/bin activate

Installing MySQL Python Bindings

http://sourceforge.net/projects/mysql-python/files/mysql-python-test/1.2.3c1/MySQL-python-1.2.3c1.tar.gz/download

(virtual)$ cd downloads
(virtual)$ wget http://internap.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.3c1.tar.gz
(virtual)$ tar xvzf MySQL-python-1.2.3c1.tar.gz
(virtual)$ cd MySQL-python-1.2.3c1
(virtual)$ python setup.py install

Check that you can import the database correctly (NB: do this from a different directory from the one where MySQL-python was installed, as you will get Module already imported warnings):

(virtual)$ cd ~
(virtual)$ python
>>> import MySQLdb

If you don't get an error all succeeded OK.

Install Django

(virtual)$ pip install Django

How to Automatically Log In Using SSH On Dreamhost

2009-09-25T00:00:00+01:00

How to Automatically Log In Using SSH On Dreamhost

If you build web sites and host (or want to host) them on Dreamhost, you will be well aware that at some stage you will be using SSH to log in and out of your hosting server. If you have many sites then this gradually becomes tiresome - especially if you use tools like Capistrano to deploy your applications, as you will constantly have to re-enter your passwords.

Fortunately you can certify your local computer, such that it will automatically enter the SSH password for you. Here are some quick tips on how to do it.

Generate Your Security Keys

Generate your private/public key pair on your local machine, from your home directory:

ssh-keygen -d

Now DON'T enter a password - just press return, otherwise you 
will still need to enter a password, when you log in with SSH.

Set Privileges And Copy The Public Key To Your Server

Use the following command to copy your newly created ~/.ssh/id_dsa.pub file up to the server you wish to remote log in to. Note that this command below will set the permissions both locally and remotely. The remote .ssh/* needs to be set to 0600.

N.B. Replace USERNAME and MY_DOMAIN with your own username and domain.

ssh USERNAME@ftp.MY_DOMAIN.COM 'test -d .ssh || mkdir -m 0700 .ssh ; cat >> .ssh/authorized_keys && chmod 0600 .ssh/*' < ~/.ssh/id_dsa.pub

  Enter the normal USERNAME password.

Log In Using SSH

Now log in using:

ssh USERNAME@ftp.MY_DOMAIN.COM

You shouldn't neet to enter a password.

Taking It Further

To simplify things even further you can modify the ~/.ssh/config file such:

Host NICKNAME
Hostname MY_DOMAIN.COM
User USERNAME

Now you only need to type:

ssh NICKNAME

And you will be automatically logged in.

Create Sinatra Partials The Rails Way

2009-09-25T00:00:00+01:00

Create Sinatra Partials The Rails Way

This is just a quick post. I've been playing around with the Sinatra micro web framework, and have to say that it is excellent. It really is the perfect tool for rapidly creating small and useful Ruby micro-sites.

Being small and lightweight it doesn't have many of the features of larger frameworks like Ruby on Rails. One of the features I've begun to need is the handling of Rails like partials.

Notably I need to pass both local variables and collections to the partial code and have the partials rendered accordingly. I started implementing my own version, and then came across a nice little Github project: Sinatra-Helpers by sbfaulkner.

Anyway here is the code he uses for Sinatra partials:

def haml_partial(name, options = {})
  item_name = name.to_sym
  counter_name = "#{name}_counter".to_sym
  if collection = options.delete(:collection)
    collection.enum_for(:each_with_index).collect do |item,index|
      haml_partial name, options.merge(:locals => {item_name => item, counter_name => index+1})
    end.join
  elsif object = options.delete(:object)
    haml_partial name, options.merge(:locals => {item_name => object, counter_name => nil})
  else
    haml "_#{name}".to_sym, options.merge(:layout => false)
  end
end

You can use his project or if you just need this feature just pop this code into your Sinatra helpers file application_helper.rb:

helpers do

  def partial(name, options = {})
    item_name = name.to_sym
    counter_name = "#{name}_counter".to_sym
    if collection = options.delete(:collection)
      collection.enum_for(:each_with_index).collect do |item, index|
        partial(name, options.merge(:locals => { item_name => item, counter_name => index + 1 }))
      end.join

    elsif object = options.delete(:object)
      partial name, options.merge(:locals => {item_name => object, counter_name => nil})
    else
      haml "_#{name}".to_sym, options.merge(:layout => false)
    end
  end
end

Now you can call the partial from a your page e.g. index.haml

= partial("mypartial", :locals => {:var1 => "value 1", :var2 => "value 2 and so on"})

Create a partial in the same directory as the calling page. Note that this code doesn't like filenames truncated with a '-', so avoid this if possible. Also like in Rails your partials should start with an underscore. e.g. _mypartial.haml

To access the variable from this partial simply call the local variable name in your code:

%h2 
  This is var1:
  = var1
  
%p 
  And this is var2:
  = var2

Handling collections, again, is similar to the way Rails handles partial collections. From your page call the partial with a collection such as:

= partial("listitem", :collection => ["item 1", "item 2 and so on"])

Now in the _listitem.haml partial code simply provide the partial name as a variable or the partialname with _counter for the increment number e.g.

%li
  = listitem
  is the 
  = listitem_counter
  in this array

Anyway all pretty basic stuff but I hope it helps the casual Sinatra surfer.

Goodluck.

18 Useful bash scripts for web developers

2009-06-05T00:00:00+01:00

18 Useful bash scripts for web developers

Using bash scripts to become a more efficient web developer

Here are a few scripts, that I find really useful for speeding up my web development time.

I've been building up this list as I needed to use them - so they maybe a little raw.

For example often clients send me images with filenames that don't match my naming standard, so running the appropriate script really helps keep me focussed on the job in hand and not waste too much time reformatting filenames etc.

Finally if you have any useful little bash scripts why don't you add them to the comments below?

Bash script to append a .txt extension to a filename

Iterate over the current directory, get all files with .log and append .txt to the end of the entire filename:

for i in *.log*; do mv "$i" "$i.txt"; done

Script to make filenames lowercase

Converts all the file names in a directory and converts them to lowercase.

echo version:

for i in *.log*; do echo mv \"$i\" \"`echo $i| tr [A-Z] [a-z]`\"; done

real version:

for i in *.txt; do mv "$i" "`echo $i| tr [A-Z] [a-z]`"; done

Quick note on how to use grep to search for a string in file

-o print Only the matched parts of a matching line, with each part on a separate line. -H print the filename for each match. -n prefix each line of output with the 1 based line number. -R, -r recursively read all files under subdirectories.

grep -oHnr "some pattern"  *.txt

{txt,log} this globs the file extensions together. 2>/dev/null this passes all errors to a blackhole, so that they won't be displayed.

grep 123741 ./adapter_logs/*.{txt,log} 2>/dev/null

Find all occurrences of 123741 in all .txt and .log files in all subdirectories.

for i in 'find . -type d'; do grep 123741 $i/*.{txt,log}; done 2>/dev/null

List all files that match this regular expression in the current direcory

All files with the number 5 in their name.

ls | grep -e .*5.*

Bash script to check whether a tag exists in a subversion repository

svn ls http://www.mysvnserver.co.uk/myproject/tags | grep mytag-0.7.0.1/

The [ ] is known as a Test, by default it tests integers Also to end an if statement you use fi

if [ 1 = 1 ] ; then echo YES; else echo NO; fi

if [ "`svn ls http://mysvnserver/mysvnrepository/myproject/tags | grep R-9.20.400.0/`" = "R-9.20.400.0/" ]; then echo YES; else echo NO; fi

Note need the / in the evaluation part of "it-0.9.0.7/", don't really need it in grep it-0.9.0.7/

$ if [ "`svn ls http://www.mysvnserver.co.uk/myproject/tags | grep it-0.7.0.1/`" = "it-0.7.0.1/" ]; then echo YES; else echo NO; fi

Quick note on how to increment a variable, in a bash script

j=0; j=$(($j+1))

incrementing in a loop

j=0;for i in *.txt; do echo "some_file_"$j".txt"; j=$(($j+1)); done

Using a bash for loop to list all text files and append whoopee

for i in *.txt; do echo $i"whoopee"; done

Change all filenames to be a specific incremented name, starting from file 16

j=16;for i in *.jpg; do mv "$i" "gallery_"$j".jpg"; j=$(($j+1)); done;ls

Script to loop through all .txt filenames and make them lower case

for i in *.txt; do mv "$i" "`echo $i| tr [A-Z] [a-z]`"; done

Get all JPG files and create the appropriate HTML list tags for them and add them to a file

Create an HTML version:

for i in *.jpg; do echo "\t<li>\r\t\t<img src='images/$i' alt='' />\r\t</li>"; done > list_items.html

Create a HAML version:

for i in *.jpg; do echo "\t%li\r\t\t%img{ :src => 'images/$i', :alt => '' }\r"; done > imgs.haml

Batch change a misspelled filename of certain files, using string replacement

for i in aples*.jpg; do mv $i ${i/aples/apples} ; done

Script to batch create files based on filenames in a file

Create a file called something like "my_files.txt" with the following content:

archive_error_112480_0040.txt archive_error_114390_0043.txt etc...

Commands:

for i in `sed -n -e 'p' my_files.txt`; do mkdir a_test/$i; done
or
while read x; do touch $x; done < my_files.txt
or
cat my_files.txt | xargs touch
(xargs passes each line as an argument into EACH touch)

Using bash Expansions, and allow escapes

-e is an undocumented flag that allows escapes Expansion ${command/parameter/substitution} everything else needs to be escaped.

echo -e ${PATH//\:/\\n}

Executing a .bash_script after making changes

source .bash_profile
or
. .bash_profile

NB: you can 'include' other bash .sh scripts in a .bash_profile file by using 'source' or '.'

Quick tip on using curl for downloading files off the web

Use -O to name the remote file, curl will save it locally but remove the rest of the path.

curl -O http://rubyforge-files.ruby-forum.com/rubygems/rubygems-0.9.0.tgz

Bash command for testing the existence of files and directories

If a directory does NOT exist create it

[ -d "/var/cache/git" ] || mkdir /var/cache/git

If a file exists echo a message:

[ -f "./new_file.txt" ] && echo "its there"

Bash command to remove spaces from filenames

for i in *.html; do mv "$i" "`echo $i| tr -d ' '`"; done

replace spaces for underscores

for i in *.html; do mv "$i" "`echo $i| tr ' ' '_'`"; done

Vim regex to replace spaces in images file names of ah HTML page

:%s:\(images\/\)\(\w\+\)\s\(\w\+\.jpg\):\1\2_\3:g

Copy text to the clipboard

echo 'hello world'|pbcopy
cmd + v

Conclusion

Some of these are quite basic but useful none-the-less, please add your own to the comments below. Many thanks, and good luck.

Updated...

Script function for .bash_profile to cd to the last opened finder location

This is a script written by a colleague. James Power. Add the following function to your .bash_profile in Apple OSX. Now open a directory in Finder, then open terminal and type cdf, your terminal will now change directory to the same one as the last Finder opened.

# function to change directory to the one set in the last opened finder.
cdf () {
   currFolderPath=$( /usr/bin/osascript <<"           EOT"
       tell application "Finder"
           try
               set currFolder to (folder of the front window as alias)
           on error
               set currFolder to (path to desktop folder as alias)
           end try
           POSIX path of currFolder
       end tell
            EOT
   )
   echo "cd to \"$currFolderPath\""
   cd "$currFolderPath"
}

Also for a much more comprehensive list of commands check out: http://www.commandlinefu.com/commands/tagged/34/bash

Ruby Rush command-line tool

If you use Ruby maybe check out Rush, it allows you do use Ruby in a bash command-line like way.

Halcyon Events logo on Jenson Button's Formula One helmet

2009-03-28T00:00:00+00:00

Halcyon Events logo on Jenson Button's Formula One helmet

Jenson Button has just got pole position ready for the Australian Formula One race tomorrow. This is pretty exciting as on either side of his helmet is the Halcyon Events Ltd. logo.

Halcyon Events is a company that specialises in high end bespoke Formula One events. Through their contacts they can take you and your clients behind the scenes to areas not normally permitted, and give you a unique experience into Formula One (F1).

The web site was built by me, the design was created by Scamper Branding and the logo on Jenson's helmet was designed by Firefly Creative.

Anyway fingers crossed for Jenson Button to win the Australian Grand Prix!.

-- Update -- Fantastic Jenson Button won the Melbourne Grand Prix! How exciting.

An apology and a promise

2009-03-12T00:00:00+00:00

An apology and a promise

This is just a quick note to apologise for not posting for the last couple of months. A lot has been happening and I have actually written a number of posts but never quite finished them. Also I promise to write more regularly come the end of March 2009 (hopefully).

Well my New Year started with a 'Looney Dook' in the Firth of Fourth on New Years day. Man it was cold and a struggle to get in, but I felt fantastic afterwards. Its seems like quite a good analogy of how my year has been so far.

Anyway I've been busy building a couple of web sites, with a number still under development, if you get a moment please check them out and let me know what you think:

I've also been coming up with a number of little side line projects which I hope to share with you soon. One of them involves using Twitter and if you want to hear more when the time comes you can follow me on twitter/emson.

Finally this year for me, and I'm sure everyone, seems to be about how to restructure and think in terms of business and income. I expect if you are open to change and think a bit laterally there are many opportunities, if you don't change you are likely to struggle.

The business world is changing beneath our feet and we need to be quick to act on the opportunities it presents us. Seth Godin's quick article highlights this rather well.

So I'll start putting together some interesting articles from the beginning of April onwards - I hope you can wait till then, and thanks for following me.

Ben...

Chrome is a Desktop Web Application platform

2008-09-04T00:00:00+01:00

Chrome is a Desktop Web Application platform

Google Chrome is not just a browser

Google Chrome was launched a few days ago on the 2nd September 2008, since then there has been much hype and articles written about it. So what is it? Well, Wikipedia describes it: "Google Chrome is a free and open source web browser developed by Google.", however it is more than that.

It comes dressed as a web browser but is actually a Desktop Web Application platform. It facilitates the creation of Web Applications that are desktop like in functionality. These Web Applications will, be much more responsive, secure and stable than the previous generation of web applications because these new technologies are forming a suitable infrasructure.

It's one of the technologies ushering in a new era for the Internet.

I wrote an article recently that describes how these technologies are laying down foundations for applications should use the web: Desktop web applications using SproutCore/

Why is Chrome significant

Recently a number of the big Internet centred companies have been making strides to enable the next generation of Web Applications.

Adobe with its AIR technology, Microsoft with Silverlight, Apple with its endorsement of the SproutCore JavaScript framework, and Google, until this announcement, Gears.

The problem is none of these offerings is perfect, and some of them are very much propietary, tying users into their technology.

The problem Google has is that Gears is actually a very good product, but why would anybody download and install it if there aren't may web sites using it? At the moment this is an additional step resulting in the fact that most users have no compelling reason to install it. In a nut shell nobody is using it.

However with Chrome Google is able to hedge its bets, it has merged several standalone technologies into one very capable and pleasing package.

It has an awesome JavaScript engine enabling JavaScript to approach Desktop Application speeds.
It has Gears enabling developers to create offline applications that can interact with your desktop.
It can open a Web Application directly in its own window, without any of the traditional browser tool bar and menu overheads. Making Web Applications look like Desktop Applications.
It has added stability by making each tab a separate process, essentially each tab behaves like its very own browser.
It is secure in that it has been designed from the ground up following proven security models, such as sandboxing.
Its elegant, meaning that it feels small, light and fit for purpose. It has taken some leaves out of Apple's book and made the UI simple yet powerful.

Chrome is OpenSource and follows standards

This next era of the Internet is going to very busy. Everyday there are vast numbers of web pages and Web Applications created, and with this colosal amounts of data and information to sift through.

For an individual it is becoming harder and harder to focus and extract the useful information from all that data. In essance attention is a currency and with all these web sites and applications vying for our eyeballs how do we as users filter the wheat from the chaff?

That part of the Internet isn't completely written yet but for certain, by following Web Standards and by incorporating OpenSource technologies we are laying the foundations for future tools to help us do this. Chrome is one such OpenSource technology.

Chrome needs to follow standards as these will pull it along and it will evolve into something very powerful. Even if it doesn't some of its components may find there way into some technology that does. I dare say that Google will have a finger in that pie no matter. As a result it is a win win situation for Google.

This also highlights the point that Microsoft's Internet Explorer browser is gradually getting left behind. Admittedly standards aren't followed by all these companies but Microsoft has certainly been more relunctant to endorse them.

JavaScript is a key web technology

JavaScript is old, it appeared in the early browser wars and there hasn't been any significant changes since. Its been buggy, doesn't scale well, and there are differences on just about every browser implementation, but it is everywhere.

Recently, with Web 2.0, JavaScript has had a bit of a resurgance and some very useful frameworks have emerged which mitigate the implementation issues and improve its stability. However up until recently it has been slow. Too slow to be useful in terms of creating sophisticated Desktop like Web Applications.

It seems that in order to compete in the new Desktop Web Application arena companies need to implement a fast JavaScript engine. Apple's Safari is implementing SquirrelFish, and Firefox is implementing TraceMonkey and now Google has created V8.

John Resig has done a comparison of the relevant JavaScript engines and it seems that V8 is very fast. Matthieu Riou says that V8 is closer to a compiler than a traditional VM. It takes JavaScript code and converts it into low level byte code. It is still early days but we can expect this performance to keep improving.

Desktop Web Applications, Slickening the Web.

This era of the Internet is seeing a 'slickening of the web'. Web Applications are becoming more Desktop like. Platforms like Chrome are only going to push us faster along this path.

We can expect to see more applications like Gmail, MobileMe, GoogleMaps, GoogleDocs, etc. but Chrome puts Google very definately in one of the front seats for the future.

Its not fully there yet as its still missing features but it will evolve quickly. Google has vast resources and is able to test the browser against millions of web sites before it ever sees a user. This will only help to refine and improve it.

With Google Chrome, the Web is changing up a Gear.

Cannot select text with mouse in Microsoft Word, fix.

2008-08-21T00:00:00+01:00

Cannot select text with mouse in Microsoft Word, fix.

A friend had the following problem with their laptop the other day. When they open Microsoft Word they cannot select any text with the mouse, they couldn't even place the cursor into the text by clicking the mouse on the text.

She said that the situation seemed to have occured when Norton Antivirus requested an update at the same time she was doing something in Word.

I suspected that it was that Word had somehow got corrupted, so I ran a repair, by inserting her Microsoft Office DVD and bring up the installer, and selecting Repair.

This made no difference, I then reinstalled Office, still the problem persisted. This made me think that the problem was related to the Registry, so I did a Google search.

Fixing Word text selection

I found these articles:

http://support.microsoft.com/kb/921541 which describes how to troubleshoot problems with Microsoft Word 2007/2003/2002. (Really I'm not impressed it just stinks of shoddy programming.)
http://www.vistax64.com/vista-general/39390-cant-select-text-word-2007-vista.html

To summarise the fix:

Backup the Word\Data part of the Registry first
delete the the Word\Data section
restart Word
if it works then its completed, else restore the backup and keep Google searching.

Warning about Word registry changes

This fix involves removing the Word/Data registry key. By deleting this key you will loose some options you have set, this just means that you will need to reset them again. Additionally you will loose the list of files in your recently used files list at the bottom of the file menu.

Additionally be really careful about deleting anything out of the registry, because it can cause serious problems. All the main Windows programs use it, and you could end up reinstalling everything including your copy of Windows.

Backup the offending part of your computer's registry:

Exit all Office programs, Word, Excel, Outlook etc if they are running.
Open regedit by: Clicking Start/Run, then type regedit, and select OK.
Navigate the registry and select the appropriate Word registry subkey:
- Word 2002:
  HKEY_CURRENT_USER\Software\Microsoft Office\10.0\Word\Data
- Word 2003:
  HKEY_CURRENT_USER\Software\Microsoft Office\11.0\Word\Data
- Word 2007:
  HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Word\Data
Select the Data node, then from the File menu choose Export.
Name this backup file something like word_data.reg, and save it your desktop.

Delete the Word\Data registry node

Select the appropriate Word\Data node mentioned in part 3.
From the Edit menu select Delete, and then click Yes.
Exit regedit.

Restart Word

Restart Word. This will cause Word to recreate this registry node, but you may have to re-enter some of your Word options. If everything is working as expected you can finish. If you still have a problem follow the next section.

Restore the backed up registry node

If the problem still persists then restore your registry back, and keep looking for a solution. To restore the registry back to normal simply:

Double-click the word_data.reg file on your desktop.
and select Yes, and then OK.

SproutCore and Web Applications

2008-08-18T00:00:00+01:00

SproutCore and Web Applications

It's been a wee while since my last article, which has largely been due to my summer holiday. Anyway I've been reading up and experimenting with the SproutCore JavaScript framework.

Its excellent but I wanted to explain why companies and developers should consider using it. So I've written an article on my business partner's web site http://rapidappsgroup.com/content/desktop-web-applications-using-sproutcore/.

Please check it out as it tries to answer a lot of the general questions about web applications and JavaScript frameworks.

BE...

Everyday MySQL Commands

2008-08-18T00:00:00+01:00

Everyday MySQL Commands

I'm trying to put together a number of blog articles and many of them use the MySQL database, so I figured that I would post up some of my most used everyday MySQL SQL commands. Most likely I will append further commands as I write more articles so this is page is really a work in progress.

I've also assumed that you want to have full rights when executing these commands so I've used sudo, however you may not need this depending on your set up.

Finally if some other web site covers the command better or I'm feeling lazy I'll just pop it in the links list at the bottom of this page.

Useful MySQL Commands

Starting MySQL Demon

Start your MySQL database and get it running as a background process:

sudo mysqld_safe
Ctrl + Z
bg

MySQL create databases script

Make a SQL script that creates 3 databases and assigns all rights to my username. Copy the following SQL and paste it into a file called create_databases.sql:

create database mydb_development;
create database mydb_test;
create database mydb_production;
grant all on mydb_development.* to 'my_username'@'localhost';
grant all on mydb_test.* to 'my_username'@'localhost';
grant all on mydb_production.* to 'my_username'@'localhost' identified by 'my_password';

MySQL drop databases script

Make a script to drop databases. Copy the following SQL and paste it into a file called drop_databases.sql:

drop database nemos_production;
drop database mydb_development;
drop database nemos_test;

Executing a script from MySQL

The above scripts can be executed from the command-line with the either of these commands:

sudo mysql < create_database.sql
or
sudo mysql < drop_database.sql

Showing MySQL databases

To check that the databases were created :

sudo mysqlshow
or
sudo mysql
show databases;

Displaying database tables

Displaying the tables of a database:

sudo mysqlshow mydb_development
or
sudo mysql
show tables from mydb_development;

Displaying details about a database table

Displaying details about a particular database table:

sudo mysqlshow mydb_development mytable

Droping a database table

Dropping a table from a database:

sudo mysql
use mydb_development;
drop table mytable;

Displaying information about a table

Displaying the fields and field metadata about a table:

show fields from my_table; 
or
desc my_table;

Useful Links

http://openmaniak.com/mysql.php

These are just a few quick and simple MySQL commands that you will always use. Hope this helps.

An Almost Fix for Creating RubyGems on Windows

2008-06-25T00:00:00+01:00

An Almost Fix for Creating RubyGems on Windows

Introduction

I wrote this article originally so that I could write command-line apps using Ruby, as I got further and further down the line I realised that what I really wanted to do was create a command-line app using RubyGems. I use both Windows and an Apple Mac and I quickly found myself having problems trying to create RubyGems on Windows.

In this article I try to explain my thought process and the problems I encountered. Unfortunately I had to employ a manual process when trying to package up my project into a tar file, as there isn't a simple way to do this in Windows (let me know if you have a solution), however everything else seems to work.

What's a RubyGem?

One of the joys of Ruby is the RubyGems packaging system. It is essentially a tool written in Ruby, for packaging up source code for deployment.

The RubyGems site describes it as follows:

RubyGems is the premier ruby packaging system. It provides:

A standard format for destributing Ruby programs and libraries.

An easy to use tool for managing the installation of gem packages.

A gem server utility for serving gems from any machine where RubyGems is installed.

Why should I create a RubyGem?

Maybe a better question should be when should I package my code up into a RubyGem. Obviously there are no hard and fast rules but if you are going to share your code libraries with others or even with other applications of yours, you should think about using a gem.

Now you might think that you don't want to make all your code public on RubyForge, and there are very good reason not to publish every gem, so you don't have to. RubyGems comes with its own local gem server so you can create your own gem library and share it amoung your other Ruby applications.

This is really quite useful, you get the power of RubyGems without the commitment and maintenance work of making your code OpenSource. Plus at any point in time you can then make it OpenSource, and live off the joy of the Ruby community :).

The reason I wrote this article is because I wanted to create a simple command-line application using Ruby. I went through a number of steps before realising that the best way was to use RubyGems.

I started by creating my own command-line structure and then found this great article by Todd Werth. He has a single Ruby file that you just replace various sections and viola you have a very simple command-line app. But it wasn't quite what I wanted. I didn't want to have to type:

./my_super_app 'some params'

I wanted to type:

my_super_app 'some params'

I also wanted the structure of the application to follow a standard structure and I didn't want to put '.' into my path, for security reasons. Dr Nic came to the rescue with a couple of excellent articles describing how to create gems using newgem and specifically command-line apps as gems.

Installing newgem on Windows

To install newgem on Windows use the RubyGems install method.

gem install newgem

This will install the gem and its dependencies. However one of its dependencies is the Hoe gem, and there is a slight issue using it on Windows.

What is hoe?

Hoe is a compliment to rake, it was created by Ryan Davis and provides rake tasks for testing, packaging, and releasing RubyGems. Geoffrey Grosenbach has a nice tutorial for using hoe and also describes how to create a project structure using the sow task.

Hoe uses a .hoerc file for configuration, and this file should reside in the user's home directory, something like:

C:\Documents and Settings\MyUserName

An example .hoerc file should look like the one below. Please note that this is a YAML file and if you copy it from here remember to follow the indentation convention:

---
publish_on_announce: false
exclude: !ruby/regexp/tmp$|CVS|\.svn|_darcs|\.git|log$|local/.*\.rb$|Makefile|.*\.o$|.*\.bundle$|.*\.so$|.*\.dll$/ 
str: ""
"@taguri": tag:ruby.yaml.org,2002:regexp/tmp$|CVS|\.svn|_darcs|\.git|log$|local/.*\.rb$|Makefile|.*\.o$|.*\.bundle$|.*\.so$|.*\.dll$/
blogs:
- user: user
  url: url
  extra_headers:
    mt_convert_breaks: markdown
  blog_id: blog_id
  password: password

On Unix systems there is a shortcut to the user's home directory which is '~' and on Windows this isn't a shortcut. Windows however does have an environment variable HOMEPATH which will point to the user's home directory. The problem is that the Hoe gem code uses '~' in its hoe.rb file and this causes problems for Windows users.

Patching the hoe.rb file

OK, be brave, lets tweak the hoe.rb file so that it will work on the Windows platform. The Hoe gem hoe.rb file can be found in the following directory (assuming you installed Ruby to the standard location):

C:\ruby\lib\ruby\gems\1.8\gems\hoe-1.6.0\lib

To make it work for Windows PCs you will need to modify this file with the following changes.

Locate the define_tasks method:

def define_tasks # :nodoc:
  def with_config # :nodoc:
    rc = File.expand_path("~/.hoerc")
    exists = File.exist? rc
    config = exists ? YAML.load_file(rc) : {}
    yield(config, rc)
  end

And add a new method above it such as this, and modify the line rc = File.expand_path("~/.hoerc") as below:

def get_home_dir
  home_file_path = "~/"
  home_file_path = "#{ENV['HOMEPATH']}\\" if WINDOZE
  home_file_path
end

def define_tasks # :nodoc:
 def with_config # :nodoc:
   rc = File.expand_path(get_home_dir + '.hoerc')
   exists = File.exist? rc
   config = exists ? YAML.load_file(rc) : {}
   yield(config, rc)
 end

Finally you will need to locate and change the task :config_hoe so that it reads:

"signing_key_file" => "#{get_home_dir}.gem/gem-private_key.pem",
"signing_cert_file" => "#{get_home_dir}.gem/gem-public_cert.pem",

If all has gone correctly save this file and you are now ready to create a RubyGem.

Creating a basic RubyGem

Using the newgem RubyGem enter the following command at the command prompt:

newgem my_app -b greetings -T rspec -W

This will create a my_app subdirectory, with a command-line command app called greetings located in the bin directory. It will use the testing framework RSpec and finally will not generate the RubyForge web site code. This is assuming you don't want to relese your code as OpenSource. The output will be something like this:

     create
     create  config
     create  doc
     create  lib
     create  script
     ... blah ...
     create  script/console
     create  script/console.cmd
     create  Manifest.txt
     readme  readme
Important
=========

* Open config/hoe.rb
* Update missing details (gem description, dependent gems, etc.)

If you want to add RSpec Stories to this project there is a useful command-line generator to do this. First navigate into the root of you new project app directory, in this case my_app. Now execute the generate script:

ruby script/generate install_rspec_stories

You should then get the following output:

  create  stories/steps
  create  stories/steps/my_app_steps.rb
  create  stories/sell_my_app.story
  create  stories/all.rb

Modify your greetings command so that it does something. Do this by editing the file located here:

my_app/bin/greetings

Open up this file and at the very end, add some code such as this, and save the file:

# do stuff
puts 'Greetings the world is my oyster!'

Packaging caveat

As mentioned in the begining of this article I couldn't use rake to tar this project up for deployment to RubyForge. The reason is that rake makes a Unix platform dependent call to tar the package using a Ruby task file packagetask.rb located:

C:\ruby\lib\ruby\gems\1.8\gems\rake-0.8.1\lib\rake

This means that on Windows you will have to manually tar this file.

However you can still create the gem using the rake task, and a gem file my_app-0.0.1.gem, will appear in the pkg directory:

rake gem

This new RubyGem can now be installed by using the gem install commands:

gem install pkg/my_app-0.0.1.gem

You should get the following output:

For more information on my_app, see http://my_app.rubyforge.org

NOTE: Change this information in PostInstall.txt
You can also delete it if you don't want it.


Successfully installed my_app, version 0.0.1
Installing ri documentation for my_app-0.0.1...
Installing RDoc documentation for my_app-0.0.1...

Finally you should now be able to execute your command-line command in the command prompt, simply by typing:

greetings

I hope this is useful and helps with your understanding of how RubyGems work, specifically on Windows, also please jot down some comments if you have any ideas or suggestions.

Links

Here is a list of useful RubyGems links:

Understanding RSpec Stories - a Tutorial

2008-06-10T00:00:00+01:00

Understanding RSpec Stories - a Tutorial

When I initially started writing this RSpec guide I had planned to cover both traditional RSpec Specs and the new Stories feature, however the guide quickly became to big so I decided to focus purely on RSpec Ruby Stories, as opposed to RSpec Rails Stories.

Why Unit Testing?

Every pragmatic programmer tells us that we should be writing unit tests, but are they really that important? Well unfortunately they are. Unit test ensure that we write good quality code mitigating the number of bugs we introduce into a software system. However more than that, they help us think about the problem from different perspectives which leads to further insights and gradually makes the domain problems more manageable.

Unit testing is especially important when the language you use is interpreted. With compiled languages like C and C++ for example, the compiler picks up on compile time errors, however with interpreted languages like Ruby, Perl and Python there is no compiler and bugs can easily be introduced which won't be uncovered until the interpreter executes that branch of the code.

Traditionally tests have been written using a Unit Test framework like JUnit, NUnit or RUnit. However it is easy to spending a lot of time writing tests that test every unit of code in your software system, what RSPec does is to provide a subtle shift of focus from Unit testing to Behaviour testing or Behaviour Driven Development (BDD). By focusing on the behaviour of the system it helps clarify in our minds what the system should actually be doing. It also means that our energy is directed at more 'useful' tests. Useful tests, cover what the system should be doing and build in enough redundancy so that it should be easy to refactor our code without having to re-write every test.

What is RSpec? Give me some background.

RSpec is a Behavioural Driven Development (BDD) tool, originally created by Dave Astels and Steven Baker.

However David Chelimsky is really the gatekeeper of the RSpec project, and because it is an OpenSource project you can find the code on the GitHub RSpec Repository. If you would like to use Git to 'check out' a copy of the source code see my article: Installing Git on Apple OSX

RSpec is really two projects merged into one. The RSpec project pages describes these merged projects as:

a Story Framework for describing behaviour at the application level
a Spec Framework for describing behaviour at the object level

Dan North created rbehave which is the Story framework and he describes it as:

rbehave is a framework for defining and executing application requirements. Using the vocabulary of behaviour-driven development, you define a feature in terms of a Story with Scenarios that describe how the feature behaves. Using a minimum of syntax (a few â€œquotesâ€ mostly), this becomes an executable and self-describing requirements document.

The Spec framework was created by David Chelimsky, and is orientated at testing the behaviour of objects in your system.

By encompassing two frameworks RSpec equips a programmer with a thorough set of testing tools, allowing you to think about your software problem from a number of perspectives.

What is an RSpec Story?

Dan North has a good article about what a Story actually is, but here is a quick summary. A Story describes the functionality of a specific software feature, and it describes it in a way that is easy to understand from the point of view of a client. In fact a story should be thought of as a conversation between a client and a programmer over some feature of the software.
As this story unfolds the programmer guides the client by prompting different Scenarios and using specific words Given, When and Then, to capture the essence of the systems behaviour.

A typical story template will take this structure but does not necessarily have to:

Title (one line describing the story)

Narrative:
As a [role]
I want [feature]
So that [benefit]

Acceptance Criteria: (presented as Scenarios)

Scenario 1: Title
Given [context]
  And [some more context]...
When  [event]
Then  [outcome]
  And [another outcome]...

Scenario 2: ...

The key points to a story are:

Title - this should describe an activity or action
Narrative - should include a role, feature and a benefit. "As a [role] I want [feature] so that [benefit]"
Scenario - describes what is different in the story. NB you can have a number of different scenarios.
Scenario: Given/Events/Outcomes - Given [some context], When [I do something], Then [this happens]. It is important to use the Given, When, Then language.
Givens should define all of, and no more than, the required context
Event should describe the feature

RSpec actually has four implementations of a Story, which given the sporadic documentation can lead to much confusion. Essentially from an RSpec point of view a Story consists of three parts:

the Story
Steps that the Story undertakes
Components to be tested, described by the Story

Installing RSpec on Apple OSX

The quickest and easiest way to install RSpec is to use Ruby gems. In the Apple Terminal.app command prompt enter the following command, and install RSpec as root:

sudo gem install -r rspec

It should be noted that according to how you have setup your Mac, you may need to use sudo to install this gem. If all goes well you should get output such as this:

Bulk updating Gem source index for: http://gems.rubyforge.org/
Successfully installed rspec-1.1.4
1 gem installed
Installing ri documentation for rspec-1.1.4...
Installing RDoc documentation for rspec-1.1.4...

Finally if you are an Apple user and are using TextMate you might want to read my article on how to install the TextMate RSpec bundle.

How to Create an RSpec Story

The simplest type of RSpec Story consists of a single file that contains both the Story and the executable Steps, all you need to do is require the library or class you wish to test.
For example if we wish to use RSpec to test an Account object you would create your library file such as account.rb:

class Account
  attr_accessor :balance

  def initialize(amount)
    @balance = amount
  end

  def transfer_to(account, amount)
    @balance = @balance - amount.to_f
    account.balance = account.balance.to_f + amount.to_f
  end
end

Now create your RSpec Story and save this file as account_story.rb with the following content. Note that on Apple OSX systems you seem to have to require the rubygems library:

require 'rubygems'
require 'spec'
require 'spec/story'
require 'account'

Story "transfer to cash account",
%(As a savings account holder
  I want to transfer money from my savings account
  So that I can get cash easily from an ATM) do

  Scenario "savings account is in credit" do
    Given "my savings account balance is", 100 do |balance|
      @savings_account = Account.new(balance)
    end
    And "my cash account balance is", 10 do |balance|
      @cash_account = Account.new(balance)
    end
    When "I transfer", 20 do |amount|
      @savings_account.transfer_to(@cash_account, amount)
    end
    Then "my savings account balance should be", 80 do |expected_amount|
      @savings_account.balance.should == expected_amount
    end
    And "my cash account balance should be", 30 do |expected_amount|
      @cash_account.balance.should == expected_amount
    end
  end

  Scenario "savings account is overdrawn" do
    Given "my savings account balance is", -20
    And "my cash account balance is", 10
    When "I transfer", 20
    Then "my savings account balance should be", -40
    And "my cash account balance should be", 30
  end
end

Note that although you did not explicitly define the code in the Steps of the second Scenario the RSpec framework uses the Given, When, Then methods defined in the first Scenario. In fact if you look at the Story and think of it as a plain old Ruby code file you gradually understand what is happening. The file defines a number of methods Story, Scenario, Given, When and Then, note that when RSpec analyses this Story file it parses the two Scenarios and is able to match the Given, When, Then methods with the other Scenario.

RSpec Plain Text Stories

This previous example gives us the basic understanding of how a Story works and is structured, but what about the RSpec plain text stories, how do I create one and how are they structured?

The goal of a Plain Text Story is to extract the Steps into a separate file and write the Story in such a way that it looks just like plain text with no Ruby code. In fact it is possible to name the Story file such that it does not even have the .rb file extension.

However by extracting the Steps from the Story we will require a helper file to bind the Story to the Steps. Therefore a plain text Story file will consist of four parts:

the Story
Steps that the Story undertakes
Components to be tested, described by the Story
Helper file to bind the Story to the Steps

There isn't any hard and fast rules about how you should structure the file system with respect to Stories and Steps, however best practice seems to describe a stories directory and a steps subdirectory.

stories +
        + steps

Story files are created without the .rb file extension and are placed in the stories directory and consequently a Step file is created with the same name as the Story file, but with _step.rb extension. Helper files can be put in the stories directory, and should have the same name as the story file but this time with the .rb file extension.

Create a Story file with the following file name, bank

This is a story about an transferring money between bank accounts. 
Note that the text up here above the Story: declaration won't be 
processed, so you can write whatever you wish!

Story: transfer to cash account
  As a savings account holder
  I want to transfer money from my savings account
  So that I can get cash easily from an ATM

  Scenario: savings account is in credit
    Given my savings account balance is '100'
    And my cash account balance is '10'
    When I transfer '20'
    Then my savings account balance should be '80'

  Scenario: savings account is overdrawn
    Given my savings account balance is '-20'
    And my cash account balance is '10'
    When I transfer '20'
    Then my savings account balance should be '-40'

Now create a steps file with the following name and place it in the steps directory, bank_steps.rb

require 'rubygems'
require 'spec/story'
require 'spec/story/extensions/main'
require File.join(File.dirname(__FILE__), "../account")

steps_for(:bank) do
  Given("my savings account balance is '$savings'") do |savings|
    @savings_account ||= Account.new(savings)
  end
end

steps_for(:bank) do
  Given("my cash account balance is '$cash'") do |cash|
    @cash_account ||= Account.new(cash)
  end
end

steps_for(:bank) do
  When("I transfer '$amount'")  do |amount|
     @savings_account.transfer_to(@cash_account, amount)
  end
end

steps_for(:bank) do
  Then("my savings account balance should be '$expected_amount'") do |expected_amount| 
     @savings_account.balance.should == expected_amount.to_f
  end
end

The helper file by default is given the same name as the Story file but this time with the .rb extension. Here is an example of such a file, called bank.rb.

#!/usr/bin/env ruby
require 'rubygems'
require 'spec'
require 'spec/story/runner'
require 'account'

# require all the steps files so that we can link the story to them
Dir[File.join(File.dirname(__FILE__), "steps/*.rb")].each do |file|
  require file
end

# execute the steps in the steps file
with_steps_for :bank do
  run File.expand_path(__FILE__).gsub(".rb","")
end

Now to run this story all you have to do is from the Apple Terminal.app navigate to the stories directory and execute the following Ruby command:

ruby bank.rb

If all is successful you should see the following output:

requiring files ./steps/addition_steps.rb
requiring files ./steps/bank_steps.rb
Running 2 scenarios

Story: transfer to cash account

  As a savings account holder
  I want to transfer money from my savings account
  So that I can get cash easily from an ATM

  Scenario: savings account is in credit

    Given my savings account balance is '100'
    And my cash account balance is '10'

    When I transfer '20'

    Then my savings account balance should be '80'

  Scenario: savings account is overdrawn

    Given my savings account balance is '-20'
    And my cash account balance is '10'

    When I transfer '20'

    Then my savings account balance should be '-40'

2 scenarios: 2 succeeded, 0 failed, 0 pending

I hope this gives you a good introduction and a basis for understanding how RSpec Stories work. Finally here are a list of links which may help you further:

An introduction to RSpec - Part I
An introduction to RSpec Stories - Part I
An introduction to RSpec Stories - Part II, although the run_story method now seems to be depreciated in the source code.
An introduction to RSpec Stories - Part III
Quick overview of stories
RSpec Story Runner Creation
Blog describing creating a story and steps
Clear introduction to stories

How to determine whether a column exists on SQLServer

2008-06-10T00:00:00+01:00

How to determine whether a column exists on SQLServer

I run some SQL scripts against Microsoft's SQLServer in a number of environments, test, staging and production.

One of my problems is that these scripts alter the database schema and I only want them to execute if the change hasn't already been made.

One of these changes was to remove a column from the database, so my question is 'how can I ask SQLServer whether a particular column exists'.

Well the SQL query to check if a column exists and execute a query if it does is as follows:

IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.COLUMNS
                 WHERE TABLE_NAME='my_table_name' AND
                 COLUMN_NAME='my_column_name' )
BEGIN
  PRINT 'Execute some query here'
END

Installing the Latest RSpec TextMate Bundle

2008-06-04T00:00:00+01:00

Installing the Latest RSpec TextMate Bundle

If you are using TextMate on your Apple Mac you may wish to download the RSpec TextMate Bundle and install it. The TextMate bundle can be found in the RSpec Git repository. In your Terminal.app enter the following commands. Please note that you will need Git installed on your Apple Mac in order to do this, please see my article Installing Git on Apple OSX on how to do this:

cd /Applications/TextMate.app/Contents/SharedSupport/Bundles
git clone git://github.com/dchelimsky/rspec-tmbundle.git RSpec.tmbundle

You should get output like this in your Terminal window:

Initialized empty Git repository in /Applications/TextMate.app/Contents/SharedSupport/Bundles/RSpec.tmbundle/.git/
remote: Counting objects: 46199, done.
remote: Compressing objects: 100% (10514/10514), done.
Indexing 46199 objects...
remote: Total 46199 (delta 33049), reused 46199 (delta 33049)
 100% (46199/46199) done
Resolving 33049 deltas...
 100% (33049/33049) done

Now you will need to reload your TextMate Bundle Editor for this new bundle to be applied. In TextMate select the following menu items and button:

  Bundles > Bundle Editor > Reload Bundles

If you ever need to get a more uptodate version of the RSpec.tmbundle then simply navigate to your TextMate Bundle directory and use Git to pull the latest version down to your machine. Don't forget to reload your bundles once this has been done. To pull the latest version from the Git hub use this command:

git pull

For older instructions on how to do this see Robby on Rails - RSpec Bundle for TextMate

Installing Sqlite3 on Windows for Rails

2008-06-04T00:00:00+01:00

Installing Sqlite3 on Windows for Rails

The Windows version of Ruby on Rails does not ship with the Sqlite3 database, even though the database.yaml configuration file is preconfigured to use Sqlite.

This is a how to guide on how to install Sqlite3 on your Windows PC. This article assumes that you already have Ruby and Ruby on Rails installed on your PC.

First you will need to download two files from the Sqlite web site http://www.sqlite.org/download.html:

sqlite-3_5_9.zip (214.32 KiB)
A command-line program for accessing and modifing SQLite databases. 
See the documentation for additional information.

sqlitedll-3_5_9.zip (213.17 KiB)
This is a DLL of the SQLite library without the TCL bindings. 
The only external dependency is MSVCRT.DLL.

The first file is the Sqlite command line program used for modifing the Sqlite database. You may or may not use this.

The second file is the Windows DLL library file and Ruby uses this when Rails makes Sqlite database calls.

When both these ZIP files have been extracted you should have the following files:

sqlite3.exe
sqlite3.def
sqlite3.dll

Copy these file to the bin directory of your Ruby installation, if you followed the default Ruby installation it will be located here:

    C:\ruby\bin

Now that you have the Sqlite3 files installed you need to tell Ruby how to use them. To do this you need to download the Ruby bindings for Sqlite3.
Fortunately this is easy to do, using Ruby gems. Simply at the command prompt type the following command:

    gem install sqlite3-ruby

You will now need to tell Gems which version you need as you will be presented with the following output:

    Bulk updating Gem source index for: http://gems.rubyforge.org
    Select which gem to install for your platform (i386-mswin32)
     1. sqlite3-ruby 1.2.2 (mswin32)
     2. sqlite3-ruby 1.2.2 (ruby)
     3. sqlite3-ruby 1.2.1 (mswin32)
     4. sqlite3-ruby 1.2.1 (ruby)
     5. Skip this gem
     6. Cancel installation
    >_

Please select option 1, sqlite3-ruby 1.2.2 (mswin32). All being successful you will get some output like this:

    Successfully installed sqlite3-ruby-1.2.2-mswin32
    Installing ri documentation for sqlite3-ruby-1.2.2-mswin32...
    Installing RDoc documentation for sqlite3-ruby-1.2.2-mswin32...

If you are using Rails 2+ you should be able to run the following rake tasks from your Rails application directory. For example say you created a Rails application located here: C:\MyApp you should be able to execute:

    C:\MyApp>rake db:create
    or
    C:\MyApp>rake db:migrate

Hope this is useful.

Installing Git on Apple OSX

2008-06-03T00:00:00+01:00

Installing Git on Apple OSX

Git is a version control system like CVS and Subversion. It was created by Linus Torvalds and used for Linux. It has recently been adopted by the Ruby community and many of the new Ruby and Ruby on Rails projects now use it. One of the reason's for it's popularity amongst the Ruby community is because of github. In there own words:

GitHub is the easiest (and prettiest) way to participate in that collaboration: fork projects, send pull requests, monitor development, all with ease.

How to quickly setup Git on Apple OSX

The simplest way to install Git on Apple OSX (for Intel Mac) is to use this pre-packaged installer: http://metastatic.org/text/Concern/2007/08/18/git-package-for-mac-os-x/, however if you have a PowerPC Mac or wish to use an alternative installation method, try the information on this site: http://bc.tech.coop/blog/070827.html

Launch the installer, and accept the license agreements. Once the installer has completed open up OSX Terminal.app (cmd + space, then type terminal), and type git at the command prompt, if all has been successful you should see something like this:

usage: git [--version] [--exec-path[=GIT_EXEC_PATH]] [-p|--paginate|--no-pager] [--bare] [--git-dir=GIT_DIR] [--work-tree=GIT_WORK_TREE] [--help] COMMAND [ARGS]

The most commonly used git commands are:
   add           Add file contents to the index
   apply         Apply a patch on a git index file and a working tree
   archive       Create an archive of files from a named tree
   bisect        Find the change that introduced a bug by binary search
   branch        List, create, or delete branches
   checkout      Checkout and switch to a branch

...  continued with more commands ...

Configuring Git

One of the interesting things about Git is that it can be configured from the command line. Therefore the first thing to do is setup your user details so that when you add changes it uses your details. This web site has some very good instructions on how to get up and running using Git very quickly http://git.or.cz/course/svn.html.

To configure your Git user details enter the following at the command prompt:

git config --global user.name "Your Name Comes Here"
git config --global user.email you@yourdomain.example.com

Clone / Checkout the RSpec Git Repository

Now you should be able to clone (Subversion calls this checkout) a repository so that you can view the source code. To do this enter the following command at the command prompt. I intend to checkout the RSpec repository from github:

git clone git://github.com/dchelimsky/rspec.git

All being well you should see output in your Terminal like this:

Initialized empty Git repository in /Users/myname/my_repositories/rspec/.git/
remote: Counting objects: 46810, done.
Compressing objects: 100% (10646/10646), done.)   
Indexing 46810 objects...
remote: Total 46810 (delta 33521), reused 46810 (delta 33521)
 100% (46810/46810) done
Resolving 33521 deltas...
 100% (33521/33521) done

Now you should have all the source code in your new repository.

Named Pipes and TCP/IP Connection Strings

2008-05-28T00:00:00+01:00

Named Pipes and TCP/IP Connection Strings

Manually Creating a SQL Connection String

There are many different connection strings for connecting to a SQLServer database for C#. Have a look at this web site http://www.connectionstrings.com/?carrier=sqlserver for examples.

I needed to create a connection to a SQLServer and started using this as my connection string:

server=myServer;database=myDatabase;uid=someUser;pwd=somePassword

However I got the following error:

An error has occurred while establishing a connection to the server. 
When connecting to SQL Server 2005, this failure may be caused by
the fact that under the default settings SQL Server does not allow 
remote connections. (provider: Named Pipes Provider, error: 40 - 
Could not open a connection to SQL Server)

Pipes are actually quite an old technology, but what are they?

What are Named Pipes on Windows

MSDN defines a pipe as:

A named pipe is a named, one-way or duplex pipe for communication between the pipe server and one or more pipe clients.

Pipes are like sockets and allow peer-to-peer communication. Additionally any process can act as either a server or a client. Pipes can be accessed remotely but you can also deny access to it from other networked authorities.

Really we should use TCP/IP for our database connections and initially I thought I had to create a special connection string defining TCP/IP settings.

Again as I looked into the problem more closely it turned out that I was again connecting to a server that was not running SQLServer. Anyway I am now using the following connection string:

Data Source=myServer;Database=myDatabase;User ID=someUser;Password=somePassword;

List of Apple OSX Terminal Commands

2008-05-28T00:00:00+01:00

List of Apple OSX Terminal Commands

This web site has a list and description of a large amount of Apple OSX Terminal commands:

http://www.ss64.com/osx/

IManage error - The semaphore is set and cannot be closed

2008-05-28T00:00:00+01:00

IManage error - The semaphore is set and cannot be closed

Using the Interwoven IManage API, I was trying to create a connection to the IManage server and kept getting this error message:

The semaphore is set and cannot be closed.

It turns out that this error message is misleading and can hide a multitude of symptoms. In this case it was returned because the system cannot create a connection to the server, as I had accidentally put in an incorrect server name into my connection method.

I did however do some research into what a semaphore actually is. The choice of word comes from a time when flags were used to signal telegraph messages.

However the Dictionary computer definition of a semaphore is:

semaphores are a software mechanism to regulate process synchronization, proposed by Edsger W. Dijkstra.

Wikipedia gives a more detailed explaination of a semaphore, see below:

A semaphore, in computer science, is a protected variable (an entity storing a value) or abstract data type (an entity grouping several variables that may or may not be numerical) which constitutes the classic method for restricting access to shared resources, such as shared memory, in a multiprogramming environment (a system where several programs may be executing, or taking turns to execute, at once). Semaphores exist in many variants, though usually the term refers to a counting semaphore, since a binary semaphore is better known as a mutex. A counting semaphore is a counter for a set of available resources, rather than a locked/unlocked flag of a single resource. It was invented by Edsger Dijkstra. Semaphores are the classic solution to preventing race conditions in the dining philosophers problem, although they do not prevent resource deadlocks.

Rapid Apps Group for Enterprise 2.0

2008-05-23T00:00:00+01:00

Rapid Apps Group for Enterprise 2.0

Welcome to the first post of my blog. It is actually my third attempt at writing a blog so hopefully this will be third time lucky.

I have also started a company with a colleague and you can see our web site:

http://www.rapidappsgroup.com

This company already has a number of software products revolving around Web 2.0 for the Enterprise and SMS services.

Anyway please have a look and let me know what you think.

.Net not recognising regional settings

2008-05-23T00:00:00+01:00

.Net not recognising regional settings

The other day I had the following problem. One of my test servers had been setup with US regional settings and should have been setup with UK settings.

The problem had been unnoticed for a while until our C# .Net application tried to call a stored procedure on a database on another server. This database server had UK regional settings. The stored procedure expected to get a date in UK 'Long format'. The error message was a database error saying something like:

convertion error can't convert from NVarChar to DateTime

As the application was already compiled and being used on a live server I could not modify the code, I therefore had to run SQL Profiler and see what values were being passed into the stored procedure. This is where I saw the US 'Long Date' format being passed in.

.Net regional settings fix

So I tried changing the regional settings on my server back to the UK, but the application was still sending the US 'Long Date' value. I dug real deep and suspected that IIS was doing something funny and found further regional settings in an Advanced tab, but still this didn't fix the issue.

The problem I soon discovered is that when you install the .Net framework it internalises the regional settings but when you try and change them on the server these internalised settings don't get changed.
The only solution to this is then to modify the machine.config file.

The machine.config file is in each installation of the .Net framework. I happen to have both .Net 1.1 and .Net 2.0, however I changed the machine.config file in only the .Net 1.1 installation directory (even though our application is .Net 2.0) and all worked fine. The machine.config can be found in this directory:

C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\CONFIG

If you are using .Net 2.0 I'm sure you can make the same changes to the globalization XML node. Please change the globalization XML node to have the correct cultural settings in our case en-GB:

<globalization
  fileEncoding="utf-8"
  requestEncoding="utf-8"
  responseEncoding="utf-8"
  culture="en-GB"
  uiCulture="en-GB"
/>

To test that a .Net application would send the correct settings I wrote a simple test app which would print the 'Long Date' to the console. Here is the source code if you want to quickly test it yourself:

ShowLongDate.cs

using System;
    
namespace examples
{
    class ShowLongDate
    {
        static void Main(string[] args)
        {
            DateTime current = DateTime.Now;
            string longDate = current.ToLongDateString();
            System.Console.WriteLine(string.Format("Date as longDate: {0}", longDate));
        }
    }
}

Interesting things I learned

You can open the regional settings from the commandline or from Start/Run by typing

    intl.cpl

Also all the regional settings are located in the registry here:

    HKEY_CURRENT_USER/Control Panel/International

Useful URLs: http://www.microsoft.com/globaldev/reference/localetable.mspx

Anyway I hope this helps if you have the same problem.