Mendable.com

Compiling Power DNS on Snow Leopard

2010-09-11T00:00:00-07:00

This is how to install PowerDNS on Mac OSX Snow Leopard 10.6.4.

1) Install 64-bit Mysql server from the MySQL dmg package available from the MySQL website

2) Install the boost c++ library by ports (at time of writing this is boost version 1.44)

sudo port install boost

3) Download the authoritative server tar.gz file from [powerdns.com][], and unpack.

4) Build and Install.

CXXFLAGS=-DDARWIN CPPFLAGS=-I/opt/local/include ./configure \
 --with-mysql-includes=/usr/local/mysql/include/
make
sudo make install

5) Install a default config file, examples within the package.

sudo cp /usr/local/etc/pdns.conf-dist /usr/local/etc/pdns.conf

6) Edit /usr/local/etc/pdns.conf to taste. To setup the MySQL backend, you will need to add the following to the file:

----
launch=gmysql
gmysql-host=127.0.0.1
gmysql-user=root
gmysql-dbname=power_dns
----

7) Create the database and database schema. You can find it here.

8) Start the server.

There is an init script file in "pdns/pdns" in the tar.gz package after compiling that you can use to stop/start the server, you must have root privileges (ie, use sudo) to start/stop the server because the daemon listens on ports < 1024.

You can find server log output in /var/log/system.log.

New Project Launched - ChatUp.com

2010-06-06T00:00:00-07:00

I have launched a new Ruby on Rails website, ChatUp.com. This is a personal project that i've been working on in my spare time with a little help from my girlfriend, and it's great to finally have it out there.

I've got some really cool stuff integrated, including Facebook authentication using the new OAuth2 API and HTML5 GeoLocation. Doing GeoLocation well and having it work for any user of the site turned out to be a bit tricky, i might post more on that later. Back end searching is done using the latest Sphinx search daemon and ThinkingSphinx plugin.

There are still 101 things I want to improve, i'm working on getting a new design for the blog sorted right now, but as an initial version 1 it does all the things you would expect of any dating site.

I'm going to be writing about my efforts to build out the site further, and probably more importantly how I actually am getting traffic to the website (let's face it, that's one of the hardest things to do, programming is easy).

If you want to follow the project along, please check out the ChatUp Blog and grab the RSS feed.

Please check it out, Feedback welcome!

Thinking Sphinx Performance - Split your re-indexing into seperate tasks

2010-06-03T00:00:00-07:00

If you have multiple indexes, you need to consider if they all need to be re-indexed at the same time.

For example, you might have a "users" index that changes frequently that you want to re-index from scratch or merge delta changes, and a "spare car parts" index that may only change once or twice a day.

The accepted way to perform a re-index your data is of course to run "rake ts:index" which will re-create ALL of your indexes, but that assumes all indexes are equal and all need re-indexing at the same time - usually that is not true. When you have to re-index a huge index when all you really want is to update the smaller one... it's inefficient.

So, what does rake ts:index actually do?

The thinking sphinx index task works out what config file to use based on your rails environment and Rails.root directory, re-generates the config file, and then calls the sphinx indexer program with a link to the config file.

The indexer program has a whole range of options which you might like to get familiar with:

$ indexer
Sphinx 0.9.9-release (r2117)
Copyright (c) 2001-2009, Andrew Aksyonoff

Usage: indexer [OPTIONS] [indexname1 [indexname2 [...]]]

Options are:
--config      read configuration from specified file
            (default is sphinx.conf)
--all           reindex all configured indexes
--quiet         be quiet, only print errors
--noprogress        do not display progress
            (automatically on if output is not to a tty)
--rotate        send SIGHUP to searchd when indexing is over
            to rotate updated indexes automatically
--buildstops  
            build top N stopwords and write them to given file
--buildfreqs        store words frequencies to output.txt
            (used with --buildstops only)
--merge  
            merge 'src-index' into 'dst-index'
            'dst-index' will receive merge result
            'src-index' will not be modified
--merge-dst-range   
            filter 'dst-index' on merge, keep only those documents
            where 'attr' is between 'min' and 'max' (inclusive)
--merge-killlists           merge src and dst killlists instead of applying src killlist to dst
Examples:
indexer --quiet myidx1  reindex 'myidx1' defined in 'sphinx.conf'
indexer --all       reindex all indexes defined in 'sphinx.conf'

We are interested in indexing only specific indexes at specific times, following from the example above, the "spare car parts" index, instead of running rake ts:in from cron to re-index everything every hour (for example), you can split those indexes out to separate tasks like this:

indexer --config /var/www/your-website.com/current/config/production_slave.sphinx.conf spare_car_parts_core --rotate
indexer --config /var/www/your-website.com/current/config/production_slave.sphinx.conf users_core --rotate

Put those into cron with your desired time schedule, and you will have a happier database server. Important to note is that you must include the _core part, as this is how the ThinkingSphinx gem names the index in the sphinx configuration.

When you deploy remember to run "rake ts:config" with the correct Rails environment to generate your config.

Thinking Sphinx Performance - How to Index from Slave MySQL Database

2010-05-31T00:00:00-07:00

For one of the sites I work with, we are running a load balanced web server setup with separate MySQL database servers in master-slave configuration on the back end. Each of the web servers run their own Sphinx searchd daemon (to reduce latency of client connection/queries) using a non-distributed sphinx index.

While looking at our Munin graphs I was concerned to see the amount of bandwidth and disk IO on the master MySQL database server caused by periodic re-indexing, and that load increases in line with the number of front-end web servers currently in use.

So that got me thinking... these are huge read-only queries, a small time lag is acceptable, they should be happening on the Slave MySQL database.

ThinkingSphinx does not have any official documentation for how to configure this, so here is the secret to setup the Sphinx indexer to use your slave MySQL database...

1/. Create a Read Only MySQL User On your Master MySQL server create a read-only MySQL user if you do not have one already. It will be replicated automatically onto the slave servers. This is just good practice, you do NOT want any changes accidentally made on your slave databases or you run the risk of breaking MySQL replication.

2/. Create new production_slave environment We will use a separate Rails environment to hold our slave database and sphinx configuration, but you need to get that environment working. To do this in your rails project, copy config/environments/production.rb to config/environments/production_slave.rb.

3/. Configure database.yml In your database yml, create a new entry for your production_slave environment, pointing to your slave MySQL database, and use your read-only MySQL user.

4/. Edit your config/sphinx.yml file Take a copy of your production section and duplicate it under production_slave. It is important that you use the same port number and settings of the production environment.

5/. Commit your changes and deploy live Push our your code to the servers, so your servers now have access to your new production_slave environment. Edit your yml configs on the servers if your deploy process does not mange this for you.

6/. Setup Sphinx Indexer for new Environment The sphinx indexer should now use the RAILS_ENV=production_slave. As this is the first time you should now run "RAILS_ENV=production_slave rake ts:in" which will automatically generate a new config/production_slave.sphinx.conf file for you that searchd and indexer can use, and generate your first set of indexes in db/sphinx/production_slave/.

7/. Restart Sphinx Stop your production sphinx instance, and restart sphinx with the production_slave environment. It needs to read the same config file as your indexer in case of future schema changes etc.

8/. Update RAILS_ENV for sphinx elsewhere This will largely be dependent on your setup, but you need to change your cron-jobs, init.d scripts, deploy scripts, process monitoring etc to use the production_slave environment for your sphinx daemon and sphinx indexing tasks.

That's it!

The reason this works it because of step #4, where we use the same host & port for our sphinx server for both the production and production_slave environments. Although your app continues to run as production, your sphinx server runs as production slave on the port expected in the production environment so you are now querying the sphinx daemon that is indexed from the slave MySQL database.

uninitialized constant ThinkingSphinx::Deltas::Job

2010-04-18T00:00:00-07:00

If you are getting this error during a sphinx index, it is likely because you have a copy of the gem 'thinking-sphinx' installed, but you also have an older version of the gem from the old github repository also installed (eg, freelancing-god-thinking-sphinx).

The solution is to remove the old github gem version.

Also this should be a reminder that github will stop hosting gems in October 2010, so you need to allocate time to switching all of your gem dependencies be running gems from github to their respective rubygems/gemcutter versions.

FFMpeg SWF File Conversion

2010-04-18T00:00:00-07:00

FFMpeg has multiple problems and bugs with writing out to SWF files, I run into a few of these at my day-job and wrote a patch for one of the problems and submitted it back to FFMpeg. It was a nice change to go back and do some C programming for a while.

Anyway, list of ffmpeg swf audio problems as follows: 1) Converting to SWF was hard coding the number of audio frames to 6000 in the generated SWF file, whereas the SWF Format specification document from adobe says this must match the number of frames in the file. It was also hard-coding an arbitrary file size into the SWF header as well, rather than the correct file size of the generated file. These are the problems I fixed.

2) FFMpeg SWF encoder writes too many frames to a file. Yeah, really. It writes all of your audio, then a few minutes of empty sound as well. Haven't got a fix for it yet, need to step through an FFMpeg run in GDB to figure out why it's doing it. You can open a SWF in a binary editor and manually fudge the number of frames in the SWF Header if you have a scrobbler that is dependent upon the number of frames to determine play duration / progress.. or do something a little more automated.

3) The SWF encoder in FFMpeg is writing out audio only streams using version 4 of the SWF file format, where as the file format specification is now up to version 10. Not sure why it's writing out in such an old version (maximum compatibility?). Could probably use some documentation.

This was the patch I submitted for problem #1.

http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100415/c02bf648/attachment.obj

On the mailing list:

Problem Description: When converting any Audio to SWF, the File Size is always hard-coded as 104857600, and Frame Count is always hard coded to 6000. The incorrect frame count can cause problems when ffmpeg generated swfs are used inside a flash application and the _totalframes method is used.

Example command line:

ffmpeg -i 157346.mp3 -ar 44100 -ab 96k -ac 1 -y 157346.swf

When inspecting the generated swf using the swfdump tool (swftools.org), the following is shown:

==== Error: Real Filesize (1495887) doesn't match header Filesize
(104857600) ====
[HEADER]        File version: 4
[HEADER]        File size: 104857600
[HEADER]        Frame rate: 10.000000
[HEADER]        Frame count: 6000
[HEADER]        Movie width: 320.00
[HEADER]        Movie height: 200.00
[02d]         6 SOUNDSTREAMHEAD2
[013]       317 SOUNDSTREAMBLOCK
[001]         0 SHOWFRAME 1 (00:00:00,000)
[013]       318 SOUNDSTREAMBLOCK
[001]         0 SHOWFRAME 2 (00:00:00,100)
.........
[013]       318 SOUNDSTREAMBLOCK
[001]         0 SHOWFRAME 4595 (00:07:39,382)
[013]       317 SOUNDSTREAMBLOCK
[001]         0 SHOWFRAME 4596 (00:07:39,482)
[000]         0 END

As you can see, the last frame is 4596, and the file size specified in the header does not match the real file size (swfdump gives a warning about this).

File size should indicate 1495887 bytes:

$ ls -al 157346.swf
-rw-r--r-- 1 username default 1495887 Apr 14 11:25 157346.swf

Problem Solution: Correctly set the File Size in the SWF header section to the real number of bytes written, and correctly set the number of frames included in the SWF.

Details of the SWF file format header information can be confirmed on page 25 of the SWF File Format Specification document from Adobe, available here: http://www.adobe.com/devnet/swf/

After my patch is applied, using ffmpeg with the same command line options as above and then inspecting the generated swf with swfdump shows:

[HEADER]        File version: 4
[HEADER]        File size: 1495887
[HEADER]        Frame rate: 10.000000
[HEADER]        Frame count: 4596
[HEADER]        Movie width: 320.00
[HEADER]        Movie height: 200.00
[02d]         6 SOUNDSTREAMHEAD2
[013]       317 SOUNDSTREAMBLOCK
[001]         0 SHOWFRAME 1 (00:00:00,000)
[013]       318 SOUNDSTREAMBLOCK
[001]         0 SHOWFRAME 2 (00:00:00,100)
.........
[013]       318 SOUNDSTREAMBLOCK
[001]         0 SHOWFRAME 4595 (00:07:39,382)
[013]       317 SOUNDSTREAMBLOCK
[001]         0 SHOWFRAME 4596 (00:07:39,482)
[000]         0 END

Note that the Frame Count is correct, and the file size is correctly reflects the total number of bytes in the file:

$ ls -al 157346.swf
-rw-r--r-- 1 username default 1495887 Apr 15 06:58 157346.swf

Programmer Philosophy

2010-04-09T00:00:00-07:00

I write code for a living. Even when I am writing code by myself, there are actually three people working with that code.

First, there is me. I am a clever and talented programmer. I know to keep my code clean and easy to read. I try to do things the right way and I know that if old code is wrong that I am better off fixing it now then trying to code around it.

Second, there is past me. This guy is a fracking moron. He pisses me off on a daily basis as I am having to constant re-write his code. He is generally clueless and causes me more headaches than anyone else.

Finally there is the future me. This guy is smarter than I am, has seen more than I have and has done more. My goal, in addition to making sure my code works, is to make sure that my intentions are clear for him so that I don’t piss him off too badly.

When I claim to be a moron, it is a matter of perspective. Compared to my 8 year old self, I am brilliant. Compared to myself in 10 years; not so much. Therefore, it is safe to say that anything I wrote in the past was written by someone dumber than I am.

I came across those words of wisdom here - very eloquently expressed, but really focuses in on the fact that as programmers we are always improving.

Polymorphic Single Table Inheritance (STI)

2010-03-30T00:00:00-07:00

Migration:

  create_table :addresses do |t|
    ....
    t.references :addressable, :polymorphic => true
    t.string :type
    t.timestamps
  end

Address Models:

class Address < ActiveRecord::Base
    belongs_to :addressable, :polymorphic => true
end
class InvoiceAddress < Address
end
class DeliveryAddress < Address
end

User Model:

class User < ActiveRecord::Base
  has_many :addresses, :dependent => :destroy
  has_one :invoice_address, :as => :addressable
  has_one :delivery_address, :as => :addressable
end

Order Model:

class Order < ActiveRecord::Base
  has_many :addresses, :dependent => :destroy
  has_one :invoice_address, :as => :addressable
  has_one :delivery_address, :as => :addressable
end

Get Funky:

>> u = User.new
=> <User id: nil ...>
>> u.build_invoice_address
=> #<InvoiceAddress id: nil, addressable_id: nil, addressable_type: "User", 
created_at: nil, updated_at: nil, type: "InvoiceAddress">
>> u.build_delivery_address
=> <DeliveryAddress id: nil, addressable_id: nil, addressable_type: "User", 
created_at: nil, updated_at: nil, type: "DeliveryAddress">
>> o = Order.new
=> #<Order id: nil ...>
>> o.build_invoice_address
=> #<InvoiceAddress id: nil, addressable_id: nil, addressable_type: "Order", 
created_at: nil, updated_at: nil, type: "InvoiceAddress">

Sweet :)

Quick and easy static pages in Rails

2010-02-10T00:00:00-08:00

Sometimes you have a Rails app, the core functionality is not a CMS, and you need to add some static pages to your app that don't get changed often, eg, an about us page, privacy policy, etc.

This post shows you a quick and easy way to do that in about 60 seconds, without using a database model or having to write any sort of CMS.

1) First off, add some routes for your static pages. Here I want an FAQ page, About page, Privacy page, and Terms and Conditions pages.

map.with_options :controller => 'static_pages', :action => 'show' do |static_page|
  static_page.faq 'faq', :id => 'faq'
  static_page.about 'about', :id => 'about'
  static_page.privacy 'privacy', :id => 'privacy'
  static_page.terms_and_conditions 'terms_and_conditions', :id => 'terms_and_conditions'
end

2) Add a functional test so we know it works now, and we can check we don't retrospectively break it later. Create a new file in test/functionals/static_pages_controller_test.rb:

require File.dirname(__FILE__) + '/../test_helper'

class StaticPagesControllerTest < ActionController::TestCase

  should_route :get, '/privacy', :action => 'show', :id => 'privacy'
  should_route :get, '/terms_and_conditions', :action => 'show', :id => 'terms_and_conditions'
  should_route :get, '/faq', :action => 'show', :id => 'faq'
  should_route :get, '/about', :action => 'show', :id => 'about'

  %w(privacy about terms_and_conditions faq).each do |static_page_template|
    context "Accessing the #{static_page_template} page" do
      setup do
        get :show, :id => static_page_template
      end

      should_respond_with :success
      should_render_template static_page_template
    end
  end
end

This is fairly simple but covers everything we need it to. It just checks your routing is setup and the pages render from the static template files correctly.

3) Create your controller, app/controllers/static_pages_controller.rb:

class StaticPagesController < ApplicationController
  
  def show
    page = params[:id]
    render :template => "static_pages/#{params[:id]}"
  end

end

Again, really simple. Use a RESTful 'show' method to select a template to render, then render it.

4) from a shell, make a directory for your pages to live in, and create the empty files:

$ mkdir app/views/static_pages
$ touch app/views/static_pages/about.html.erb
$ touch app/views/static_pages/faq.html.erb
$ touch app/views/static_pages/privacy.html.erb
$ touch app/views/static_pages/terms_and_conditions.html.erb

5) Check it works:

$ ruby test/functional/static_pages_controller_test.rb 
Loaded suite test/functional/static_pages_controller_test
Started
............
Finished in 0.156958 seconds.

12 tests, 24 assertions, 0 failures, 0 errors

Done! Now put the content in your pages, test, check in and deploy.

Rails 3 Beta is out - check out Rails Upgrade plugin

2010-02-07T00:00:00-08:00

In case you haven't heard, Rails 3 is now in Beta.

Just a quick post to say, if you haven't already, check out the Rails upgrade plugin, this will quickly show you what bits of your code need to be updated to work with Rails 3.

Also, check out the full release notes, lots of juicy stuff in there.

Sometimes the best way to create is to remove

2010-02-06T00:00:00-08:00

Over the life cycle of a project many features may be developed, sometimes just to account for one variation or tiny use case, get used once, and are then not used again for the next 5 years (if ever).

One of my personal pet peeves is overly complicated software with too many redundant, old, unused features. This sort of redundant complexity increases the learning curve for new developers coming into the software project and creates extra work in maintaining the software,

I often think the best way to create and maintain software, is to periodically remove all the old crap not being used any more. Some people do not like the idea of losing a feature in their application, as in "OMG, you can't take that away, I was planning to use that one day", and maybe they were. But often, you have to look at what they say they are going to do, and compare that with what they actually do.

I am sure this is some sort of psychological effect where the fear of losing a feature has more of an emotional impact on the end user / client than the prospect of faster software development and the addition of new features, as well as cheaper maintenance.

If the feature works, can't it just stay?

Hypothetically yes, but sometimes the part the end user sees is that the old features only continue to work because when adding new features we are able to test we have not broken the old features, and in some cases that means a ton of work when a big structural change is occurring.

Also consider the development of a new software project, or a big addition to an existing project. The number of proposed features grows exponentially with the number of people involved. In reality, only 20% of the features will make 80% of the software, but you may end up coding 80% of the features to make up 90% of the functionality, just because people don't get agile, or the business structure around you does not allow for it to happen so easily (extensive budgeting approvals, waterfall methodology), or of the HiPPO effect (Highest paid person's opinion).

This post has been a bit of a rant, but the points are these: * Don't develop crap you don't need * Remove old crap you don't use any more

Do points 1 & 2 and you will get: * Faster software development * Software that is easier to maintain * Easier to bring in new developers to a project * Happier developers * Reduced costs all round

Decrypt/Reverse MD5 / SHA1 Password Hashes

2010-01-15T00:00:00-08:00

There is a nice little tool over on tmto.org that allows you to decrypt (or should I say, reverse) MD5 or SHA1 hashes. Essentially, someone has forward hash'ed all a-z letters, put them in a database, then matches your hash against what is in the DB to find what the original string was. This is somewhat limited, it only currently does a-z by the looks of things, but a very nice idea. Should also re-enforce the benefits of double-salting your passwords with a per-size and per-user based salt as well.

Installing AutoTest

2009-08-06T00:00:00-07:00

I think autotest is one of the best things about developing in Rails. If you don't know, autotest runs in the background watching all of your application files. When you edit any of them, it automatically runs the sections of your test suite that apply to the modified file, and can then give you feedback as to if your tests are passing or failing. This means more time developing, less time switching windows to run rake test!

I thought I would post a note about how to correctly install autotest, as some of the other documentation is out of date.

sudo gem uninstall ZenTest
sudo gem install ZenTest
sudo gem install autotest-rails

Something that is missing in a lot of other online documentation is the fact that you have to install the autotest-rails gem as well, if you don't do this, you will get an error any time you try to run autotest:

Autotest style autotest/rails doesn't seem to exist. Aborting.

Another note, if you want to get desktop notifications working, this page has details of how to achieve that using Mac OSX, KDE, Gnome, etc.

Wireframing your Rails Apps

2009-07-25T00:00:00-07:00

A really useful way to mockup your applications before coding is to use wire framing software to create the screens and layout of your application. By doing this, you can ensure everyone working on the project can help to decide upon what the first version should functionally look like, and everyone will know what the end goal should approximately look like.

With this sort of discussion, you really need a tool that will just let you focus on the functional aspect, and avoid too much discussion about colors, branding, etc, all of which can come later.

A really useful tool I have been using is Balsamiq Mockups, it allows you to quickly create screens for your app and includes all the standard page controls you would expect. Here is an example of what you can produce:

There is a nice video demo on the site as well.

Is your conversion rate good?

2009-07-11T00:00:00-07:00

Something I have been asked a few times by online retailers is, "how does my conversion rate compare to others in my industry?". Speaking to one online retailer a few weeks ago, they specifically mentioned having access to lots of data from Google analytics, but they have no idea if they are doing better or worse than their competitors or others in their industry.

If you work with E-Commerce, check out the Fireclick Index which publishes a list of conversion rates for different types of e-commerce websites each month, and that can offer some sort of guide as to average conversion rates, and how you are performing.

You should of course be aiming to out perform the average, as this makes your advertising cheaper (lower customer acquisition costs), can result in a higher absolute number of customers, which can result in a higher absolute number of repeat-sales.

Google Analytics and Rails in 60 Seconds

2009-07-08T00:00:00-07:00

This quick post will show you how to install Google Analytics into your Rails powered website in less than 60 seconds.

Installing Google Analytics into your Ruby on Rails website is very easy. First of all, ensure you are signed up for Google Analytics if you are not already.

Once you have your Analytics account setup and you are logged in, you should be able to see the website for which you are tracking statistics, and the tracking ID (will be in the format of "UA-nnnnnn-n". You will need this now.

$ mkdir app/views/common

Add this code to app/views/common/_google_analytics.html.erb:

<script type="text/javascript">

  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-XXXXX-X']);
  _gaq.push(['_trackPageview']);

  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
  })();

</script>

(the latest code can be obtained from here)

You need to replace the "UA-XXXXX-X" in that code with your own tracking ID. Save and close.

Rails allows you to easily keep your website design easy to manage, by allowing you to use standardized site layout templates for your entire website (application.html.erb) or for specific controllers or even actions. These files are stored in the app/views/layouts/ directory. One at a time, open each of the files in this directory, and add the following to the end of the file, just before the closing "<>/body>" HTML tag:

<%= render :partial => 'common/google_analytics' %>

All done!

If you want an alternative method of inserting your Google Analytics code into your application, there are a few existing plugins that will do the work (what work?) for you. Personally, for such a trivial task, why bother with a plugin? Using plugins all of the time is not the right approach, I would even consider it an anti-pattern of sorts. It creates a whole new set of maintainability problems (what if rails changes the way it works, and those plugins no longer work after upgrade? What if the Google Analytics code needs to be updated, but the plugin is no longer being maintained, but you had no idea that was the case? In every case, the decision to use plugins or not can only be decided after weighing up the pro's and con's of each choice. My general rule (some exceptions obviously) is, if I can do it in 5 minutes or less, a plugin is probably not a good idea.

Correct Format Plugin Released

2009-07-04T00:00:00-07:00

I have released a new Plugin for Rails, called Correct-Format.

Github URL: github.com/mendable/correct-format

This plugin allows you to automatically correct simple user input mistakes and format user-input without raising an ActiveRecord Error and without inserting inconsistently formatted data into your database. Using this plugin will enhance the usability and user-friendlyness of your application and increase your data integrity.

You can automatically:

Make a field uppercase
Make a field lowercase
Capitalize the first letter of the first word (and lower case everything else)
Capitalize the first letter of all words (and lower case everything else)
Replace comma with period and downcase email addresses

Installation

$ git submodule add git://github.com/mendable/correct-format.git vendor/plugins/correct-format
$ git commit -am "add correct-format plugin"

Ensure that you have this in your config/deploy.rb:

set :git_enable_submodules, true

so that the submodules are pulled down when you deploy your application.

Example Usage

  class User < ActiveRecord::Base
    # Make usernames consistently lower case
    correct_format_downcase :username

    # Replace comma's with periods in email address, and make email address all lower case
    correct_format_email :email

    # Capitalize first letter of first word, and downcase the rest
    correct_format_capitalize :username

    # Capitalize all first letters of ALL WORDS in the string
    correct_format_capitalize_each :address_line_1, :address_line_2

    # UK Postcodes are upper case
    correct_format_upcase :postcode

    # apply a function to EVERY string field in a record
    correct_format_capitalize self.attributes.select{|k, v| self.column_for_attribute(k).type == :string }.map(&amp;:first)
  end

Feedback and patches welcome.

Howto Display a GitHub Changelog in your application

2009-06-27T00:00:00-07:00

In this post I will show you a simple way to display a list of your GitHub changes inside your application.

This can be useful for so many reasons, not least helping to create a better relationship between your users and your developers. This obviously would not be applicable in all circumstances, but in many cases, having a Changelog inside your application (especially where the end users already work closely with the application developers) can enable them to see what work has been done, and what new features are available to them. This is also fantastically useful for allowing Project managers to keep track of development changes.

The code below will pull out the last 20 GitHub commit comments from your GitHub project, and display them in a table, and with a bit of CSS styling will look something like the screenshot above, or better!

<table border="0" width="100%">
<tr>
<th>When</th>
<th>Description</th>
</tr>
<% recent_website_changes = `git log -20 --abbrev-commit --pretty=format:"%H %cr - %s"` rescue ""   
recent_website_changes.split(/\n/).each do |ln|     
    commit_code = ln.split(/ /)[0]     
    without_commit = ln.gsub("#{commit_code} ", "")     
    commit_time = without_commit.split(/ - /)[0]     
    message = without_commit.gsub("#{commit_time} - ", "") %>
    <tr>
    <td><%= commit_time %></td>
    <td><%= message %></td>
    <td><%= link_to('view', "https://github.com/your-github-username/your-project-name/commit/#{commit_code}") %></td>
    </tr>
<% end %>
</table>

Another advantage of displaying your commit log publicly to other non-developers is that it can incentivize your developers to think carefully about writing better commit messages, knowing they will have to be read by others.

Set your Rails Environment Globally

2009-06-20T00:00:00-07:00

In much of the rails documentation it is suggested that when you want to run a task in a specific environment, you should set the RAILS_ENV environment variable on the command line where you are running the task.

For example, assume you want to run rake db:migrate on the production system (should be done automatically by Capistrano, but anyway..), because it is the production system, you would have to declare your rails environment in the command line, so you would actually have to run RAILS_ENV=production rake db:migrate.

Well, this becomes tedious sometimes, and open to human error as well. Luckily there is a way around this.

The solution is simple.

Add it to your /etc/environment file:

RAILS_ENV=production

You may have to log out and log in again of your shell for it to take effect. Put this on your production server, and any command that runs on that production server will automatically default to the production environment. This includes cron jobs, and other users logging into the same system.

Handy!

How to protect a folder by IP address in Nginx

2009-06-20T00:00:00-07:00

To protect a folder by IP address in nginx is fairly simple, see the example below that demonstrates only allowing access to the /protected folder for people in the 192.168.0.* network.

server {
  ....
  location /protected {
    allow 192.168.0.0/24;
    deny all;
  }
  ...
}

Nudge Book and User-Website Interactions

2009-06-18T00:00:00-07:00

I have been reading the book Nudge: Improving Decisions About Health, Wealth and Happiness recently.

The book talks about how "choice architects" can make small changes to the way choices are presented (in every day life) and have a huge impact on the choices that people make.

What I found particularly interesting is how many of the idea's in the book can be taken and applied to the Internet and websites we build, helping guide users to the best choices and also increase conversion rates for the Website, creating a better experience for the user, increasing user retention rates, user satisfaction, and ultimately profitability for the business.

Highly recommended.

5 Simple rules for managing your Email Inbox

2009-06-12T00:00:00-07:00

A few years ago, one of the most time consuming but un-productive parts of my day was checking and managing email Inbox. Back then, I used to have one big list of all my emails (the Inbox), with things I did not need to see ever again continuously taking up screen space and brain time.

Then It dawned on me... there must be a better way to handle this.

There was.

All emails can be categorised in 5 ways, and depending on what category any particular email falls into, depends on what action to take. This simple set of rules increased my productivity so much, that at the end of a days work, I almost always have a completely empty Inbox!

Things I need to reply to -Fairly simple. If I need to reply to it, then I reply to it straight away. The trick with this rule is, never let something sit there for a few hours (or even days) before replying to it. (otherwise, you end up with a big Inbox full of emails you intended to reply to, but...)
Things I have replied to - After replying to an email, unless there is more work for me to do, get rid of it. File it away. Archive it. If it is delt with, then it must disappear. Solution for this? The "Archive" button in Gmail. Generally, if it was important enough for me to reply to it, it is probably worth saving around out of sight. Another tip here: If someone emails you asking for something, you email them back (eg, requesting more info, etc), Archive the email right now. The ball is in their court, don't let your own brain spend time looking at things where you are waiting on other people.
Things I need to do - eg, someone has requested I do something etc. This is the ONLY thing that is allowed to stay in my Inbox.
Things I care about, but don't need to action - Say for example, receipts for purchases etc, in this case, it can be Archived straight away.
Things I do not care about - DELETE. But, before I do, I need to know, will I get this again next week? Having to click delete every week is still time consuming. My strategy here is to try and always close accounts of services that I do not use any more, un-subscribe from mailing lists I am no longer interested in. Where this is not possible, setup email filters to automatically delete those emails.

Following these simple rules, I have managed to significantly reduce the amount of time I spend each day handling my email. The number of emails I end up looking at each day is probably down by 80% compared to several years ago. The process of checking mail is fast, efficient and hassle free. Just the way I like it. Now time for a cup of Tea!

How do you maximise your email productivity?

Mass Find/replace for Rails Tests

2009-05-26T00:00:00-07:00

After upgrading an old project to Rails 2.3.2,my tests were failing to run, bombing out with an error like:

./test/unit/my_model.rb:4: undefined method `fixtures' for MyModelTest:Class (NoMethodError)

This was because in Rails 2.3.2, tests have to derive from ActiveSupport::TestCase, and not the old Test::Unit::TestCase.

A quick find and replace soon has us up and running again: perl -pi -w -e 's/Test::Unit::TestCase/ActiveSupport::TestCase/g;' test/unit/*.rb

Effectively, this one-liner searches through all the files in the test/unit directory, and replaces all instances of "Test::Unit::TestCase" with "ActiveSupport::TestCase"

Git Submodules and Capistrano

2009-04-22T00:00:00-07:00

Small point of reference, if you use git submodules in your rails application, then remember to add:

set :git_enable_submodules, true

to your deploy recipe. This is not very well documented as a lot of documentation on the web for capistrano was written before this feature was included, and before git was widely used.

Where are your users from? - (By SQL Email Domain)

2009-04-19T00:00:00-07:00

Ever wondered who your users use as their email provider? Who are the biggest email providers of the users on your website? Luckily SQL has the answer in 1 easy step using the SUBSTRING_INDEX string function.

mysql> SELECT COUNT(*) AS Total, 
  SUBSTRING_INDEX(email, '@', -1) AS Domain
  FROM users 
  GROUP BY SUBSTRING_INDEX(email, '@', -1)
  ORDER BY COUNT(*) DESC
  LIMIT 10;
+-------+------------------+
| Total | Domain           |
+-------+------------------+
|  1000 | hotmail.com      |
|   999 | yahoo.co.uk      |
|   888 | aol.com          |
|   777 | btinternet.com   |
|   666 | yahoo.com        |
|   555 | hotmail.co.uk    |
|   444 | gmail.com        |
|   333 | ntlworld.com     |
|   222 | tiscali.co.uk    |
|   111 | eircom.net       |
+-------+------------------+
10 rows in set (0.00 sec)