Simple thoughts - articles

Mephisto_i18n update

2007-11-02T15:23:00Z

Just a quick heads up on the status of this for those who are interested.

Though it may seem that this project has died from lack of activity, I am still very much maintaining this project and have been adding fixes and tweaks on and off during the last few months.

In fact, I’ve just committed a couple of fixes:

Correct subdomain parsing for non-multisite mephisto instances (Bug was introduced when adding multisite support)
Making the dynamic i18n link switcher drop method handle paged articles whose section is the home section.

The future status of this project is basically pending a rewrite of globalize. I had already written the next version of globalize (currently stored in a branch) with support for rfc4646 but with the new rails i18n api talks going on, I’m waiting for these to conclude and get the rewrite for globalize 2 done before updating this plugin for that new version.

So….

To all those currently/thinking of using the plugin, feel free to bug me with questions, bug reports and help. I’m still commited to maintaining this.

Btw, the latest version in trunk also has support for plugin template handlers, so you can now use erb, haml or whatever else takes your fancy.

more validations

2007-10-23T09:14:00Z

I wrote up a validates_condition_between validations class method the other day. It basically validates that the values of two attributes matches the condition supplied. Only one condition can be used at a time. I searched around for something like this but couldn’t find anything so I wrote it.

Usage:



   class Person < ActiveRecord::Base
     validates_condition_between :start_time, :end_time, :condition=>:greater_than
     validates_condition_between :credit, :max_credit, :condition=>:less_than_or_equal_to, :allow_blank => true
     validates_condition_between :key, :keys, :condition=>:inclusion, :allow_nil => true
   end

I’ve stuck it in the following plugin (just the one validation for now):

12	script/plugin install svn://rubyforge.org/var/svn/sidirodromos/plugins/more_validations

After Ostrava

2007-07-04T06:20:00Z

Well it’s been a couple of weeks since my weekend escapade to Ostrava on Rails in the Czech Republic.

I had a thoroughly good time. My trip out was as uneventful as Toby’s was eventful. I had about an hour of time to kill at the Holešovice station and as I hadn’t eaten I decided to try some Czech cuisine. Granted, I probably didn’t choose the best representation of Czech restauranteurs but my fried cheese with deeply fried chips wasn’t too bad at all. I actually managed to order ‘ketchup’ without having to point to it.

When the train finally arrived I met up with Jamis , Toby and Filip. They were all really nice guys and I hardly noticed the 3½ hours the (very comfy might I add) pendolino train took to take us to Ostrava.

When we actually got there, it was pissing down with rain. Thankfully, Lucie was there to pick us up and though her car’s demisting system decided to go on holiday and the trip to the hotel was a bit harrowing (at least from where I was sitting :), it was a short trip and we were soon settled in. Lucie kindly pointed us to the local diners and in the end we chose a pizza place, whose name I forgot but which was really very nice. Though I didn’t try one myself the pizza’s were huge and looked very tasty.

Day 1 of the conference, and Lucie was there again bright and early to pick us up. I really enjoyed the nice relaxing walk over to the conference center. That really made the world of difference and helped me calm my nerves.

Jamis was up first and talking about capistrano was inevitably a hot topic and very interesting. I got a few questions in as regards deprec and cap2 which was of especial interest to me as I want to see when will be the right moment to integrate capservext with deprec. I’d rather not have to repeat myself.

I was up next and I felt I did a good job mostly though I couldn’t completely shake off the nerves. There certainly was a lot of interest from the public judging from the questions I got which I thought was a very good sign for globalize and i18n on rails in general. You can see the videos of the talks here. The slides I used are available here.

We then had an interesting talk about possible rails networking initiatives in the Czech republic and the chance to talk to an existing java user group in Luxembourg. I must say that I agree with Toby and Jamis when they say that these things are better off started on a smaller more local scale via occasional meet ups and communication via interested developers and then scale up. Nevertheless, the Ostrava conference showed that there was plenty of interest in the Czech republic (and in the greater eastern european area) for rails and I think it was a great platform to bootstrap the kind of rails communites required to build up a solid rails presence.

Certainly, the efficiency with which the whole conference was handled is a great testiment to their commitment for promoting rails. I am thoroughly looking forward to next years edition, and though it would make more sense to do it in Prague, I felt right at home in Ostrava, thanks to people like Robert Cigán, Jiří Kubica and Lucie Maliková.

After the video conference, Tobias Luetke was up talking about how he got shopify to where it is today. A really interesting talk and very eye opening. I smiled when he said ”....if in doubt the designer is always right” (or something like that). I’ve had a number of ‘disputes’ shall we say with my ‘designer’ (Thomas Maas) and inevitably he ends up being right.

Luke Francl, talked up using sms gateways to send/receive sms/mms and showed us how he came up with a nice simple way to automate this. I might have use for that some time soon but seems like spanish carriers don’t like to play ball in this respect.

I really enjoyed Tim Lossen’s talk on setting up a rails cluster for 100.000 CZK. He suggested virtualizing a physical server into separate ‘logical’ servers for the app servers and db and then expanded on this. He mentioned switfiplied mongrel which I’ve been meaning to test. He also showed a site where you can find good deals on dedicated servers. All in all an excellent talk. Check out his slides when they come out.

We finished off the day with a final discussion. A few questions were heard then but mostly people were ready for the bar :)

After the Day 1 talks were over we were taken over to the downtown area on Stodoiny street. We had an excellant dinner at a cocktail bar and then some really nice czech beer at theatre-themed bar which we had practically to ourselves.

Day 2, despite having the slides in english was all in czech and though I’ve an ear for languages, I really need an extra one for czech :) Most of the talks were introductory ruby/rails aimed at students. Robert Cigán gave a talk on the porting of Skvely.cz main product to rails. From the slides I understood he was using a separate database for each customer and had come up with a neat technique for how to switch database connections for each client. Jiří Kubíček talk on rails deployment was also very interesting. He works for railshosting.cz which is offering free accounts for developers to try deploying their rails apps. When you sign up they actually send you a preconfigured cap deploy recipe as an attachment in the welcome mail so all you need to be up and running is modify the repo url and cap deploy. Very nice. We spent the rest of Saturday after the talks in a bar decorated in american 60s style. That was our last evening in Ostrava.

All in all I had a great time. I met some lovely people, listened to some interesting talks, had interesting discussions and hopefully made one or two friends. I found myself connecting with and discovering that I had many things in common with others that live on the other side of the world. And all this was thanks to the great success that was the Ostrava on Rails conference.

Talking about Globalize in Ostrava

2007-06-20T19:07:00Z

I’m off tomorrow to Prague in the Czech republic and will then make my way to Ostrava where I’ll be giving a talk about Globalize and it’s future.

I’ll be posting the slides here on my blog once I’m back.

Say hello if you’re there :)

saimonmoore.net now runs multiple globalized sites

2007-05-28T13:56:00Z

I recently upgraded my mephisto_i18n plugin to add complete support for multiple sites.

In the process, I converted the mephisto instance running http://saimonmoore.net into a multisite instance. Currently, the other instance is our companie’s site webtypes.com which I ported over from textpattern.

You can read more about how I did it in the third installment of my globalizing mephisto series.

Other improvements to the site have been:

Added better spam protection via Sven’s article
Added an email-based contact form so I’ll hopefully not miss any feedback this time.

ActiveEvent: My latest plugin

2007-05-17T09:58:00Z

For one of my current applications I had a need for an event handling system.

I needed to able to dispatch events during the application’s lifecycle, and have observers respond to them appropriately. I also needed to keep an audit trail of the user’s interaction with the application.

Rails has the basis of an event handling system for ActiveRecord models in the form of it’s callbacks that ActiveRecord::Observer’s can react upon. However, this is limited to a set amount of callbacks and doesn’t support a wide variety of events.

So I wrote ActiveEvent to do this.

I actually wrote this code about a year ago for Rails 1.0 when I was developing ocium.com but I’ve since needed this for one of our current internal applications and I’ve refactored and packaged into a plugin that others can use.

“What do I use this for and how do I use it?” – A very good question.

Use it for:

Creating an synchronous order event handling system with events like OrderPlacedEvent, OrderShippedEvent, OrderRefusedEvent. Have observers that react to each event in a distinct and customised way. Perhaps, you want to send an email informing the client of the fact that their order has been shipped (very simple use case). Your imagination is the limit.

(NOTE: ActiveEvent in combination with the ActsAsStatemachine plugin makes for a very powerful way of controlling your application’s lifecycle.)

Want to be able to easily track a users actions within your application? This plugin makes this a piece of cake.

Let’s see it in action


class Order < ActiveRecord::Base

  has_active_events    #Assumes an existing User model class but is customizable

  def place_order(msg)
    send_event(ActiveEvent::OrderPlacedEvent.new(self, nil, msg)
  end
end

class OrderObserver < ActiveRecord::Observer

  def after_save
    #do something here
  end

  def on_event(order, event)
    #react to the event
    #e.g.
    case Event
      when OrderShippedEvent
        OrderNotifier.deliver_order_shipped(order, event)
      when OrderCancelledEvent
        order.cancel! #Change model state via ActsAsStatemachine plugin
    end

    #Add the event to the audit system
    ActiveEvent::AuditTrail.add_event(event) 
  end
end

Get it installed:

1234	script/plugin install svn://rubyforge.org/var/svn/sidirodromos/plugins/active_event/trunkscript/generate active_event migrations

Change coming soon:

Add an option to has_active_events method to allow transparent event tracking. i.e. In above example, you’d no longer need to add the ActiveEvent::AuditTrail.add_event(event) call.

e.g.

1234	class Order < ActiveRecord::Base has_active_events :audit => trueend

Make the auditing system completely optional (No need to create its tables unless you want it)
Add asynchronous notifications support via the backgroundrb plugin

Actually I thought these todo’s were so important I added them in already :)

Have a look at the plugins README for details and examples. Then have a look at it’s tests.

P.S. I’m thinking about adding support for the Memento pattern to provide granularized undo possibilities…

Localized urls

2007-05-17T09:57:00Z

In this article, I’m going to describe a simple way to localize your application’s urls using globalize.

Localize your urls

What do I mean by “localize your urls” exactly? For example, you browse to :


   http://ocium.com/artists/millivanilli

and are shown the “Milli Vanilli” artist info page in the base locale (which is ‘english’ for arguments sake). You click a link to switch locale within the page and you’re transported to:


    http://es.ocium.com/artistas/millivanilli

As you can see, the whole url has been translated into the language of the currently active locale.

Locale defined as a path parameter

In this example, I’m using subdomains to indicate the locale, but you can easily use path/query parameters to define the locale.

e.g.


   http://ocium.com/en/artists/millivanilli
   =>
   http://ocium.com/es/artistas/millivanilli

The plugin

I’ve wrapped up the code to do that within the LocalizedRoutes plugin:

Installation

Assuming you have the latest version of globalize installed (at least for-1.2),

Install the plugin via :


script/plugin install svn://rubyforge.org/var/svn/sidirodromos/plugins/localized_routes/trunk

Usage

You can now translate the urls in your routes via:


ActionController::Routing::Routes.draw do |map|

  #Will translate 'list' to the default namespace
  map.list 'list'.t, :controller => 'blog', :action => 'list'

  #Will translate 'list' to the 'urls' namespace
  map.list 'list'>> :urls, :controller => 'blog', :action => 'list'
end

Note: The use of the ‘urls’ namespace when translating routes. This is recommended as it’s then easy to separate url translations from other translations.

The localized_routes plugin looks at the incoming request and calculates which locale is to be set as active from one of three possible methods in the following order:

“Query parameter” named “locale” e.g


http://saimonmoore.net/listar?locale=es

“Path parameter” extracted by matching request path against Globalize::LocalizedRoutes::LOCALE_PATH_MATCHING_RE e.g.
123
Globalize::LocalizedRoutes::LOCALE_PATH_MATCHING_RE => /^\/([\w-]+)|\/([\w-]+)\/.*$/
```
 http://saimonmoore.net/es/listar
 or
 http://saimonmoore.net/es
```
Note: If you want to define the locale in the path differently, then just override the regexp.
“Subdomain” e.g.
```
http://es.saimonmoore.net/listar
```

If you’re using the path parameter to define your locale then it’s a good idea to specify it in the route like so:

1234	ActionController::Routing::Routes.draw do \|map\| map.list ':locale/list'>> :urls, :controller => 'blog', :action => 'list'end

Then you’ll have access to the “locale” parameter in the controller and it’s useful for url helper methods.

Some examples:

You want to define the “locale via a path parameter”.

1234	ActionController::Routing::Routes.draw do \|map\| map.list ':locale/list'>> :urls, :controller => 'blog', :action => 'list'end

You translate the urls for each language you support:


Locale.set_base_language('en-GB')

Locale.set('es-ES')
Locale.set_translation_with_namespace(':locale/list', :urls, ':locale/listar')

Locale.set('fr-FR')
Locale.set_translation_with_namespace(':locale/list', :urls, ':locale/enumerer')

You can now access:


 http://saimonmoore.net/en/list
 http://saimonmoore.net/es/listar
 http://saimonmoore.net/fr/enumerer

To access:

http://saimonmoore.net/list

you’d need to additionally define:

12	map.list 'list', :controller => 'blog', :action => 'list'

If, on the other hand, you’d like to define the “locale via subdomains”, then:

1234	ActionController::Routing::Routes.draw do \|map\| map.list 'list'>> :urls, :controller => 'blog', :action => 'list'end

Again you translate the urls for each language you support:


Locale.set_base_language('en-GB')

Locale.set('es-ES')
Locale.set_translation_with_namespace('/list', :urls, '/listar')

Locale.set('fr-FR')
Locale.set_translation_with_namespace('/list', :urls, '/enumerer')

You can now access:


http://saimonmoore.net/list
http://es.saimonmoore.net/listar
http://fr.saimonmoore.net/enumerer

Switch Locale Links

Here are examples of how you’d create a list of links to switch the current locale using all three methods for defining the locale:

Assuming routes defined as:


ActionController::Routing::Routes.draw do |map|
  map.list_with_locale ':locale/list' >> :urls, :controller => 'application', :action => 'list'
  map.list 'list' >> :urls, :controller => 'application', :action => 'list'
  map.connect ':controller/:action/:id'
end

Note: For this example I’ve already translated the urls for each of the supported languages.

Then in your partial (or extract to a helper):

path parameter


      <% SupportedLocales.supported_language_codes.each do |code| -%>
        <% Locale.switch_locale(code) do -%>
          <% ActionController::Routing::Routes.reload -%>
          <%= link_to Locale.language.native_name, list_with_locale_url('locale' => code) %>
        <% end -%>
      <% end -%>

Note:To make it even simpler you could use one of these techniques to only specify “list_with_locale_url”
Browsing to http://saimonmoore.net/list generates:


    <a href="http://saimonmoore.net/en/list">English</a>
    <a href="http://saimonmoore.net/el/katalago">Ελληνικά</a>
    <a href="http://saimonmoore.net/es/listar">Español</a>
    <a href="http://saimonmoore.net/ca/listar">Català</a>

query parameter


      <% SupportedLocales.supported_language_codes.each do |code| -%>
        <% Locale.switch_locale(code) do -%>
          <% ActionController::Routing::Routes.reload -%>
          <%= link_to Locale.language.native_name, list_url('locale' => code) %>
        <% end -%>
      <% end -%>

Browsing to http://saimonmoore.net/list generates:


      <a href="http://saimonmoore.net/list?locale=en">English</a>
      <a href="http://saimonmoore.net/katalogo?locale=el">Ελληνικά</a>
      <a href="http://saimonmoore.net/listar?locale=es">Español</a>
      <a href="http://saimonmoore.net/listar?locale=ca">Català</a>

subdomain


      <% SupportedLocales.supported_language_codes.each do |code| -%>
        <% Locale.switch_locale(code) do -%>
          <% ActionController::Routing::Routes.reload -%>
          <%= link_to Locale.language.native_name, subdomain_for(:list_url) %>
        <% end -%>
      <% end -%>

Browsing to http://saimonmoore.net/list generates:


      <a href="http://saimonmoore.net/list">English</a>
      <a href="http://el.saimonmoore.net/katalogo">Ελληνικά</a>
      <a href="http://es.saimonmoore.net/listar">Español</a>
      <a href="http://ca.saimonmoore.net/listar">Català</a>

Test application

If you’d like to experiment with this, I’ve bundled up a test application which has everything you need to play around with this. Just set up a db & run rake globalize:setup. In the db directory you can find an sql file, with all the translations used in this example.

Note: In the last example I used a subdomain_for() helper to correctly generate the right host for each locale. This is very simple and you can see the code in the example app.

The code

I’ve babbled a lot about how you can use this plugin but let’s take a look at the actual code. It’s really quite simple.

In essence, I realized that for each request that goes through rails, the dispatcher takes care of running it through the defined routes in order to recognize the appropriate controller and action for the request and then passes it along to the controller for processing.

What I needed then is to ensure that the locale is set before routing recognition occurs

So I simple overrode Dispatcher.dispatch() to add in a call to a set_locale() method before the call to ActionController::Routing::Routes.recognize. This means Globalize::Locale is now set when the routing code does it’s recognition which means you can then translate strings within the route definitions.

i.e.


      def dispatch_with_locale(cgi = nil, session_options = ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS, output = $stdout)
        controller = nil
        if cgi ||= new_cgi(output)
          request, response = ActionController::CgiRequest.new(cgi, session_options), ActionController::CgiResponse.new(cgi)
=>        set_locale(request)  #Set the locale before route recognition
          prepare_application
          controller = ActionController::Routing::Routes.recognize(request)
          controller.process(request, response).out(output)
        end
      ....

Note: I hadn’t seen Dispatcher.to_prepare before and Josh Sierles brought it to my attention however, this only executes the callback on the first request in production mode so it’s not really usefull in this case. Another problem is that set_locale needs access to the request. I’m thinking of sending in a patch to rails to make it configurable and will then convert the code to using this.

Take a look at the code to see what set_locale does (though it’s nothing fancy).

Hope that you find this handy. I certainly have.

Saimon

P.S. I’ve incorporated this functionality directly into my mephisto_i18n plugin which makes it handy if you need to extend mephisto with custom routes.

Update: I’ve refactored the code slightly to use a patched Dispatcher which adds support for request_callbacks (similar to preparation_callbacks but which are executed on every request and which have access to the request object).

Problem?: The code now reloads the routes for the current locale on every request. This is so that you can reload routes within Locale.switch_locale blocks and generate the localized url correctly. I realize this may have a performance hit but initiall tests show this to minimal.

saimonmoore.net is now Globalized!

2007-03-21T17:59:00Z

Back up again…

The process of rebuilding my slice was relatively painless. It took me all of four hours to go from my original mephisto installation, to a completely new slice with a globalized mephisto trunk, restored subversion repository.

Twice! As I also host a veterinary clinic site for a friend.

saimonmoore.net down for maintenance

2007-03-18T13:16:00Z

To all my loyal readers (all 6 of you :).

I’m taking the site down temporarily as I want to rebuild it using Capistrano Server Extensions and the latest Mephisto trunk with my Mephisto I18n plugin (An article about this plugin will be coming forthwith ). To eat my own dog food as they say.

Also I’m not very happy about the site’s styling so I’m going to try and change that soon as well.

Back soon…

Namespaced View Translations

2007-03-18T13:11:00Z

ViewTranslations are a very handy way of translating all of your content that isn’t represented by an ActiveRecord model, which is actually quite a lot of content.

And they work great:


#In es locale

"translate me".t => "Traduceme"

"%s, can you translate this?" / 'saimon' => '¿Saimon, puedes traducir esto?'

"%d translations" / 0 => "No hay traducciónes"
"%d translations" / 1 => "1 traducción"
"%d translations" / 10 => "10 traducciónes"

But imagine you’re using view translations all over the place in your application.

You use them to translate the static content of your view templates. You use them to translate the urls in your links. You use them to translate your administration interface.

After a while the list of translations starts to become a bit too unwieldy.

Let’s say that you decide to bring in professional translators, who may be decidedly non technical, to translate your site’s content.

You point them to your new spiffy, paginated, searchable list of words/phrases to be translated and leave them to it.

The day after your boss phones you up and bellows:

- None of the site’s links are working in Slovenian?

Oops! You start to investigate and find that the stupid translators, went ahead and translated:


'section' => 'sección' (In Spanish)

which you were using to translate the url:


http://example.com/section => http://example.com/seccion

- Jeez, don’t translators know you can’t use non-ascii characters in urls?

- Umm, well no. They’re translators. That’s your job.

- Ah, well, ok I’ll just give them a list of words that are urls and should be translated without accents.

- But we use the word ‘section’ in the view as well…

- ummm… Let me think….. (time passes)

- Also we need the site in Greek which has a lot of strange characters. And come to think of it, this list of urls is getting pretty hard to maintain.

-Well, what we need is to be able to have two sorts of translation of the word ‘section’, one for the front end and one for the urls.

Using namespaced view translations

And that’s what namespaced view translations are all about:


#In es locale

"section".t         => "sección"  #The default translation with no namespace

"section" >> 'urls' => "seccion"  #The translation in the 'urls' namespace
or
"section".tn(:urls)
or
"draw".translate_with_namespace(:urls)

#The translation in the 'admin' namespace
"section" >> 'admin' => "sección"

# Text interpolation of the translation in the 'admin' namespace
"%s, can you translate this?".tn(:admin, 'saimon') => '¿Saimon, puedes traducir esto?'

#Numeric interpolation of the translation in the 'admin' namespace with a zero value
"%d translations".tn(:admin, 0) => "No hay traducciónes"

#Numeric interpolation of the translation in the 'admin' namespace with non-zero values
"%d translations".tn(:admin, 1) => "1 traducción"
"%d translations".tn(:admin, 10) => "10 traducciónes"

#Translation in the 'admin' namespace with a default value if no
#translation is available
"should be translated".tn(:admin,nil, 'untranslated') => "untranslated"

Adding namespaced view translations to the database

Adding translations with namespaces to the database uses a differently named method but is basically the same syntax as Locale.set_translation:


Locale.set('es-ES')

#No namespace
Locale.set_translation("section", "sección")

#With namespace 'urls'
Locale.set_translation_with_namespace("section", "urls", "seccion")

or adding translations with plural forms:


Locale.set('es-ES')

#No namespace
Locale.set_translation("draw %d times", "dibujar una vez", "dibujar %d veces")

#With namespace 'lottery'
Locale.set_translation_with_namespace("draw %d times", "lottery", "seleccionar una vez", "seleccionar %d veces")

Filtering translations by namespace

Once you have view translations categorised under different namespaces, you’ll most likely want to create a friendly interface for your translators so that they don’t translate phrases in the ‘urls’ namespace for example and can concentrate on the translations of the front-end.


url_translations = ViewTranslations.find(:all, :conditions => {:namespace => 'urls', :language => spanish})

Of course that’s very simplistic and you’d probably want to also distinguish between translations with plural forms, paginate etc but you get the idea.

Upgrading your schema

I needed to change the globalize view_translations table and add an extra varchar column to support namespaced view translations. So in order to use the for-1.2 release with your application, you need to upgrade your schema.

If you install the for-1.2 release via:


script/plugin install http://svn.globalize-rails.org/svn/globalize/branches/for-1.2

then the plugin’s install hook will automatically update your globalize tables for you if they already exist.

If you don’t use the script/plugin install method, then you can simply run the following rake task which does the same thing:


rake globalize:upgrade_schema_to_1_dot_2

If this is the first time you’re installing Globalize then just run the default globalize setup rake task:


rake globalize:setup

That concludes this article. Watch out for the next article in the series where I’ll be talking about Globalize::SupportedLocales, a class that encapsulates the idea of an application’s supported locales.

Making migrating to Globalize's internal storage mechanism easy

2007-03-17T19:56:00Z

As I mentioned in my previous article, the week spot of the internal storage mechanism, is taking care of adding the extra columns to the schema.

I also mentioned that using ActiveRecord::Migrations would make your life a lot easier and, really, if you aren’t already using them as standard in your rails development, your missing out on a lot of automation.

However, you’ve still got to mundanely type these things out, and we all know what us humans are like (We’ll I know myself at least :).

So, currently in the for-1.2 branch you can find a rails generator that will automate this procedure for you.

How does this work?

Imagine we have the following models:


#Assuming:

Globalize::DbTranslate.keep_translations_in_model = false

#or unset

class Noddy < ActiveRecord::Base; end

class Dummy < ActiveRecord::Base
  self.keep_translations_in_model = true
  translates :name, :base_as_default => true
end

class Epi < ActiveRecord::Base
  self.keep_translations_in_model = true
  set_table_name :epifanias

  translates :name, :surnames
end

class Blas < ActiveRecord::Base
  self.keep_translations_in_model = true
  translates :hobby
end

Let’s open up a shell and see


$ cd myapp
$ script/generate globalize internal es,fr
      exists  db/migrate
      exists  db/migrate
      create  db/migrate/006_globalize_add_translated_fields_for_dummy_epi_blas.rb

The script has created a migration for us…Let’s have a look see:


class GlobalizeAddTranslatedFieldsForDummyEpiBlas < ActiveRecord::Migration
  def self.up
    #Fields for Dummy
    add_column :dummies, :name_es, :string
    add_column :dummies, :name_fr, :string
    #Fields for Epi
    add_column :epifanias, :name_es, :string
    add_column :epifanias, :surnames_es, :string
    add_column :epifanias, :name_fr, :string
    add_column :epifanias, :surnames_fr, :string
    #Fields for Blas
    add_column :blas, :hobby_es, :string
    add_column :blas, :hobby_fr, :string
  end

  def self.down
    #Fields for Dummy
    remove_column :dummies, :name_es
    remove_column :dummies, :name_fr
    #Fields for Epi
    remove_column :epifanias, :name_es
    remove_column :epifanias, :surnames_es
    remove_column :epifanias, :name_fr
    remove_column :epifanias, :surnames_fr
    #Fields for Blas
    remove_column :blas, :hobby_es
    remove_column :blas, :hobby_fr
  end
end

Cool! The migration contains all the missing columns required to support es (Spanish) and fr (French) apart from the base locale.

As it’s a generator we can also tell it to rollback that change:


$ script/destroy globalize internal es,fr
    notempty  db/migrate
    notempty  db
    rm  db/migrate/006_globalize_add_translated_fields_for_dummy_epi_blas.rb
    notempty  db/migrate
    notempty  db

So once you’ve created your application, this generator makes it easy to maintain your schema if you decide to you use the internal storage mechanism.

Note for the future: Currently you need to specify the languages your application supports on the command line. I’ve commited some code to Globalize trunk which encapsulates the idea of an application’s supported locales into a dedicated class. So for trunk I’ll be updating this generator to first check whether the SupportLocales.supported_locales attribute has been defined, so you won’t need to type in the locales on the command line (less chance for errors).

That’s what’s available to you in the for-1.2 release to help out with migrating to the new storage mechanism.

However, I’ve gone a step further and written a migration tool that more or less does all the work required for you to convert an existing application using the external storage system into one using the internal storage mechanism.

Let’s look at the class definitions again. This time we’re assuming this application is using the external storage mechanism already:


#Assuming:
Globalize::DbTranslate.keep_translations_in_model = false       #or unset


class Blas < ActiveRecord::Base
  self.keep_translations_in_model = false
  translates :hobby
end

class Dummy < ActiveRecord::Base
  translates :name, :base_as_default => true
end

class Epi < ActiveRecord::Base
  set_table_name :epifanias

  translates :name, :surnames
end

class Noddy < ActiveRecord::Base; end

Let’s also look at the current contents of the database tables these models represent:


#_blas_
id  name    hobby
1   Tony    skiing
2   George  football
3   Sandra  tennis

#_dummies_
id  name
1   Epiphany
2   Socks

#_epifanias_
id  name    surnames
1  Saimon    Moore
2  Thomas    Maas
3  Maria     Perez

#_noddies_ has no data

Let’s imagine that this is an application that already has some model translations using the external storage mechanism i.e. in the globalize_translations table.

These are the current translations for these models (supports spanish (es) and (polish):


id  item_id  table_name     facet       text          language_id
103     1       blas        hobby       esquipl         4
100     1       blas        hobby       esqui           7
104     2       blas        hobby       futbolpl        4
101     2       blas        hobby       futbol          7
105     3       blas        hobby       tenispl         4
102     3       blas        hobby       tenis           7
120     1       dummies     name        Saimon (pl)     4
118     1       dummies     name        Saimon (es)     7
121     2       dummies     name        Epifania (pl)   4
119     2       dummies     name        Epifania (es)   7
109     1       epifanias   name        Saimon pl       4
115     1       epifanias   surnames    Moore pl        4
106     1       epifanias   name        Saimon es       7
112     1       epifanias   surnames    Moore es        7
110     2       epifanias   name        Thomas pl       4
116     2       epifanias   surnames    Maas pl         4
107     2       epifanias   name        Thomas es       7
113     2       epifanias   surnames    Maas es         7
111     3       epifanias   name        Maria pl        4
117     3       epifanias   surnames    Perez pl        4
108     3       epifanias   name        Maria es        7
114     3       epifanias   surnames    Perez es        7

So let’s open up a console and type:


$ rake globalize:migrate_to_internal_storage LANGS=es,pl
Logging to /home/saimon/dev/projects/localize_for-1.2/config/../log/internal_storage_migration_development.log
Migrate source for 'Blas'? (Yes/No/All)
Y
Migrating ruby source for: /home/saimon/dev/projects/localize_for-1.2/config/../app/models/blas.rb
Migrate source for 'Dummy'? (Yes/No/All)
Y
Migrating ruby source for: /home/saimon/dev/projects/localize_for-1.2/config/../app/models/dummy.rb
Migrate source for 'Epi'? (Yes/No/All)
Y
Migrating ruby source for: /home/saimon/dev/projects/localize_for-1.2/config/../app/models/epi.rb
Generate & execute db migrations? (Yes/No)
Y
Generating db migrations...
      exists  db/migrate
      exists  db/migrate
      create  db/migrate/006_globalize_add_translated_fields_for_dummy_epi_blas.rb
Executing db migrations...
(in /home/saimon/dev/projects/localize_for-1.2)
== GlobalizeAddTranslatedFieldsForDummyEpiBlas: migrating =====================
-- add_column(:dummies, :name_es, :string)
   -> 0.6783s
-- add_column(:dummies, :name_pl, :string)
   -> 0.1232s
-- add_column(:epifanias, :name_es, :string)
   -> 0.1216s
-- add_column(:epifanias, :surnames_es, :string)
   -> 0.1105s
-- add_column(:epifanias, :name_pl, :string)
   -> 0.1438s
-- add_column(:epifanias, :surnames_pl, :string)
   -> 0.0994s
-- add_column(:blas, :hobby_es, :string)
   -> 0.1217s
-- add_column(:blas, :hobby_pl, :string)
   -> 0.1217s
== GlobalizeAddTranslatedFieldsForDummyEpiBlas: migrated (1.5246s) ============

Migrate translations for 'Blas'? (Yes/No/All)
Y
Also delete old external translations for 'Blas'? (Yes/No default: Y)
Y
Migrated translations for Blas
Migrate translations for 'Dummy'? (Yes/No/All)
Y
Also delete old external translations for 'Dummy'? (Yes/No default: Y)
Y
Migrated translations for Dummy
Migrate translations for 'Epi'? (Yes/No/All)
Y
Also delete old external translations for 'Epi'? (Yes/No default: Y)
Y
Migrated translations for Epi

Whoa! What was all that?

Well, let’s have a look at what happened:

The class definitions have been modified:


class Blas < ActiveRecord::Base
  self.keep_translations_in_model = true
  translates :hobby
end

class Dummy < ActiveRecord::Base
  self.keep_translations_in_model = true
  translates :name, :base_as_default => true
end

class Epi < ActiveRecord::Base
  set_table_name :epifanias

  self.keep_translations_in_model = true
  translates :name, :surnames
end

It has modified our model class definitions to mark them to use the internal storage mechanism.

Ah but what about the database?

Well let’s have a look if anything changed?


#_blas_
id  name  hobby       hobby_es  hobby_pl
1  Tony    skiing     esqui     esquipl
2  George  football   futbol    futbolpl
3  Sandra  tennis     tenis     tenispl

#_dummies_
id  name     name_es        name_pl
1  Epiphany  Saimon (es)    Saimon (pl)
2  Socks     Epifania (es)  Epifania (pl)

#_epifanias_
id  name  surnames  name_es      surnames_es  name_pl       surnames_pl
1  Simon  Moore     Saimon es    Moore es     Saimon pl     Moore pl
2  Thomas Maas      Thomas es    Maas es      Thomas pl     Maas pl
3  Maria  Perez     Maria es     Perez es     Maria pl      Perez pl

That’s nice. The script has automatically created a migration to add all the extra language columns to the model tables AND has migrated the actual translation data that was previously in the globalize_translations table to their respective places in the model tables.

Let’s have a look at the migration it created for us:


class GlobalizeAddTranslatedFieldsForDummyEpiBlas < ActiveRecord::Migration
  def self.up
    #Fields for Dummy
    add_column :dummies, :name_es, :string
    add_column :dummies, :name_pl, :string
    #Fields for Epi
    add_column :epifanias, :name_es, :string
    add_column :epifanias, :surnames_es, :string
    add_column :epifanias, :name_pl, :string
    add_column :epifanias, :surnames_pl, :string
    #Fields for Blas
    add_column :blas, :hobby_es, :string
    add_column :blas, :hobby_pl, :string
  end

  def self.down
    #Fields for Dummy
    remove_column :dummies, :name_es
    remove_column :dummies, :name_pl
    #Fields for Epi
    remove_column :epifanias, :name_es
    remove_column :epifanias, :surnames_es
    remove_column :epifanias, :name_pl
    remove_column :epifanias, :surnames_pl
    #Fields for Blas
    remove_column :blas, :hobby_es
    remove_column :blas, :hobby_pl
  end
end

Also notice that the script is very wary and asks you at every stage for consent before any action. You can also decide to let it automatically migrate all models it has determined should be migrated.

And finally, it even allows you to remove the old translations from globalize_translations.

This script isn’t currently in trunk, as we’re testing it a bit but if you’d like to try it out you can get it by checking it out from here

These tools should make the migration to the internal storage a lot less painful for you.

Well, that’s it for now.

My next article will cover namespaced view translations

The Internal Storage Mechanism For Globalize for Rails Model Translations

2007-03-17T06:41:00Z

The Globalize for Rails for-1.2 release introduced a new storage mechanism for Model Translations.

If you’re new to Globalize, let me give you a bit of background to model translations (Click to view) in Globalize. Those of you reading this in a feed reader and are familiar with globalize feel free to just read ahead to the Before for-1.2 section.

Introduction To Model Translations
Before for-1.2
Introducing the “Internal Storage Mechanism”
Exploring the “Internal Storage Mechanism”
Trouble spots?
Conclusion

Intro to Model Translations for Globalize beginners

By installing the Globalize plugin in your rails application you have the possibility of localising your ActiveRecord models.

e.g. by defining your class like this:


class Product < ActiveRecord::Base

  translates :name

end

the Product model will automatically and transparently save the value of the ‘name’ attribute distinctly depending on the current locale.

An example of what this means in code follows:

Assumption Let’s assume that throughout this article our base locale (By base locale I mean the default locale/language from which content in other languages will be translated from) is ‘en-US’ and the following code has been executed (Usually this is added to your environment.rb file):


include Globalize                                   #Include the Globalize module to save on typing
Locale.set_base_language('en-US')

Let’s explore what that code definition has done for us:


$ script/console
Loading development environment
>>Locale.set('en-US')                                         #We set the current locale to en-US (English)
>>product = Product.create(:name => 'Socks')                  #Create a new product
>>product.name                                                #Asking for the value of the 'name' attribute
=> "Socks"                                                    #has correctly shown 'Socks'

>>Locale.set('es-ES')                                         #We change the current locale to es-ES (Spanish)
>>product.reload                                              #Reload the product instance (Required when current locale changes)
>> product.name
=> "Socks"                                                    #We're still getting 'Socks'
>> product.name = 'Calcetines'                                #In Spanish, 'Socks' are called 'Calcetines'
>> product.save                                               #Save the product instance
>> product.name
=> "Calcetines"                                               #We now get the correctly translated value

>>Locale.set('en-US')                                         #We change the current locale back to english
>>product.reload                                              #Reload the product instance
>> product.name
=> "Socks"                                                    #We correctly get back 'Socks'

>>Locale.set('es-ES')                                         #Once more we change the current locale
>>product.reload                                              #Reload the product instance
>> product.name
=> "Calcetines"                                               #We correctly get back 'Calcetines'

So that little bit of code has afforded you quite some functionality. But what’s happening under the covers? With Globalized installed, it automatically overrides ActiveRecord::Base and ensures that when saving your product instance:

if the currently active locale is the same as the base locale then the product data is saved as normal to the ‘products’ table
but if the currently active locale is any other locale, then globalize will ensure that the data for this locale is saved to an external table (globalize_translations).

Then when you reload a model instance in a particular locale and ask it for the value of an attribute marked as being translatable, globalize is intelligent enough to correctly look for the data either in the products own table (‘products’) or in the external globalize_translations table.

The actual dynamics of this process is slightly different but you get the gist.

Before for-1.2

Before the for-1.2 release, Globalize used an external table (globalize_translations) to store translations. To achieve this, it need to override the ActiveRecord::Base.find_every method to rewrite the query sent to the database and coalesce the model table’s columns with the globalize_translations table.

This rewriting of the sql query implicitly sets up a number of limitations, the most obvious of which is that it disallows overriding the :select option to the find method because in effect that’s exactly what globalize is already doing.

This leads to other limitations like disallowing the :include option as well. (Why? Same reason as before)

There are workarounds for most of these limitations (For example, you can replace :include with :include_translated) especially if you’re writing your application from scratch.

But if you’re trying to globalize an existing application you’ve got to go through quite a number of hoops.

Introducing the “Internal Storage Mechanism”

The new internal storage mechanism gets rid of these limitations. Because it stores the translations within the models own table, all the translations are loaded every time the instance is loaded. This means that there is no need to override the sql query, which consequently means you no longer have any limitation on ActiveRecord::Base.find. Only a little bit of ruby magic is required to get things to work.

So let’s see an example of it in action:

If you read the intro to globalize, you’d have seen we used a Product model and assumed the base locale was set to ‘en-US’.

The only slight change you need to do to your ActiveRecord class definitions is switch on the internal storage mechanism like so:


class Product < ActiveRecord::Base

  self.keep_translations_in_model = true
  translates :name

end

Note: This use of “self.keep_translations_in_model = true” in the class definition allows activation of this mechanism on a per class level since the normal external storage mechanism is active by default. However, you can also set the internal storage to be the default mechanism on an application-wide level by setting the following in your environment.rb:


Globalize::DbTranslate.keep_translations_in_model = true

As this method depends on storing the translations within the model’s own table we need to add in extra columns for all the attributes that are to be localised and one each for each locale to be supported in the application.

I’ve written a tool to automate this step but for now we’ll take a more manual approach and code up a migration. In our example application, we’ll have support for both english and spanish.


class AddLocalizedFieldsForProduct < ActiveRecord::Migration
  def self.up
    add_column :products, :name_es, :string
  end

  def self.down
    remove_column :products, :name_es
  end
end

Notice, that since we’ve only a got a single attribute marked as translatable in our Product class definition and we’re only supporting a single extra locale (es-ES), we only need to add one extra column to the products table.


'attribute' + language suffix

e.g

'name' + 'es' => 'name_es'

As you can see, the name of the new column is simply the original attribute name and a suffix which is simply the language part of the locale to be supported.

So let’s open up a console and see what we can do now that the internal storage mechanism is active.


$ script/console
Loading development environment
>>Locale.set('en-US')                           #We set the current locale to en-US (English)
>>product = Product.create(:name => 'Shoes')    #Create a new product
>>product.name                                  #Asking for the value of the 'name' attribute
=> "Socks"                                      #has correctly shown 'Shoes'

>>Locale.set('es-ES')                           #We change the current locale to es-ES (Spanish)

#Note: You no longer are required to reload the product instance on locale changes

>> product.name
=> nil                                          #By default, if there are no translations nil is returned
>> product.name = 'Zapatos'                     #In Spanish, 'Shoes' are called 'Zapatos'
>> product.save                                 #Save the product instance
>> product.name
=> "Zapatos"                                    #We now get the correctly translated value

>>Locale.set('en-US')                           #We change the current locale back to english
>> product.name
=> "Shoes"                                      #We correctly get back 'Shoes'

>>Locale.set('es-ES')                           #Once more we change the current locale
>> product.name
=> "Zapatos"                                    #We correctly get back 'Zapatos'

Exploring the “Internal Storage Mechanism”

So much for the simple example let’s see what else we can do:


#Let's specify that the base locales content should be returned if untranslated
class Product < ActiveRecord::Base
  self.keep_translations_in_model = true
  translates :name, :base_as_default => true

  belongs_to :sku
  belongs_to :price_list
  has_many :commerce_items
end

$ script/console
Loading development environment
>>Locale.set('en-US')
>> product = Product.find(:last)    #Simple query
>> product.name
=> "Shoes"
>>Locale.set('es-ES')
>> product.name
=> "Zapatos"

>>product2 = Product.create(:name => 'Shirts')
>>product.name
=> "Shirts"
>>Locale.set('es-ES')
>> product.name
=> "Shirts"    #_base_as_default_ = true means untranslated attributes will return the base locale value
>> product.name = 'Camisas'
>> product.save
>> product.name
=> "Camisas"

#Now for a slightly more complex query
>>Locale.set('en-US')
>> products = Product.find(:all, :select => 'select distinct(name)', :include => :items)
=> [<#Product :name => 'Socks'>, <#Product :name => 'Shoes' >]
>> products.first.name
=> "Socks"
>>Locale.set('es-ES')
>> products.first.name
=> "Calcetines"

#Now for some conditions

>>Locale.set('en-US')
>> product = Product.find(:first, :conditions => {:name => 'Socks'})
=> <#Product :name => 'Socks'>
>> product.name
=> "Socks"
>>Locale.set('es-ES')
>> product = Product.find(:first, :conditions => {:name => 'Calcetines})
=> nil    #Eh? What happened here?

Let’s look at the query that got executed:


select * from products where products.name = 'Calcetines';

Remember that for the Spanish locale, the translated data has been stored in the name_es column so obviously, the find method wasn’t being clever enough to recognize that it should have generated:


select * from products where products.name_es = 'Calcetines';

It would have been too complex to add functionality to make this seamless and would have ended up defeating the whole purpose of this method as it would most likely need to override the sql query generation mechanism and we’d be back to the same problem as the external storage mechanism has.

However, there’s a simple solution to this problem.


>>Locale.set('es-ES')
>> product = Product.find(:first, :conditions => ["#{Product.localized_facet(:name)} = ?",'Calcetines'])
=> <#Product :name => 'Calcetines'>
>> product.name
=> "Calcetines"
>>Locale.set('en-US')
>> product.name
=> "Socks"

For simple condition statements like that we can go one better and use dynamic finders:


>>Locale.set('es-ES')
>> product = Product.find_by_name('Calcetines')
=> <#Product :name => 'Calcetines'>
>> product.name
=> "Calcetines"
>>Locale.set('en-US')
>> product = Product.find_by_name('Socks')
>> product.name
=> "Socks"

Activating the internal storage mechanism, overrides the AR method_missing to ensure that dynamic finders correctly use the right column name for the current locale.

What other things can we do:


>>Locale.set('en-US')
>> product = Product.find_by_name('Socks')
>> product.name
=> "Socks"
>>Locale.set('en-US')
>> product.name
=> "Calcetines"
>> product._name  #Using the '_' prefix to the attribute means you want the base locale content
=> "Socks"

>> product.name_before_type_cast
=> "Calcetines"
>> product._name_before_type_cast
=> "Socks"

Trouble spots?

Well, this all seems hunky dory, super fantastic, right? Right :) However, there are a few things that you should keep in mind.

Maintenance of the database schema

In our simple example, we only had to add one extra column to our products table as we were only supporting one other language. But let’s move onto something more realistic. Imagine you’re developing a “typical” web application with something like 10 models and which supports 6 different languages. And there an average of at least 5 translatable attributes per model. That works out to 300 extra fields in your tables. You can imagine the scope for errors here. Now imagine you’ve got a much larger application, schema evolution…extrapolate? :(

This is where the external mechanism’s advantage kicks in. No matter how many additional languages you want to support, or how your schema evolves, you never need to modify the schema to support these. With the internal mechanism, you have to be alert and ensure you add the right columns with the right suffix as your schema evolves.

Luckily two things go a long way to reduce the pain of this:

ActiveRecord Migrations
the for-1.2 rake tasks that automatically generate a migration to add the required columns. (The subject of my next article)

Database row limitations

The fact that you’re stuffing a table full of duplicated columns means that for some databases you may begin to reach certain limits, especially if those columns are blobs. e.g. MySQL InnoDB tables have a limit on row size that is easily reached by globalizing a table with about 4 varchar/text columns when you have to support 4 different languages. In this case, you need to switch to MyISAM tables which means you lose transactions (or you could switch your db to postgres or something like that).

So it’s important you keep these points in mind when deciding on which storage mechanism to use.

Conclusion

Phew…That’s was quite a long article. Anyway I may have a very heavy writing style and may tend to overwrite but if anything I hope this has been a fairly enlightening walk through globalize’s new storage mechanism.

As I mentioned in this article, I’ve also written a couple of tools which should make migrating your existing globalized application to this system a breeze. Look out for the next article in this series where I’ll be writing a short tutorial on how to use these tools. It’ll be a lot shorter, I promise :)

I'm a globalize committer

2007-03-16T11:20:00Z

Thanks to the work I did on the alternative storage mechanism for globalize , Josh Harvey invited me on to the Globalize team.

I’m happy to announce that the Globalize team have recently released Globalize For-1.2 which provides full compatibility with Rails 1.2.x and introduces a number of new features some of which I wrote.

In the following series of articles I’m going to be writing about the work I’ve done on Globalize:

My first task was to integrate my alternative storage plugin into globalize itself (available in the for-1.2 branch )

I then kept myself busy by writing a few tools to help the migration over to this new storage mechanism (partly available in the for-1.2 branch )

I then introduced the concept of namespaced view translations (available in the for-1.2 branch )

Recently I’ve added a class that encapsulates the idea of an application’s supported locales to globalize trunk.

From this work and work done while developing our latest application I’ve also recently released a plugin that Globalizes Mephisto (http://mephistoblog.com)

P.S. (Shameless plug) If anyone out there would like help globalizing their site/application then we’re available for hire right now

Introducing Capistrano Server Extensions

2007-03-16T09:50:00Z

I’ve just released a cool tool :) lamely called Capistrano Server Extensions. It’s a Capistrano extension based on the deprec gem and allows you to very easily setup full rails stacks on Ubuntu servers using any combination of Nginx / Apache / MySQL or PostgreSQL

e.g.


  $ cap prepare_ssh
  $ cap prepare_host

That’s it (more or less :)

For the full article read Announcing Capistrano Server Extensions.

I should take this opportunity to also announce that along with super fantastic Thomas Maas I’m a Web Type too. So I’ll be writing articles on both these sites.

Dotime doing well

2007-01-19T14:06:00Z

Out of curiosity to see how many other people were interested in DoTime, I took a look the gem stats

I was surprised to see that dotime has been downloaded a total of 50 times.

That makes me happy. I’m going to fix some more bugs soon and make another alpha release. I also have to check wether basecamp has updated their api much lately and see if I can optimize it a bit more.