I created a simple web page to compare the performance of various localization methods for PHP. It only contains 3 localized strings and does not use advanced features of gettext (e.g. plurals). I wrote a version using the gettext PHP extension (“gettext Ext.”), one using PHP-gettext (“gettext PHP”, a gettext implementation written in pure PHP) and a version that does not use gettext at all, instead it uses an array that contains all the translations (“String ID”).

I put all three pages on a Debian machine with the latest Apache 2.0.55 and PHP 5.1.2 and used the Apache HTTP server benchmarking tool to measure the performance of the different methods. I always made two tests – using the default locale (English) and a translation (German), because gettext does not have to use a locale file for the default language (it’s embedded in the page).

The results

Here are the results (requests per second, more is better, I used “ab -n 5000 URL”):

As you can see, the version using the PHP gettext extension is the fastest solution. It is only marginally slower when using a language file, because the extension caches the translations (the downside is that you have to restart the webserver when you change a locale file). The String ID version is equally fast for either locale, because it always has to lookup the text in the locale array. The pure PHP gettext implementation is the slowest solution, and even slower when it needs to use a locale file. This is understandable because it always has to read the whole file for every request.

You can download the test files if you’re interested.

The gettext extension

When using the gettext extension on Linux, make sure all used locales are installed on the system. For example, in Debian (and probably other distributions, too), add the required locales to /etc/locale.gen and run locale-gen. For this test, I added “de_DE.UTF-8 UTF-8″ for the German UTF-8 version.

Conclusion

The native gettext extension for PHP performed best in the benchmarks. Using the gettext Extension allows you to create clean code by only wrapping strings in _() and including a file that loads the localizations. As I already wrote in my previous post, gettext also allows painless updates of the localizations by automatically finding new or changed strings. The only downside is that the webserver must be restarted to load the new language files. This can be a problem for users on shared hosts. In these cases, a pure PHP implementation can be used (which is not very fast though).

If you can restart the webserver, I recommend using gettext. It is very useful especially for larger projects or projects where not all translations can be updated simultaneously, because it always provides a fallback language. It also makes the source code much easier to read, because the default texts are directly in the source code.

I have now extended Scintilla’s Ruby lexer and Notepad2 to support seperate syntax highlighting for symbols, instance variables, class variables and global variables. I also added the keywords “private”, “protected” and “public” to Notepad2′s keyword list.

Download the new, modified Notepad2. This version includes English and German binaries and the complete source code including Scintilla.

Update: I have made some more small modifications to the Ruby lexer which fix some bugs. A single ‘:’ character is now no longer highlighted as a symbol, and expressions such as “@test.x” are now highlighted correctly (@test as a local variable, . as an operator and x as an identifier). Download the fixed version from above.

Download modified Notepad2 (includes English and German binaries + source code)

This version comes with syntax highlighting support for HTML, XML, CSS, JavaScript, VBScript, ASP, PHP, CSS, Perl, C, C++, C#, Java, Visual Basic, Pascal, Assembler, SQL, Python, NSIS, INI, REG, INF, BAT, DIFF, Ruby, YAML, Bash and Apache configuration files.

Update: I have replaced the download with a newer version which includes an improved Ruby lexer. See my post Improved Ruby syntax highlighting for Notepad2. If you need the old version, here it is.

The MelCache DLL (Melaxis.MelCache.dll) is signed with a strong name and can be installed in the GAC. It requires ICSharpCode.SharpZipLib.dll (included in the ZIP file).

Download MelCache (Documentation included)

You can use MelCache in freeware and open source projects without restriction. Please contact me if you want to use it otherwise. If you find any bugs in MelCache or if you have an idea how it could be improved, please leave a comment.

Download MultiWP 1.1

Using gettext with PHP

A widely used framework for internationalization is gettext. It can be used with a variety of programming languages, including PHP. There are basically two ways to use gettext in your PHP applications. You can use the native gettext PHP extension or you can use a library implemented in PHP that does not need any extension, such as php-gettext. I will use the native PHP extension, but once you have read the post you should be able to use the php-gettext-library, too (have a look at the included example).

Using gettext in your application

Using gettext to get translated strings couldn't be easier. Just call gettext("Text to be translated") and you get a localized version of "Text to be translated" if available, or "Text to be translated" otherwise. If you're lazy, you can use _() instead of gettext().
Let's try this out. Create a new PHP file (we'll call it test.php), and insert the following code:

When you open that page in your browser, you will see "Hello World!".

Localizing your application

Now that you have created a first version of your script, you may want to create localized versions for different languages. In order to do that, you either need the gettext utilities (windows version) or the graphical editor poEdit, which we will be using. So install and launch poEdit. Create a new catalog (File -> new catalog). Choose the language you want to translate the application to. I'm going to use German, GERMANY and iso-8859-1 for both character sets. The last option (plural forms) is an advanced feature of gettext which I'm going to explain later, so for now, just leave it blank. On the paths tab, set the base path to the directory containing your test.php file, and add "." as a path. On the keywords tab you can add names of additional functions that call gettext. You may want to use this if you're using php-gettext. Now click OK to open the save dialog. Now create a sub folder called "locale" in your script directory. Create a subdirectory inside locale for every language you support. We'll create one for "de_DE" (the first part is the language, the second part is the country). Inside that folder create another one called "LC_MESSAGES". You should now have a directory structure like locale/de_DE/LC_MESSAGES/. Save the file inside that directory as "messages.po". poEdit will now automatically scan all source files inside the path you specified earlier and extract all strings that are passed to gettext() or _() (or any other methods you may have added in the keywords tab). Click OK. Now that's cool! poEdit just extracted all strings you want to be localized.

In the upper half of the poEdit window, you have a list of strings (the original string on the left and your translation on the right). Select the first string in the list. In the lower left hand corner, you have 2 text boxes. The first one contains the original string, and the second one is still empty. Type your translation into that box. I'll enter "Hallo Welt!". If you had more entries in your file, you could navigate between them by pressing ctrl-up and ctrl-down. Save the file. poEdit automatically created "messages.mo" in the same directory as "messages.po". This is the compiled version that will later be used by PHP.

Initializing the gettext library

Let's see if our script can now display the localized string. The first thing you have to do is telling the gettext library which locale you want to use and where the language files are stored. Let's create a new file called "localization.php" that will handle all the gettext initialization. Copy the following code into localization.php (Note: this code is for the php gettext extension, php libraries such as php-gettext may require different initialization):

In the first line, we set the default locale to "de_DE". The second line allows you to override this by appending ?locale=... to the URL. You should replace this by real code to select the locale (perhaps by looking at the Accept-Language header), but it works for our tutorial. The next two lines actually set the current locale to that value and the bindtextdomain call creates a text domain which will use our messages.mo file in the locale directory. The textdomain function selects that domain as the default domain.

Include that file in your test.php by adding a require_once("localization.php"); line to the top of the file. Now reload test.php in your browser and it should now say "Hallo Welt!". Now try test.php?locale=en_US and it should say "Hello World!". You have just created a multi language PHP script!

Updating your application

Let's add another string to the script. I'll add echo _("Welcome to my test page");. When you reload the page with the German locale, you will see that Hello World is localized, while the second string is not. Let's change that. In poEdit (reopen the .po file if you have closed it), click the update button (that's the one with the wheel). A new dialog should show one new string. Click OK, translate it (in German it would be "Willkommen auf meiner Testseite") and save the file. Reload test.php in your browser. Nothing has changed? When using the PHP gettext extension, .mo files are cached by the gettext library. The only way I have found to clear the cache is to restart the web server. Once you have done that, both strings should be translated.

Using plurals

One last thing I want to discuss is gettext's plural support. This may seem a bit complicated at first, but is actually very powerful. Sometimes you need a string that supports plural forms. A typical example would be "0 comments", "1 comment", "2 comments", "3 comments"... In English you can just add an 's' if the number n != 1. However, in other languages it's not that easy. gettext supports all these cases. In your scripts you use something like this:

If you have never used (s)printf before, have a look at it's documentation. The interesting part is of course the ngettext call. ngettext takes three arguments. The first one is the singular string, the second one the plural string and the last one is the number. While that was quite easy, localizing these strings is a bit harder. Add the two lines to your test.php script.

Before you update your language file in poEdit, open the catalog options window (catalog -> options). Now let's fill in that last field. For our German locale, set the plural forms field to "nplurals=2; plural=(n != 1);". Essentially this specifies that there are two plural forms and the plural form is determined by the expression (n != 1). This evaluates to 0 if n == 1 and 1 if n != 1. For more information about this field, including samples for various languages, please see the gettext documentation. Now, let's press the update button again and select the new string. You will notice that the lower left hand corner of the window has changed. It now displays the original singular and plural forms and the translation box has been replaced by two (depends on the number of plural forms the language has). Enter "%d Kommentar" in the first tab and "%d Kommentare" in the second tab. Save the file, reload the web server if needed and your page should show the correct translation (you may want to change the value of $n to see the effect of the ngettext function).

Improving localization in your scripts

This concludes my introduction to localizing PHP web sites using gettext. Here are a few ideas to improve localization in your applications:

• During development, you may want to use a php library instead of the PHP gettext extension because they do not require you to restart the web server every time you modify the language files. You can create a wrapper function (e.g. __()) that calls either the library function or the extension, depending on which is available or selected). You can add the name of the wrapper function as a keyword in poEdit, so that it is recognized.
• You may want to select a default locale according to the browser's Accept-Language header, so that the user does not have to select his language first.

The performance of gettext

See my follow-up post "Benchmarking PHP Localization - Is gettext fast enough?" for Benchmarks. In general, the gettext Extension is faster than using a String-Array. The pure PHP implementation of gettext is slower and not recommended if you can use the PHP Extension.

So far, I created all my dynamic web sites using PHP. Writing a simple application such as a guest book requires quite a lot of code in PHP to connect to the database, retrieve the entries, add new entries and so on. With Ruby on Rails that's much easier. You simple create a table, run a generator script to create a model and a controller. A cool feature called scaffolding automatically creates an interface to list, add, edit and remove entries. This is very useful to add some entries to a new table before you have created the real interface. But even that is really simple.

For example, by adding a line like "@entries = Entry.find_all" to a controller action, Ruby on Rails fetches all rows of the entries table (that's another cool feature: table names are lowercase and plural, model names uppercase and singular - Ruby on Rails can convert them automatically) and saves them in a variable called entries. In a template with the same name as the action, you can loop trough all entries using "<% @entries.each do |entry| %>" (Ruby code is embedded in <% %>-Blocks inside the HTML template) and then print them easily using "<%= entry.message %>" etc.

Ruby on Rails is really cool. In order to understand Rails, I decided to learn Ruby first. So far, I can recommend two links for learning Ruby. The first one is the Ruby user's guide by matz, the creator of the language, which provides a rather short introduction to Ruby. The other one is Why's (Poignant) Guide to Ruby, which is quite entertaining.
O'Reilly has a tutorial for Ruby on Rails: Rolling with Ruby on Rails (this includes installation instructions for Ruby, Ruby on Rails and MySQL) and Rolling with Ruby on Rails, Part 2. More tutorials and API references can be found on the Ruby on Rails Homepage.