I thought it might be interesting to take a look back at the technical evolution of Mozilla’s localization systems. Having personally navigated every version — first as a community localizer from 2004 to 2013, and later as staff — I’ll share my perspective. That said, I might not have all the details exactly right (or I may have removed some for the sake of my sanity), so feel free to point out any inaccuracies.

Early days: Centralized version control

Back in the early 2000s, smartphones weren’t a thing, Windows XP was an acceptable operating system — especially in comparison to Windows Me — and distributed version controls weren’t as common. Let’s be honest, centralized version controls were not fun: every commit meant interacting directly with the server. You had to remember to update your local copy, commit your changes, and then hope no one else had committed in the meantime — otherwise, you were stuck resolving conflicts.

Given the high technical barriers, localizers at that time were primarily technical users, not discouraged by crappy text editors — encoding issues, BOMs, and other amenities — and command line tools.

To make things more complicated, localizers had to deal with 2 different systems:

• CVS (Concurrent Versioning System) was used for products like Mozilla Suite, Phoenix/Firefox, etc. To increase confusion, it used branch names that followed the Gecko versions (e.g. MOZILLA_1_8_BRANCH), and those didn’t map at all to product versions. Truth be told, the whole release cadence and cycle felt like complete chaos back then, at least as a volunteer.
• SVN (Subversion) was used to localize mozilla.org, addons.mozilla.org (AMO), and other web projects.

With time, desktop and web-based applications emerged to support localizers, hiding some of the complexity of version control systems and providing translation management features:

• Mozilla Translator (a local Java application. Yes kids, Java).
• Narro.
• Pootle.
• Verbatim: a customized Pootle instance run by Mozilla, used to localize web projects like addons.mozilla.org. This was shut down in 2015 and projects transitioned to Pontoon.
• Pontoon (here’s the first idea and repository, if you’re curious).
• Aisle, an internal experiment based on C9 that never got past the initial tests.

This proliferation of new tools led to a couple of key principles that are still valid to this day:

• The repository, not the TMS (Translation Management System), is the source of truth.
• TMSs need to support bidirectional synchronization between their internal data storage and the repository, i.e. they need to read updated translated content from the repository and store it internally (establishing a conflict resolution policy), not just write updates.

This might look trivial, but it’s an outlier in the localization industry, where the tool is the source of truth, and synchronization only happens in one direction (from the TMS to the repository).

The shift to Mercurial

At the end of 2007, Mozilla made the decision to transition from CVS to Mercurial, this time opting for a distributed version control system. For localization, this meant making the move to Mercurial as well, though it took a few more months of work. This marked the beginning of a new era where the infrastructure quickly started becoming more complex.

As code development was happening in mozilla-central, localization was supposed to be stored in a matching l10n-central repository. But here’s the catch: instead of one repository, the decision was to use one repository per locale, each one including the localization for all shipping projects (Firefox, Firefox for Android, etc.). I’m not sure how many repositories that meant at the time — based on the dependencies of this bug, probably around 30 — but as of today, there are 156 l10n-central repositories, while Firefox Nightly only ships in 111 locales (a few of them added recently).

The next massive change was the adoption of the rapid release cycle in 2011:

• 3 new sets of repositories had to be created for the corresponding Firefox versions: l10n/mozilla-aurora, l10n/mozilla-beta, l10n/mozilla-release.
• Localizers working against Nightly in l10n-central would need to manually move their updates to l10n/mozilla-aurora, which was becoming the main target for localization.
• At the end of the cycle (“merge day”), someone in the localization team would manually move content from Aurora to Beta, overwriting any changes.
• In order to allow localizers to make small fixes to Beta, 2 separate projects were set up in Pontoon (one working against Aurora, one against Beta), and it was up to localizers to keep them in sync, given that content in Beta would be overwritten on merge day.

If you’re still trying to keep count, we’re now at about 600 Mercurial repositories to localize a project like Firefox (and a few hundreds more added later for Firefox OS, one for each locale and version, but that’s a whole different story).

I won’t go into the fine details, but at this point localizers were also supposed to “sign off” on the version of their localization that they wanted to ship. Over time, this was done by:

• Calling out which changeset you wanted to ship in an email thread.
• Later, requesting sign-off in a web app called Elmo (because it was hosted on l10n.mozilla.org, (e)l.m.o., got it?). Someone in the localization team had to manually go through each request, check the diff from the previous sign-off to ensure that it would not break Firefox, and either accept or reject it. For context, at the time DTDs were still heavily in use for localization, and a broken translation could easily brick the browser (yellow screen of death). 
• With the drop of Aurora in 2017, the localization team started reviewing and managing sign-offs in Elmo without waiting for localizers to make a request. Yay for localizers, one less thing to do.
• In 2020, partly because of the lay-offs that impacted the team, we completely dropped the sign-off process and decommissioned Elmo, automatically taking the latest changeset in each l10n repository.

The new kid on the block: GitHub

In 2015 we started migrating repositories from SVN to GitHub. At the time, that meant mostly web projects, managed by Pascal Chevrel and me, with the notable exception of Firefox for iOS. That part of localization had a whole infrastructure of its own: a web dashboard to track progress, a tool called langchecker to update files and identify errors, and even a file format called dotlang (.lang) that was used for a while to localize mozilla.org (we switched to Fluent in 2019).

The move to GitHub removed a lot of bureaucracy, as the team could create new repositories and grant access to localizers without going through an external team, like it was the case for Mercurial. Still today, GitHub is the go-to choice for new projects, although the introduction of SAML single sign-on created a significant hurdle when it comes to add external contributors to a project.

Introduction of cross-channel for Firefox

Remember the 600 repositories? Still there… Also, the most observant among you might wonder: didn’t Mozilla had another version of Firefox (Extended Support Release, or ESR)? You’re correct, but the compromise there was that ESR would be string-frozen, so we didn’t need another ~150 repositories: we used the content from mozilla-release at the time of launch, and that’s it, no more updates.

In 2017, the Aurora channel was “removed”, leaving Nightly (based on mozilla-central), Developer Edition and Beta (based on mozilla-beta), Release (based on mozilla-release) and ESR. I use quotes, because “aurora” is still technically the internal channel name for Dev Edition.

That was a challenge, as Aurora represented the main target for localization. That change forced us to move all locales to work on Nightly around April 2017. 

Later in the year, Axel Hecht came up with a core concept that still supports how we localize Firefox nowadays: cross-channel. What if, instead of having to extract strings from 4 huge code repositories, we create a tool that generates a superset of the strings shipping in all supported versions (channels) of Firefox, and put them in a nimble, string-only repository? That’s exactly what cross-channel did, allowing us to drop ~300 repositories (plus ~150 already dropped because of the removal of Aurora). It also gave us the opportunity to support localization updates in release and ESR. At this point, localization for any shipping version of Firefox comes out of a single repository for each locale (e.g. l10n-central/fr for French).

In hindsight, cross-channel was overly complex: it would not only create the superset content, but it would also replay the Mercurial history of the commit introducing the change. The content would land in the cross-channel repository with a reference to the original changeset (example), making it possible to annotate the file via Mercurial’s web interface. In order to do that, the code hooked directly into Mercurial internals, and it would break frequently thanks to the complexity of Mozilla’s repositories. In 2021 the code was changed to stop replaying history and only merging content.

At this point, in late 2017, Firefox localization relied on ~150 l10n repositories, and 2 source repositories for cross-channel — one used as a quarantine, the other, called gecko-strings, connected to Pontoon to expose strings for community localization.

Current Firefox infrastructure

Fast-forward to 2024, with Mozilla’s decision to move development to Git, we had an opportunity to simplify things even further, and rethink some of the initial choices:

• Instead of 2 repositories for cross-channel, we decided to use only one repository with 2 branches. The cross-channel code was completely rewritten by Eemeli Aro and now runs as a GitHub workflow.
• Instead of ~150 repositories, we now have a single l10n repository, covering all supported versions of Firefox as l10n-central used to do. All locales, except for Italian and Japanese, are localized through Pontoon.

Thunderbird has adopted a similar structure, with their own 3 repositories.

The team completed the migration to Git in June, ahead of the rest of the organization, and all current versions of Firefox ship from the firefox-l10n repository (including ESR 115 and ESR 128).

Conclusions

So, this was the not-so-short story of how Mozilla’s localization infrastructure has evolved over time, with a focus on Firefox. Looking back, it’s remarkable to see how far we’ve come. Today, we’re in a much better place, also considering the constant effort to improve Pontoon and other tools used by the community.

As I approach one of my many anniversaries — I have one for when I started as a volunteer (January 2004), when I became a member of staff as a contractor (April 2013), one “official” when I became an employee (November 2018) — it’s humbling to think about what a small team has accomplished over the past 22 years. These milestones remind me of the incredible contributions of so many brilliant individuals at Mozilla, whose passion helped build the foundations we stand on today.

It’s also bittersweet to go back and read emails from over 15 years ago, remembering just how pivotal the community was in shaping Firefox into what it is today. The dedication of volunteers and localizers helped make Firefox a truly global browser, and their impact is still felt — and sometimes missed — today.

Premise: I’ve been obese or overweight for most of my life. My best guess is that I got close to 115 kg at my heaviest, and I would be considered tall only for a hobbit basketball team. I started keeping track of my weight when I was about 20 years old, and decided to do something about it, also because my knees and ankles were falling apart. Ignoring some set-backs – the dreaded yo-yo effect – I consider my efforts quite successful.

I’ve learned a few things along the way. Not necessarily in order of importance:

• Drastic changes won’t last. You can starve yourself and lose weight, but you’ll get that weight back, and some more. It’s not a matter of “if” but “when”.
• Find a sport that you like, and use it to balance the calories intake instead of relying on drastic diets.
• Learn to cook, stop eating crap.
• Keep an eye on your weight. Stepping on a scale every day is not needed, once or twice a week is good.

But, in the end, it’s mostly about finding something that works for you. Which brings me to the title of this post: intermittent fasting (or IF).

A few years ago, I saw this documentary from BBC, called Eat, Fast, Live Longer (by Michael Mosley). I was quite skeptic at the beginning, but this turned out to be the “thing that works for me”.

The concept is simple: you have periods where you can eat (often called “windows”), alternated to periods where you don’t eat anything, and you’re only allowed to drink water, tea, or coffee (without sugar or other additives). Some folks are less strict about the fasting period, but I personally stick to water, occasionally espresso coffee.

There are different intermittent fasting patterns, the most common being:

• 16:8 – You eat within a 8 hours window, then fast for 16 hours straight.
• 18:6, 20:4 – Same concept, longer fasting period. The more extreme version is OMAD (one meal a day).
• 5:2 – You eat for 5 days, then fast for 2 days straight (sounds fun, doesn’t it?).

I tried 16:8 a few years back, skipping breakfast, but it wasn’t really working for me. About a year ago, I noticed my weight slowly but worryingly increasing (did I mention to keep track of your weight?), and decided to give it a try again. But this time I would keep track of what was happening, and how good I was at sticking to my goals.

Normally I eat breakfast quite early in the morning (often well before 7AM), have lunch around 12, then a closing snack around 2PM. In case of social events, e.g. dinners, I would try to rotate my schedule, but without losing my mind if I couldn’t fast for 16 hours for a couple of days.

Here are the results of 1 year worth of tracking weight and number of fasted hours (using an Android app called Zero), averaged by week.

On top of helping me lose more weight that I hoped for at my age, I realized how much IF helped me in COVID-19 times, while I work from home and personally cook every meal I eat. Here are a few benefits:

• Having one meal per day helps limiting the calories intake, assuming weight loss is your goal.
• I only need to cook – and wash dishes – once per day. It might seem a small thing, but working with a US company means that 4-8PM are my busiest hours of the day. So, dinner often ended up consisting of unhealthy food that didn’t require too much time or effort.
• I sleep a lot better, since I don’t end up in bed with my stomach still working on digestion.
• Energy levels are unaffected, or possibly positively affected. For example, earlier this year I lowered my half-marathon personal best by 6 minutes (around 1:50), and I’ve already exceeded my overall running goal of 1300 km for 2020.

OK, but what are the negatives?

• It works well for me because I live alone, and I’m not exactly a social animal. If you have to cook for someone else, or meals represent a social gathering in your family, then you need to adapt your schedule accordingly. And that means no breakfast, which is OK for some, terrible for others.
• It would be challenging to organize my sport activities around a different schedule: running fasted is quite OK, but you are supposed to eat soon after exercising, and my body doesn’t seem to like running in the afternoon.
• Medical conditions (e.g. diabetes) might require extra care, and IF could just not be suitable for some folks.

As they say, “your mileage might vary”, but maybe it’s worth a try now that you’re likely stuck at home.

With Firefox 58 we started moving Firefox Preferences to Fluent, and today we’re migrating the last pane (Firefox Account – Sync) in Firefox Nightly (61). The work is not done yet, there are still edge cases to migrate in the existing panes, and subdialogs, but we’re on track. If you’re interested in the details, you can read the full journey in two blog posts from Zibi (2017 and 2018), covering not only Fluent, but also the huge amount of work done on the Gecko platform to improve multilingual support.

At this point, you might be wondering: do we really need another localization system? What’s wrong with what we have?

The truth is that there is a lot wrong with the current systems. In Gecko alone, we support 4 different file formats to localize content: .dtd, .properties, .inc, .inc. And since none of them support plural forms, we built hacks on top of .properties to support pluralization.

Here are a few practical examples of why Fluent is a huge improvement over existing technologies, and will allow us to improve the quality of the localizations we ship.

DTDs and Concatenations

You want to localize this simple fragment of XUL code without using JavaScript.

 Please sign in to reconnect 

This turns into 2 separate strings in a DTD file, and a long localization comment:

Why the empty string at the end? Because, while English doesn’t need it, other languages might need to change the structure of the sentence, adding content after the email address. On top of that, some localization tools don’t support empty strings correctly, not allowing localizers to mark an empty translation as a “translated” string.

In Fluent, this is simply:

sync-signedin-login-failure = Please sign in to reconnect { $email }

One single string, full visibility on the context, flexibility to move around the email address.

Plural Forms

Plural forms are supported in Gecko only for .properties files. Fluent supports plural forms natively, and with a lot of additional flexibility.

First of all, if you’re not familiar with the complexity of plurals across languages (limiting the discussion to cardinal integer numbers):

• English, like many other European languages, only has 2 plural forms: n=1 uses one form (“1 page”), all other numbers (n!=1) use a different form (“2 pages”). Sadly, this makes a lot of people think about plural in terms of “1 vs many”, while that’s not really the case for most languages.
• French still has 2 plural forms, but uses the same form for both 0 and 1.
• Other languages can only have one form (e.g. Chinese), or have up to 6 different plural forms (e.g. Arabic). Fluent uses the CLDR categories (zero, one, two, few, many, other) to match a number to the correct plural form. For example, in Russian 1 and 21 will use the form “one”, but 11 will use “other”.
• The behavior might change if the actual number is present or not. For example, Turkic languages don’t need to pluralize a noun after a number (“1 page”), but need plural forms in sentences referencing to one or more elements (“this” vs “them”).

Consider for example this use case: in Firefox, the button to set the home page changes from “Use Current Page” to “Use Current Pages”, depending on the number of open tabs.

If you want to use a proper plural form, you need to add the number of tabs to the string. In .properties, it would look like this (plural forms are separated by a semicolon):

use-current-pages = Use Current Page;Use Current #1 Pages

This will force languages to create all plural forms for their locales, even if they might not be needed. If your language has 6 forms, you need to provide all 6 forms, even if they’re all identical. Fun, isn’t it? Note that this is not just a limitation of the plural system used in .properties, the same happens in GetText (.po files).

Here’s how Fluent improve things: first, you don’t need to add all plural forms, you can rely on the fall back to the default value (indicated by *), without raising any error:

             
use-current-pages =
     .label =
         {
            *[other] Использовать текущие страницы
         } 

More important, you can match a specific number (1 in this case):

  
use-current-pages =
     .label =
         {
             [1] Использовать текущую страницу           
            *[other] Использовать текущие страницы
         } 

In Russian, the “[one]” form would be also used for 21 tabs, while here it’s only used for 1 tab.

Let’s assume the English message looks like this:

use-current-pages =
     .label =
         { $tabCount ->
             [1] Use Current Page
            *[other] Use Current Pages
         }

Do you need to expose the number in your message and treat it like a standard plural form? You can:

use-current-pages =
     .label =
         { $tabCount ->
             [one] Use { $tabCount } Page
            *[other] Use { $tabCount } Pages
         }

Do you only need one form? Again, you can simplify it into:

use-current-pages =
     .label = Use { $tabCount } Pages

or even:

use-current-pages =
     .label = Use Current Pages

Variants

This is one of the most exciting changes introduced to the localization paradigm.

Consider this example: “Firefox Account” is a special brand within Firefox. While “Firefox” itself should not be localized or declined, “account” can be localized and moved. In Italian it’s “Account Firefox”, “Cuenta de Firefox” in Spanish.

A special entity is defined in order to be reused in other strings:

For example:

In Italian this results in “Connetti &syncBrand.fxAccount.label;”. It’s not natural, and it looks wrong, because we don’t capitalize nouns in the middle of a sentence.

My only option to improve the translation, and make it sound more natural, would have been to drop the entity and just add the translated name. That defies the entire concept of having a central definition for the brand.

Here’s what I can do in Fluent. The brand is defined as a term, a special type of message that can only be referenced from other strings (not code), and can have additional attributes.

-fxaccount-brand-name = Firefox Account
sync-signedout-account-title = Connect with a { -fxaccount-brand-name }

And now in Italian I can do:

-fxaccount-brand-name =
    {
        [lowercase] account Firefox
       *[uppercase] Account Firefox
    }
sync-signedout-account-title = Connetti il tuo { -fxaccount-brand-name[lowercase] }

While uppercase vs lowercase is a trivial example, variants can have a much deeper impact on localization quality for complex languages that use declensions, where the word “account” changes based on its role within the sentence (nominative, accusative, etc.).

This is only the tip of the iceberg, there’s more you can do with Fluent, and the new localization API will allow us to drastically improve the experience for non English users in Firefox. Here are some additional links if you want to learn more about Fluent:

• Project Fluent
• Fluent documentation for Firefox Developers
• Fluent documentation for localizers

jooliaan, even if he hasn’t been active for a few years, had a crucial role in the growth of the Italian community and the creation of the legal association known as Mozilla Italia, as Vice President first, then President, and administrator of the local support forum. If you’re using Firefox in Italian, or seeking and getting help on the forum, it’s also thanks to his work and dedicated contribution.

Today we’re sharing in the pain of his family and friends, remembering a person, a friend, who’s given a fundamental contribution to the history of the Open Source in Italy and Mozilla, and to our own lives.

Today I’ve decided to give PHP 7 a try on my local virtual machine: it’s a pretty light VM running Debian 8 (2 GB ram, 2 cores), hosted on a Mac with VMWare Fusion. Web pages are served by Apache and mod_php, not by PHP’s internal web server, and there’s no optimization of any kind on the config.

Since I and my colleague Pascal spent quite a bit of time improving code quality and performances in our tools during the last year or so, I already had a script to collect some performance data. I ran the test against Langchecker, which is the tool providing data to our Webdashboard.

Existing version: PHP 5.6.14-0+deb8u1
New version: PHP 7.0.0-5~dotdeb+8.1

Time needed to generate the page

Memory consumption

While the number of runs was pretty small (50 for each request), I got decently consistent results for memory and time. And the results speak clearly about the performance improvements in PHP 7.

I don’t personally consider PHP 7 ready for a production environment but, if you can, it’s definitely worth a try. In our case the code didn’t need any change.

This extension, currently at version 0.4.1, is localized in 16 languages: de, es-ES, fi, fr, he, it, ja, ko, pl, pt-BR, pt-PT, ru, sk, sr, th, zh-TW.

For the first time in years, I’m going to release a new version that includes new features, and therefore new strings. While I obviously love localization and localizers, I don’t intend to work with localization platforms for an add-on with a very limited set of strings and infrequent updates.

The new version (v0.5.0) will be released with all the existing languages enabled, but the new strings will be left in English (excluding Italian). Starting from the next version, I will drop locales that go below 60% unchanged strings (currently it’s about 70% for all locales).

If you want to contribute updating your localization:

• Source code is hosted on GitHub. Localizations are stored in /extensions/locale. If you’re already familiar with GitHub, great. If you’re not, you can try with this tutorial or try using the online editor available on GitHub.
• If you want to receive an email before the next release, in case I have to add new strings and you want to localize them before release, send me an email at flod(at)lodolo(dot).net and I’ll organize a mailing list.

“Subtitles” assume the viewer can hear but cannot understand the language or accent, or the speech is not entirely clear, so they only transcribe dialogue and some on-screen text. “Captions” aim to describe to the deaf and hard of hearing all significant audio content—spoken dialogue and non-speech information such as the identity of speakers and, occasionally, their manner of speaking – along with any significant music or sound effects using words or symbols.

So far I worked on two projects that involved subtitles – Web We Want and Firefox: Choose Independent – and this is what I learned in the process.

The Process

Step 1: Provide Source Content

You need the video (obviously) and English subtitles with the correct timing (less obvious). This means that the picture might no be final (sometimes they call this picture lock quality), but the audio track and its timing do.

Step 2: How Do I Localize Subtitles?

Currently the most common format for subtitles is SubRip, and it’s really simple: sequential number of the subtitle, timing (start-end), text.

For example this is the beginning of the Web We Want .srt file:

1
00:00:00,000 --> 00:00:00,864
THE WEB WE WANT

2
00:00:00,864 --> 00:00:03,388
THE WEB WE WANT
an open letter

3
00:00:05,503 --> 00:00:07,406
I am not a data point

4
00:00:07,642 --> 00:00:09,503
to be bought and sold.

Amara is a great tool to localize subtitles: you can use an interactive editor/timeline and you can adapt the timing of each sentence to your needs while watching the video. Potentially you can also choose to automatically sync subtitles between Amara and Youtube.

Step 3: Host the Video on Youtube

At this point subtitles are available, someone just needs to load them on YouTube and display the video on a web page.

Sounds simple, doesn’t it? What could go wrong?

Potential Localization Issues

The first issue is timing: if the English text requires 2 seconds to be read, that might not be enough to read the same sentence in Spanish, German or other verbose languages. That’s something you need to keep in mind from day one while producing the video and audio track.

Split sentences: sometimes a sentence is split into multiple parts because of its length. Unfortunately the sentence structure might be completely different in the target language, and some details will get lost in translation. Consider for example the final frame of the Choose Independent video:

«It’s how we keep our independence online…» (pause, switch to burning fox) «…burning bright»

This works nicely in English since “burning” is perfectly synced with the picture, but most locales won’t be able to obtain the same result. Small bits, but still reducing the impact of the message.

In general, watching a subtitled video is a sub-optimal experience. If you plan to have content in video format, don’t focus your communication exclusively on that.

Why Youtube Is not Great

For the Firefox Independent page we force YouTube to load subtitles with cc_load_subtitles.

A few hours after the launch, I started receiving complaints from localizers saying that YouTube was loading subtitles with the wrong localization for some languages: for example nl for fy-NL (Dutch instead of Frisian), de for rm (German instead of Romansh), en-US for cy (English instead of Welsh).

This is how YouTube’s embed works:

• If you’re logged in into Google, you’ll get the locale you choose for YouTube. For example I get English subtitles, even if I’m using a browser in Italian (a quick test is to watch the video in private mode).
• If you’re not logged in, you’ll get the first good locale based on the Accept Language header sent by your browser. For example, for ‘dsb’ (Lower Sorbian) this equals to “dsb, hsb, de, en-US, en”, and German is the first available language on the list.

For some unknown reasons YouTube wasn’t founding a match in Accept Language and falling back to a different language for those locales, even if the subtitles were localized and loaded. My only guess would be a mismatch in the locale format understood by YouTube, like fy_NL vs fy-NL/fy.

Then I found out that we can send a parameter called ‘cc_lang_pref’ to force the language (not exactly well documented), and that fixes cy, fy-NL, rm. For example, if you open https://www.mozilla.org/it/firefox/independent you’ll get the Italian subtitles even if you’re using a browser in a different language. Since we have a good locale detection on top of mozilla.org, this makes sense and let us use the most of our localization teams’ work.

But then YouTube becomes not smart: if the requested locale is missing, it doesn’t rely on Accept Language, but just falls back to English. So Welsh (cy) is now getting subtitles in the expected language, Lower Sorbian (dsb) is getting English instead of German and can only manually switch language. Far from great.

Talking about YouTube, I find quite silly a claim as «I support 163 languages” but don’t provide the list of them anywhere. I have at least 5 locales code that are not supported or recognized (ast, dsb, es-AR, es-CL, hsb).

How Do We Fix It?

I think the solution will be to move away from YouTube and its limitations, use native or alternative video players (example), subtitles in VTT format and take full control of the entire chain. But these are discussion and experiments that still need to start.

mozilla.org

I probably spent most of my last quarter working on mozilla.org updates. My job, together with Pascal, consists in checking new pages for localizability issues at an early stage, extracting strings in .lang files, exposing them to localizers and pushing content to production. Luckily, on the other side there’s a wonderful team of people who help us making sure that localization is always treated as a first class citizen.

The new home page design went live in October, localized in 33 languages on day one (57 as I write this post). That’s a huge accomplishment.

Ignoring smaller updates, this quarter we also had the new Firefox for Android page (42 languages), the redesigned Get Involved page (25 languages, with some bumps on the road), and the new Firefox Developer Edition landing page (34 languages).

And then this tiny thing called #fx10 happened.

Firefox 10th Anniversary

This was an interesting and exhausting experience. For example in order to localize the UI Tour (“Privacy Tour”) for 33.1 I had to manage communications with 26 teams using direct emails. And then organize the localization of subtitles for this video (I’ll talk about this in a separate post, long story). Everything trying to disclose as little information as possible to the public before the launch and avoid spoiling the surprise.

To give some numbers: we usually ask for 2/3 weeks for small localization projects, at least 4 weeks for bigger projects that require work from all ~90 locales supported across platforms (last case was Australis launch).

In this case the longest deadline was 3 weeks (privacy tour), most of the material became available 2 weeks from launch, with some edge cases (48 hours), a lot of changes, unplanned material and averted crises (huge shout-out to Erin Lancaster for coordinating so many teams and initiatives).

Size of the work requested to localizers: a rough estimate varies from a minimum of about 140 strings/1000 words, to about 330 strings/3000 words for bigger locales.

Internal Tools

To track the localization status of web parts we rely on three different tools:

• Langchecker: analyzes SVN repositories hosting strings in .lang format, provides status dashboards, error checks and feeds external tools with results.
• Webstatus: same as langchecker, but for external products/websites (SVN/Github) using Gettext files.
• Webdashboard: it’s the source used by localizers to track pending work (how many strings are missing, deadlines, errors, etc.), displaying information received from langchecker and webstatus.

These are some of the improvements introduced in the last quarter.

Langchecker

I added support for a raw file type, currently used for text files. Unlike .lang file, where we can determine the exact status of localization (translated strings, missing, errors, etc.), we can only do minimal checks for this kind of files: does the file exist in locale X? Is it mandatory or optional? Was it updated before or after the last reference update? The main usage for this is to track localization of autoresponders for the new Get Involved page.

Also added command line scripts to import strings in .lang files using Transvision API, and improved the code detecting errors in translations (e.g. missing Python variables).

Webdashboard

Besides displaying information about raw files, I focused on improving a very useful feature added last summer by our intern Théo, called “Project View”. Basically we can display on a single page the status of several files scattered across different repositories (example) and share it with other teams.

Transvision

I didn’t have much time to work on Transvision this quarter, but I managed to land a change that I consider really important: now the list of supported products and supported locales for each product is fetched from an external source. The practical result is that we don’t have to release a new version of Transvision to support Gaia version X or add a new locale on Aurora, reducing the time necessary to deploy these changes from weeks or months (new manual release) to less than 24 hours (automated cronjob).

I also worked on a view to display “unchanged strings” (i.e. strings identical to English), and some small refactor/clean-up changes.

One my goals for next quarter is to figure out with other team members a plan/schedule to release more often.

Productization

Part of my job is also to take care of productization (p12n), which means landing settings for new locales, as well as organizing mass updates (e.g. switching Wikipedia to SLL) and making sure that we don’t ship anything broken.

Last quarter I did a complete rewrite of the tool I use to extract information from all repositories. The original code was never meant to be shared; for example I realized how counter-intuitive the data structure was only after trying to explain it to another person. It’s still clumsy Python code written by someone who never saw Python until two years ago, but now with tests

The same tool is at the base of this view on Transvision, or images like this one (all searchplugin images we’re shipping on Firefox Beta across locales).

Miscellanea

As part of my daily job I also check all strings landing on master/trunk branches for Gaia, Firefox and Firefox for Android, and file bugs for any localizability issues. Some interesting things happened there (e.g. the new search UI), but that’s a discussion probably worth a different post (and more thinking on my side).

I also briefly helped Mozilla Foundation organizing the localization process of the EOY fundraising campaign. I really hope we’ll manage to organize this better next year, and involve localization teams in the process if they have resources to help.

according to the desktop metrics I had, French is the least compact locale and Chinese is the most compact one

So I had to check it somehow (in Italy they would call me Saint Thomas for being skeptical).

The basic idea was simple: use Silme to analyze all locales available in Mozilla l10n repositories, comparing string lengths between English and another language.

Here’s the resulting Python script (beware my slowly improving programming skills) and a table with the results (data can be sorted by clicking on column headers).

Sample and Reference

I’m using mozilla-beta as a reference, and comparing each locale against en-GB. Why not en-US? The reason is simple: en-US strings are scattered across the entire mozilla-central repository, so I should do tricks like Transvision in order to create a pseudo en-US string-only repository. Using en-GB leads to less precise results (see below), but for the sake of this analysis I considered it an acceptable compromise.

I’m not checking all folders, only the main ones (‘browser’, ‘dom’, ‘mail’, ‘mobile’, ‘netwerk’, ‘security’, ‘services’, ‘suite’, ‘toolkit’, ‘webapprt’). This still generates an archive of almost 18,000 strings for locales translating all products, so it seems a decent sample.

Caveats and Weird Results

String 1: en-GB 2 characters, locale X 4 characters -> +2 characters, +100%
String 2: en-GB 8 characters, locale X 4 characters -> -4 characters, -50%
Average for locale X: -1 characters, +25% (sum of differences divided by total number of items).

Not sure if this is the best choice, but I couldn’t think of an alternative. Note also that I’m ignoring single character strings (access keys, shortcuts).

In the table you’ll see a global column (average results) and “buckets”, with string groups based on en-GB original length. Too bad these groups are often unreliable because of the “concatenation conundrum”, where one string could be created by concatenating 3 different labels.

Typical example to create a sentence with a link (note that concatenation should be always avoided):

sentence.before = Hey, this is a
sentence.link = very interesting link
sentence.after = .

In Italian this could be localized as

sentence.before = Ehi, questo
sentence.link = link
sentence.after = è veramente interessante.

Do you see what just happened here? Length comparison based on groups just became less interesting, both averages and maximum/minimum differences.

Anyhow, here’s a good image (graph based on global difference in percentage) that I’d like to call “Why using English as a reference for designing UI may not be a great idea”.

Open link in a new tab/window to see the full image

Why not use Gaia directly?

This sounds like a good idea: we have a real en-US repository, and we don’t have concatenations. But there are some disadvantages as well:

• Most locales already did at least two rounds of QA, so a lot of strings have already been (heavily) shortened to fit in the UI. So data could be less useful and interesting.
• Several locales are incomplete on gaia-l10n. For this very reason I excluded all locales with less than 1000 strings translated.

Here’s the same table for Gaia. And, again, a similar graph based on global difference in percentage.

Open link in a new tab/window to see the full image

Fun facts:

• We know that en-GB is 0.16% longer than en-US, at least on Gaia.
• A simple word as “OK” (2 characters) can become as long as “Kulungile” (9) in Xhosa, or “Ceart ma-thà” (12) in Scottish Gaelic.

This week-end took place in Mozilla’s Paris office the Summit 2013 planning assembly: a gathering of about 65 people from all around the world and representing all areas of the Mozilla project, with both paid staff and volunteers, aimed to plan and shape the next Summit, that will take place the first week-end of October in Bruxelles, Toronto and Santa Clara.

TL;DR: it’s been a great assembly. If we manage to accomplish at the next Summit half of the things we’ve discussed in this week-end, it will have been the best Mozilla event ever.

The aim of the assembly was not to define a schedule for the event and fix everything but to talk about which are the important topics that animate the Mozilla project these days, start to discuss them and shape them in a way so that we can come out with a good format for the Summit to address them and try to give and propose solutions for them. To do this, all the planning committee has taken interviews to fellow Mozillians in the last month to have a wider view of which is the temperature of the project in these days and act as a representative for the comments expressed.

I went to Paris without a clear idea of what we would have accomplished there, but I’m impressed with the result we had.
First of all, this assembly was facilitated by people of unconference.net, who proposed a peculiar way to proceed with it. I was a bit skeptic with the method proposed, but it turned out that some of their methods are really great (like unpanel) and we will definitely adopt them for the next Summit, while some others still look like rubbish (I may still be proved wrong).
The second important fact is that we talked little about technology and a lot about Mozilla, its community, its communication (internal and external) and the interactions between its components and people. On one hand, as Gandalf pointed out, this is a sign that we trust implicitly our technology and the fact that it will be discussed at the Summit, because this is a big portion of what Mozilla is about. On the other hand, it’s a sign that there’s a general awareness, not only among community members but also (finally) among employees and paid staff and board of directors that we have communication problems between the different parts of the projects and especially between paid staff and volunteers, and the time is now mature to address and try to solve them. What I’m talking about is not only communication to get things done but also communication related to the decision-making process.

So, it will be interesting to experiment discussions around different time-zones and locations. I will probably post more about the assembly and the planning for the Summit in the next days, when ideas and thoughts will have settled down a bit and I’ll have had the time to read all the ideas and documentation we produced during this two days. What I felt important to communicate immediately is the fact that the next Summit will be a wonderful occasion to talk not only about our technologies but also about who we are, what we want to do and where we want to go. It will be an occasion for the community to teach and mentor the newest community members and more importantly all the new employees to let them understand and feel the power and importance of our community and it will be, in general, an occasion to have our voice finally be heard and taken into consideration not only in the tasks at hand, but in building the new policies and guidelines that will drive all the project in the future.

I’m sure we’ll try, in the next months, to provide some initial information and documentation about what have been discussed and decided so far so that you can arrive at the Summit prepared to give your contribution to the conversation, so that we can take the most out of the Summit and make it really matter in our future.

As I said at the beginning: if we manage to discuss and propose solutions to half of the problems and concerns raised during this two days, we will have had the best Mozilla event ever; one that will have strengthened and made our project more mature.