Wednesday, July 08, 2009

Rails: Internationalization

I'm reading through Agile Web Development with Rails. I'm surprised at how naive the I18n support in Rails appears to be.

It seems to get number formatting right, but it gets a lot of other things wrong. For instance, as far as I can tell reading the book, it assumes all languages have the same pluralization forms as English. It forces you to create YML-based "constants" for all your localization strings. Hence, your templates contain things like I18n.t('checkout.submit') instead of, say, _("Submit your order"). I have no clue what happens if you're missing a constant in your YML. I also don't think it understands how to degrade gracefully. For instance, if I ask for some weird sub-dialect, I bet it doesn't know how to fall back to the main language. Furthermore, the YML files painfully duplicate strings for translation. I haven't seen anything for dealing with the Accept-language header, but that could just be the book's fault.

I may be misunderstanding some things, but I think the gettext library used in C, Python, etc. is much smarter and more convenient.

Please feel free to tell me how wrong I am. However, if you do, you must also explain how the heck Russian speakers can possibly cope with their crazy pluralization rules in real time ;)

13 comments:

Marco Monteiro said...
This comment has been removed by a blog administrator.
Shannon -jj Behrens said...

I forgot to mention that automatic extraction of strings to be translated is much nicer.

Shannon -jj Behrens said...
This comment has been removed by the author.
Shannon -jj Behrens said...

Since there's no gettext command to automatically extract strings that have been flagged for translation, I'm guessing there's also no msgmerge command to merge an old, translated translation catalog with a new, untranslated translation catalog.

Mark M said...

I would say one thing regarding the duplication of strings.

In large applications and in some languages there may well be more than one translation for a single English string.

Imagine that Profile can mean many different things. Profile of a face (noun), Profile the code (verb), online user Profile, etc.
Some languages may well have different translations.

I guess this generally bites only for large complex software with lots of short strings (think Software rather than Web pages)

Mark

Shannon -jj Behrens said...

Mark, you're right. In gettext, I have do things like _("Software:File") instead of just _("File") when the context is unclear.

I think this isn't all that uncommon. For instance, in the Rails book, they use "Campo" for "Field". I do believe "Campo" means a grassy field, not a field in a form. Certainly, if you have an app that had a form field where you could configure the name of the grassy field you wanted to have a picnic in, you'd have to provide a bit more context.

However, my point remains that once you've translated "Software:Field" in one place, there's no need to translate it in 50 other places ;)

Heikki Toivonen said...

While localizing Chandler we also hit the problem of needing to translate the same English word to two (or more) different strings in other languages. We solved it by painstakingly choosing different words for the original English version so that translations could be made. However, the English Chandler also ships with English translation, so we could have chosen gibberish for these instances and rely on localizers reading the localization comments and translating the meaning, not the gibberish string.

Gettext is nicer most of the time, but you need to work around the limitations when the same text needs to be translated to multiple different texts depending on context. The problem is that unless you the developer do all the translations, you will not know when you are going to hit this issue. Therefore I regrettably favor the symbolic string values that must be fetched for every language. I believe Java and Mozilla use this way.

Shannon -jj Behrens said...

Most of the time, you should use a real string. In times where it is ambiguous, you can fall back to a made up string. Problem solved ;)

Heikki Toivonen said...

The problem is, like I tried to explain, that you won't know in advance what strings are ambiguous. If you use strings, the best you can hope for is to have localizers complain to you. Often localizations happens after a release, so you end up with a situation where you need to do a new release because language A needs disambiguation of string "foo" and this can be repeated for each and every localization attempt. This can be costly if you have help, documentation, screenshots, videos etc. that would need to be updated.

Shannon -jj Behrens said...

Ok, well argued Heikki.

However, do we really need to duplicate the word "Save" umpteen times in the translation file so that the translators can translate it differently every time? Will they actually take the time to understand the differing context of the word "Save" every single time?

The truth of the matter is that translators work with translation memories anyway. Every time they see the word "Save", they're just going to click "Ok" in their translation tool to copy the same translation they used last time. They won't even think about context.

I think to do it right, translators *must* be given access to the application and the ability to really see the string in context. They should probably start with a simple, naive translation (which they would do anyway) and then work with the app and the coders to change strings where the context warrants it.

Doing L10n right is hard, and it gets exponentially harder as you try to do a better job. There are no silver bullets.

Heikki Toivonen said...

The way I do localization, and I suspect most others do as well, is use the app in the original language first to get a feel for it. Then try to translate as many strings as possible without referring to the UI all the time. Then attack the rest by locating the string in the UI to see the context. Finally after everything is translated, use the translation and try to spot and fix awkward/incorrect translations.

As for the "Save" example, to prevent the scenario I presented, you should put in a new mnemonic for every use of "Save". For the most part the translators will just automatically pick the same translation. But if there is a case where they need to differentiate, they can. It is not possible if the developer has used the same mnemonic more than once.

Some translation tools provide ways to let developers provide comments for localizers. I think these are necessary for some concepts, even if you let the localizers run the application with and without their localizations. I think a decent example from Chandler is "item". There is no way you could translate that out of context, and even if you see the app and where it is used, you may still have trouble getting the exact meaning. The localization comment can describe the concept in detail so that accurate translation becomes possible. Alternatively you could provide documentation and/or glossary to help translators.

But yeah, localization is hard. We should all switch to Finnish ;)

Mr Darius Roberts said...

but isn't gettext is supported by I18n? Afaik it's a wrapper for Simple backend translation library, which is what you're really critiquing... i think?

Shannon -jj Behrens said...

I'm sorry, I don't quite understand what you're saying. Can you explain?