I once read a tweet that I haven’t been able to attribute back to its original source (Paul Irish has sleuthed and found the original source to be a Greg Brockman tweet), but said something along the lines of
Web programming is the science of coming up with increasingly complicated ways of concatenating strings.
I think that there’s a good reason for this.
As we turn to the next chapter of the web and start building more web applications, rather than just web documents, we begin to need to mix language with data. This presents an extremely difficult challenge if you only have to consider a single language. Often times we have to support many.
While the data density in language increases on our sites and apps, we must consider the user who has to read those sentences. The goal is to offer interesting data to the user without just generating a table and without sounding like Borat writes the copy. You can spend as much time as you’d like on the interaction and visual design, but if your app doesn’t have a good flow, you’re leaving the most important cards on the table: the content.
Let’s Focus on English for a Second
Number of Results: 15
in Number of Categories: 3
This is the type of language that we’ve all come to expect out of the web. It gets the point across, but I’d argue that it’s not very personal or natural. That may not matter in this particular example, but consider building anything social. Identity is becoming a huge part of the next wave of apps. If you want to have any chance at personalization or identity, then the following isn’t going to cut it:
Alex added 5 friend(s) to their group on March 19, 2012
This is in stark contrast to some of the more popular social networks.
Not only does this correctly address the correct pluralization for ‘people’, it also takes into consideration the offset from the total count of the 3 people that are listed explicitly. This attention to detail doesn’t stop at decent pluralization, but also gender.
Remember the old days when Facebook used to refer to everyone as a ‘their’? That was weird. Don’t fall into that trap. When it comes to good user experience, you can’t ignore the text. If you can go through your app and find any sentence that you’d write differently if it wasn’t generated by a computer, you are reducing the effectiveness of that text and degrading the overall experience.
Iteration
Assuming you’re sold on the idea of a proper language treatment of your app, let’s try to solve the problem using some of the examples we’ve already seen.
There are X result(s)
In JavaScript, the naïve solution would be to do something like the following:
if ( X === 1 ) {
return "There is one result";
}
else {
return "There are " + X + " results";
}
This results in a proper english sentence in all cases. That’s great, but let’s get a touch more complex. Now we want to support another language. The pluralization rules for English are different than the rules for French. If we want to support French, our solution starts looking more like this:
if ( lang === "en" ) {
if ( X === 1 ) {
return "There is one result";
}
else {
return "There are " + X + " results";
}
}
else if ( lang === "fr" ) {
if ( X > 1 ) {
return "Le, there are " + X + " results";
}
else {
return "Le, there is " + X + " result";
}
}
This quickly becomes an unscalable solution. You might be thinking “Well, Alex, I never plan on internationalizing my app, so the rest of this article doesn’t apply to me.” - I’d encourage you to consider the original example as a more pleasant English-only sentence:
There are 8 results in 2 categories.
Even when ignoring locale, we still have some combinatoric debts to pay. Let’s check out the code for handling this naïvely:
if ( ResCount !== 1 && CatCount !== 1 ) {
return "There are " + ResCount + " results in " + CatCount + " categories.";
}
else if ( ResCount === 1 && CatCount !== 1 ) {
return "There is one result in " + CatCount + " categories.";
}
else if ( ResCount !== 1 && CatCount === 1 ) {
return "There are " + ResCount + " results in one category.";
}
else if ( ResCount === 1 && CatCount === 1 ) {
return "There is one result in one category.";
}
This is pretty painstaking even without the chore of translating it into other languages. You cannot split up the halves of the sentences safely (especially if you want multiple languages), because the two halves may not match. You can imagine that when gender is added to this sentence, things explode even further.
Alex searched for an image. He found 5 results in one category.
Now we have to multiply that logic times the number of gender choices. Most specs have 3 gender choices: “male”, “female”, and “other”. “Other” specifically is treated “as if you cannot determine the gender of someone who is far away from you.” In the case of this simple sentence, we’d need twelve copies of the sentence - one for each combination of gender and plural form of the nouns.
Gettext (jed)
I recently released a library, Jed, for using Gettext style messages in JavaScript. Gettext is a GNU spec that’s been around for ages. I had been exposed to a little bit of Gettext from my python days, and considered it to be the most popular solution to many of these problems. The main feature of Gettext is that you can decouple the messages from the plural-forms of a given language.
var pluralForms = {
"en" : function ( x ) {
if ( x === 1 ) {
return 0;
}
return 1;
},
"fr" : function ( x ) {
if ( x > 1 ) {
return 1;
}
return 0;
}
};
We now get back an index of sorts. “Gettext” refers to the lookup/loading/encoding mechanism more than anything, but with these plural forms we could do a lookup for the correct string, stored as data.
var translations = {
"en" : {
"somekey" : [ "There is one result.", "There are %s results." ]
}
"fr" : {
"somekey" : [ "Le, there is %s result.", "Le, there are %s results." ]
}
}
var lang = "en";
var msg = sprintf( translations[ lang ][ "somekey" ][ pluralForms[ lang ]( X ) ], X );
In this case we’re using sprintf style replacement after we do the key lookup. That tends to be the most common way to do substitution with Gettext. Now we have a solution that relies on data instead of one-off code blocks for each message. Also, if your sprintf supports positional variables, you can now solve the problem that different languages order sentences differently than english does.
Soon after I released Jed it was shared on es-discuss. Immediately Norbert Lindenberg stepped up to tell me that I was making a mistake by choosing Gettext. How right he was. The best example of my oversight is actually one that we’ve already seen:
There are 8 results in 2 categories
How would we represent this in Gettext’s PO format? The plural-form functions can only take a single number to decide the plural form. This sentence would need to be split up again, which won’t work across languages and often won’t work well even in English. Gender can be added in by utlizing Gettext’s context feature, but it only goes one level deep. What if I needed an actual context AND a gender selection?
Norbert was kind enough to point me in the direction of the ICU MessageFormat spec. I could quickly see that some smart people had thought about this a lot longer than I had. Using Jed for Gettext can still be nice if you already have invested in using Gettext in other parts of your stack, but I’d generally suggest against it in favor of MessageFormat.
MessageFormat is actually just a few specs pasted together, but they look similar. They may seem vaguely familiar to those who have ever used Java’s ChoiceFormat utility. They are different in a few ways, but the important part is that they more or less solve multiple plurals and gender specificity without as much of the combinatorics game.
The MessageFormat spec contains PluralFormat and SelectFormat in the most common cases. Using the syntax in PluralFormat we can address multiple plurals in the same sentence. All the pluralization data is standardized and pulled from CLDR and not needed as user input. There are keywords that come back as a result for any given input number: “zero”, “one”, “two”, “few”, “many”, “other”. All languages can be roughly mapped to these keywords, and it is the basis for some of the keywords in the message.
I won’t go in to much detail about the syntax, as that’s not the point of this post.
There {ResCount, plural,
one {is one result}
other {are # results}
} in {CatCount, plural,
one {one category}
other {# categories}
}.
Using PluralFormat we were able to decouple the pluralization of each of the nouns.
Gender is usually handled via SelectFormat which works much like a switch statement (except default becomes other).
{GENDER, select,
male {He}
female {She}
other {They}
} just found {ResCount, plural,
one {one result}
other {# results}
} in {CatCount, plural,
one {one category}
other {# categories}
}.
At the top we are able to determine the gender to use and then reuse most of our code from above to have a multiple plural and gender-specific sentence in as few characters as possible. Exceedingly complex sentences can often still require nesting and combinatorics, but for the majority of cases, you can avoid repeating any logic.
There are also complex ‘plugins’ that can be used. The offset option will help you generate sentences like in the google plus example earlier.
Technically, I don’t think NumberFormat is part of MessageFormat - but it is usually necessary to pull in. NumberFormat allows you to internationalize things that we haven’t even covered yet. Ever consider that other countries use , characters where the US uses . and visa versa? Number format is how you handle numbers, percentages, and currencies across languages.
1234.5 //Decimal number
$1234.50 //U.S. currency
1.234,57€ //German currency
123457% //Percent
Shortly after releasing Jed, I released messageformat.js. It’s a much less sexy name, but perhaps I’ll fix that soon. Google also has an implementation for people using the Google Closure library: http://code.google.com/p/closure-library/source/browse/trunk/closure/goog/i18n/messageformat.js.
While both are likely to be sufficiently fast, I did implement messageformat.js as a compile to JS language. This means that at build time, you can ‘precompile’ your messages and ditch the majority of the library. This creates some great opportunities to be able to include MessageFormat style strings directly inside of your precompilable templates and have it all compile to a series of string concats. The readme on the project page should be quite helpful to learn the syntax as well as the integration and api.
My co-worker Oliver Wong was able to do a quick port of the Google Closure NumberFormat.js to not need Google Closure (under Apache 2).
If anyone was wondering how I handle dates in my JS apps, I figured I’d add it here. Moment.js, by Tim Wood is a library that I lean on a lot. There are plenty of additional internationalization libraries that I could start adding (for collation and rtl, etc), but for right now, the built in localization, and friendly ago syntax of moment.js make for a great user experience around dates.
All Together Now
I plan on updating Jed to actually contain this group of tools rather than the Gettext ones. I think these tools better suit the needs of modern applications. I will certainly keep the old Jed (Gettext) code around for those that require that format. It’s not terribly difficult to integrate with these tools separately now, though.
Conclusion
Language is important. It can get complex. A lot of incredibly bright people have been looking into the constraints of same-language message generation, as well as multi-langual message generation (the spec writers, not necessarily the library creators). These tools and/or ideas should be the starting point of any application that desires to have a good UX.
It doesn’t matter how many drop-shadows or rounded-corners you have, the user shouldn’t have to decode your words. The words are often the most valuable experience.