[jbosstools-dev] Rethinking internationalisation approach (LONG)

[Sean Flanigan]

Hi everyone,

While waiting for the trunk code freeze to be over (so that I can get
back to externalising strings in the source code), I've been thinking
about the approach.  (The plan was to have Gettext .po files kept in a
separate part of the source tree, with Ant tasks to generate .properties
files at build time, to create fragment plugins and an update site, etc
etc.)

(I'm sorry this is so long.  You might want to skip down to THE ACTUAL
POINT...)

I've realised there are a number of issues:

1. It turns out there is at least one independent effort to translate
JBoss Tools, an effort which has naturally been using .properties files.
 Converting these files into .po files is easy on a one-off basis, but
how should we handle/merge future updates to these translations?

2. Fragment plugins seem to be a rarely-used feature of Eclipse, and
thus a bit of an edge case.  There does finally seem to be a way of
getting Eclipse 3.4 to work with fragments (by modifying P2 metadata
after generation), but the whole area is still a bit... iffy.

3. Even though Red Hat translators use Gettext format internally, they
do work on other projects which use .properties files.  They just wait
until the relevant project's (eg Mozilla's) string freeze, convert
.properties into .po, do their thing, and convert .po into .properties
and submit to the project CVS/SVN.  This means they can't do continous
translation (they have to wait for a string freeze), but they have to
work to a schedule anyway...

4. I have created Ant tasks/scripts which can generate .properties files
and fragment plugins, but smoothly integrating this with (a) Eclipse
development environments and (b) Hudson builds, will still be a lot of
work; work which I can't do without help.  (NB: if your translations
only exist in plugin jars, it's quite painful to get a self-hosted
Eclipse to pick them up[1].)

5. My Ant scripts interpret the directory structure of the source code
to work out the owning plugin for each file.  This is kind of ugly, but
it seems to be working okay, because the JBT source base is laid out
pretty consistently.  But if this ever changes, langpacks will probably
break.  This happens to the Babel project whenever it processes a plugin
with a different directory structure.  No doubt this will be fixed, but
the point is that it is a fragile approach.

6. Having separate translation fragment plugins implies (a) modifying
the installer to install these plugins, (b) having a separate update
site for translations, (c) and leads to problem with upgrades: a user
might upgrade, say, Hibernate Tools - how do we make sure they update
the translations too?


If I recall correctly, these were the reasons for this approach:

1. We don't want dozens of languages bloating the JBoss Tools install;
we want to keep the size down.  But YAGNI.  Initially, there will only
be 5 or 6 languages anyway.  At present, the English .properties files
(for all of JBoss Tools) take up 355KB compressed.  Even if you multiply
that by 100, you only have 35MB.  Even if we start adding per-language
images (best avoided), it won't add that much.  I think we can ignore
this problem for a long while.

2. Restricting check-in access to the main source tree.  I think that
was originally my suggestion; I'm not sure how important that is really,
but I'm sure there are options.  I think most open source projects just
trust their translators with committer access.

3. Not wanting to hold up GA waiting for localisation.  I'm not sure
about the best way to deal with this.  Is it acceptable to say that
JBT/JBDS x.0 is English only, and x.0.1 is localised?  Or perhaps you
could distinguish the multilingual version x.0 from the English-only
x.0?  But I now think that generating separate langpacks and
corresponding update-site is more overhead than it's worth.


THE ACTUAL POINT
----------------

In short, I have been bludgeoned by reality into changing my mind.  I
would now recommend:

1. *Keep translations in .properties form*, right next to the English
files, in the plugin directories *in SVN*.  The translations will
automatically be included in the plugin whether at development/debug
time, or when the JBDS installer is generated, or via the main update site.

2. *Do away with the i18n build*.  This means we don't have to integrate
i18n with the Hudson build, or with Eclipse, and means we eliminate its
dependencies on (currently) external Maven artifacts like Tennera
Ant-Gettext and JGettext.

I still don't like .properties files, but fighting them is not worth it!
 If we go with the flow, it should be possible to release
internationalised versions of JBT/JBDS a lot sooner, since we can leave
out a lot of extra release engineering work.

3. Shortly before (or perhaps immediately after) GA, enact a *string
freeze* on the release branch.  Note that this *does not apply* to JBT
3.0.0, since there is still more string externalisation to be done after
GA.  (But it would make sense to have a string freeze in the 3.0 branch
after the externalisation work is finished.)

Once the string freeze starts, developers shouldn't change *any* of the
properties files.  The Localisation team grabs all the properties files,
converts to their preferred translation format (currently Gettext/PO),
and eventually converts the translated text into properties files again,
checking them in.  This will *clobber* the properties files in SVN.
That's why the string freeze is needed.  I realise fitting in a string
freeze will be an adjustment, but it's a pretty standard way of making
room for localisation[2] [3].

Eventually, it would be good to point a web translation tool, such as
Flies[4], directly at the source tree, so that no-one will have to look
at .properties files *or* PO files, and I'm hopeful that Flies workflow
support will eventually remove the need for string freezes.

Regards

Sean.

[1] http://www.jboss.org/community/docs/DOC-13256 See "Installing
langpacks the hard way (launching JBT from within Eclipse)"
[2] http://fedoraproject.org/wiki/Releases/10/Schedule#Key_Milestones
[3] http://wiki.eclipse.org/Galileo#Should_Do
[4] https://fedorahosted.org/flies/
-- 
Sean Flanigan

Senior Software Engineer
Engineering - Internationalisation
Red Hat

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 551 bytes
Desc: OpenPGP digital signature
Url : http://lists.jboss.org/pipermail/jbosstools-dev/attachments/20090220/526820c2/attachment.bin