Re: [jbosstools-dev] Rethinking internationalisation approach (LONG)

Hi everyone, While waiting for the trunk code freeze to be over (so that I can get back to externalising strings in the source code),

I've been thinking about the approach. (The plan was to have Gettext .po files kept in a separate part of the source tree, with Ant tasks to generate .properties files at build time, to create fragment plugins and an update site, etc etc.) (I'm sorry this is so long. You might want to skip down to THE ACTUAL POINT...) I've realised there are a number of issues: 1. It turns out there is at least one independent effort to translate JBoss Tools, an effort which has naturally been using .properties files. Converting these files into .po files is easy on a one-off basis, but how should we handle/merge future updates to these translations? 2. Fragment plugins seem to be a rarely-used feature of Eclipse, and thus a bit of an edge case. There does finally seem to be a way of getting Eclipse 3.4 to work with fragments (by modifying P2 metadata after generation), but the whole area is still a bit... iffy. 3. Even though Red Hat translators use Gettext format internally, they do work on other projects which use .properties files. They just wait until the relevant project's (eg Mozilla's) string freeze, convert .properties into .po, do their thing, and convert .po into .properties and submit to the project CVS/SVN. This means they can't do continous translation (they have to wait for a string freeze), but they have to work to a schedule anyway... 4. I have created Ant tasks/scripts which can generate .properties files and fragment plugins, but smoothly integrating this with (a) Eclipse development environments and (b) Hudson builds, will still be a lot of work; work which I can't do without help. (NB: if your translations only exist in plugin jars, it's quite painful to get a self-hosted Eclipse to pick them up[1].) 5. My Ant scripts interpret the directory structure of the source code to work out the owning plugin for each file. This is kind of ugly, but it seems to be working okay, because the JBT source base is laid out pretty consistently. But if this ever changes, langpacks will probably break. This happens to the Babel project whenever it processes a plugin with a different directory structure. No doubt this will be fixed, but the point is that it is a fragile approach. 6. Having separate translation fragment plugins implies (a) modifying the installer to install these plugins, (b) having a separate update site for translations, (c) and leads to problem with upgrades: a user might upgrade, say, Hibernate Tools - how do we make sure they update the translations too? If I recall correctly, these were the reasons for this approach: 1. We don't want dozens of languages bloating the JBoss Tools install; we want to keep the size down. But YAGNI. Initially, there will only be 5 or 6 languages anyway. At present, the English .properties files (for all of JBoss Tools) take up 355KB compressed. Even if you multiply that by 100, you only have 35MB. Even if we start adding per-language images (best avoided), it won't add that much. I think we can ignore this problem for a long while. 2. Restricting check-in access to the main source tree. I think that was originally my suggestion; I'm not sure how important that is really, but I'm sure there are options. I think most open source projects just trust their translators with committer access. 3. Not wanting to hold up GA waiting for localisation. I'm not sure about the best way to deal with this. Is it acceptable to say that JBT/JBDS x.0 is English only, and x.0.1 is localised? Or perhaps you could distinguish the multilingual version x.0 from the English-only x.0? But I now think that generating separate langpacks and corresponding update-site is more overhead than it's worth. THE ACTUAL POINT ---------------- In short, I have been bludgeoned by reality into changing my mind. I would now recommend: 1. *Keep translations in .properties form*, right next to the English files, in the plugin directories *in SVN*. The translations will automatically be included in the plugin whether at development/debug time, or when the JBDS installer is generated, or via the main update site. 2. *Do away with the i18n build*. This means we don't have to integrate i18n with the Hudson build, or with Eclipse, and means we eliminate its dependencies on (currently) external Maven artifacts like Tennera Ant-Gettext and JGettext. I still don't like .properties files, but fighting them is not worth it! If we go with the flow, it should be possible to release internationalised versions of JBT/JBDS a lot sooner, since we can leave out a lot of extra release engineering work. 3. Shortly before (or perhaps immediately after) GA, enact a *string freeze* on the release branch. Note that this *does not apply* to JBT 3.0.0, since there is still more string externalisation to be done after GA. (But it would make sense to have a string freeze in the 3.0 branch after the externalisation work is finished.) Once the string freeze starts, developers shouldn't change *any* of the properties files. The Localisation team grabs all the properties files, converts to their preferred translation format (currently Gettext/PO), and eventually converts the translated text into properties files again, checking them in. This will *clobber* the properties files in SVN. That's why the string freeze is needed. I realise fitting in a string freeze will be an adjustment, but it's a pretty standard way of making room for localisation[2] [3]. Eventually, it would be good to point a web translation tool, such as Flies[4], directly at the source tree, so that no-one will have to look at .properties files *or* PO files, and I'm hopeful that Flies workflow support will eventually remove the need for string freezes. Regards Sean. [1] http://www.jboss.org/community/docs/DOC-13256 See "Installing langpacks the hard way (launching JBT from within Eclipse)" [2] http://fedoraproject.org/wiki/Releases/10/Schedule#Key_Milestones [3] http://wiki.eclipse.org/Galileo#Should_Do [4] https://fedorahosted.org/flies/ ------------------------------------------------------------------------ _______________________________________________ jbosstools-dev mailing list jbosstools-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jbosstools-dev