[jboss-l10n] JBoss Portal and Ant-Gettext (was JBoss Tools L10n - help wanted!)

[Sean Flanigan]

On 05/19/2009 06:39 PM, Thomas Heute wrote:
> Do you have any idea about the support of right-to-left languages with
> PO tools ?

I must confess that I haven't really worked with RTL or bi-directional 
text.  I can't think of anything specific to RTL, just the usual things: 
don't concatenate strings; use MessageFormat to substitute variables 
into complete sentences.  I'm not sure how KBabel/Lokalizer handle RTL 
rendering though.

Did you have anything specific in mind?


> Also for Portal we are interested to store the translation in XML
> (Supported instead of plain property files, the reason for that is that
> customers want to edit the translation directly and XML is easier to
> handle in some cases (no need for native2ascii encoding)

That's XML-based ResourceBundles, I presume?  They do solve the encoding 
problem, but they still suffer from another big problem with .properties 
files - they don't maintain a connection between the English text and 
the translated text.  See 
https://fedorahosted.org/tennera/wiki/PropertiesFiles (ignoring the 
first few bullet points).

I'm really beginning to think that the best option is to use, at 
runtime, a "real" translation format like XLIFF or Gettext PO files. 
They both have their flaws, but they do at least allow UTF-8 and they 
maintain the link between a translation and its original English.

The Spacewalk project apparently uses a cut-down version of XLIFF, 
loaded at runtime:

http://git.fedorahosted.org/git/?p=spacewalk.git;a=blob;f=java/code/src/com/redhat/rhn/common/localization/XmlResourceBundle.java;h=dad255849e1bb39a54d9ad69d78490cfd04edb59;hb=HEAD

The problem is (a) it's not full XLIFF, so if someone (or a tool) uses 
an unsupported feature of XLIFF, things might break at runtime, and (b) 
good translator tools for XLIFF don't seem to exist in open source yet. 
  Gettext still has the advantage in tools.  I think Spacewalk's XLIFF 
gets converted to Gettext for the Red Hat translators, and back again 
when they're finished.

It's a bit radical, but I think PO files as a runtime format could 
actually make sense.  If and when XLIFF support (both Java libraries and 
translation tools) catches up, switching over shouldn't be hard.

Anything that eliminates a file conversion has to make life easier. 
(Especially if you want to avoid SVN conflicts.)

On the other hand, XLIFF is probably an easier sell (politically) in a 
Java environment.  It may not save a file conversion (yet), but it's 
still better than ResourceBundles.

> Just so you know ;)

Thanks for the warning!

Unless I've convinced you to switch to PO/JGettext, I'd better make sure 
Ant-Gettext can handle XML resource bundles.  I can't say I ever thought 
someone would use them! ;-)


Sean.

> Sean Flanigan wrote:
>> Thomas Heute wrote:
>>> Sean,
>>>
>>>   From what i've gathered:
>>> 	- PO is good for editing
>> Yep, compared to .properties, at least.  With PO, the translator can see
>> the original English and the translation in the same file, which (a)
>> should make translation easier and (b) makes it possible to guarantee
>> that translations are updated when the English is.
>>
>> With .properties, if someone edits the English messages.properties file
>> and changes a string, they may invalidate the messages_ja.properties,
>> messages_de.properties, etc etc.  You can only tell this has happened by
>> browsing the SVN change history.  Java will happily keep using the
>> outdated translations.
>>
>> And PO has translation editors like KBabel, Lokalize, POEdit, an emacs
>> mode.  They're pretty primitive compared to commercial translation tools
>> (so I'm told), but they're a lot better than the .properties editors
>> I've seen.
>>
>>> 	- .properties are good for programming (faster from what i heard and
>>> less verbose)
>> Well, I tend to go with .properties because the Java runtimes
>> ResourceBundle and Eclipse's NLS both expect .properties, and I didn't
>> want to mess with these APIs.  I doubt that .properties are appreciably
>> faster on a modern machine, but they probably do save little disk space.
>>   In Gettext/C land, they compile PO to MO for runtime, for much the same
>> reason.
>>
>> Now that I think about it, there could be some good arguments to try PO
>> as a runtime format.  I might try an experiment one day...
>>
>> You shouldn't give me ideas.
>>
>>> I see that you've build a Properties to PO tool, but how do you go back
>>> to properties ? I'm not sure if you've built a tool to migrate or to be
>>> used back and forth (bi-directional).
>> They're both there, just shielded by a cloak of bad documentation.  It's
>> definitely meant to be two-way.  Quoting myself from the Tennera wiki [1]:
>>
>> # Gettext2PropTask (gettext2prop) is like 'po2prop' or 'msgcat
>> --properties-output', except that it generates a directory tree of
>> .properties from each PO file (one per locale).
>>
>> That's a terrible description, because you have to know msgcat or
>> po2prop, but yes, it converts a PO file back into .properties.
>>
>>
>>> I'm definitely interested by such tools for JBoss Portal.
>> Well these tools were specifically written for JBoss [2], so let me know
>> what you need!  I'm keen to see them put to good use.
>>
>> Regards
>>
>> Sean.
>>
>> [1] https://fedorahosted.org/tennera/wiki/WikiStart
>> [2] prop2pot, prop2po and po2prop were written to be cut-down, pure Java
>> equivalents for the gettext tools, so that they could be integrated into
>> a JBoss build.  prop2gettext and gettext2prop came along when it was
>> obvious that mapping hundreds of .properties to hundreds of PO files was
>> terrible for translators.
>>
>>
>>> Thomas
>>>
>>>
>>> Sean Flanigan wrote:
>>>> G'day,
>>>>
>>>> I've been working on some translation infrastructure for JBoss Tools.
>>>>
>>>> I'm pleased to report that we've already got some Japanese translations
>>>> for JBoss Tools, courtesy of a contribution by Shinobu Nogami.  Any help
>>>> with reviewing, correcting, or completing these translations would be
>>>> appreciated!  Translations for other languages would be great too.
>>>>
>>>> My build scripts for jbosstools-l10n use Tennera Ant-Gettext to convert
>>>> between .properties files (ResourceBundles) in the Java source code and
>>>> Gettext PO/POT files.  A bit like Translate Toolkit's prop2po but with
>>>> one important difference: an entire directory tree of properties files
>>>> can be reduced to a single POT file (not a tree of POT files).
>>>>
>>>> I hope this will make life easier for translators, since dealing with
>>>> hundreds of .properties files is hard enough in a good developer's IDE.
>>>>   From what I've seen, the open source translation editors just aren't
>>>> that good at dealing with lots of small files efficiently.  I'm not
>>>> familiar with any commercial translation tools, but I don't think many
>>>> of them would edit .properties files directly - they would convert to
>>>> another file format, or a database).
>>>>
>>>> PO files have their flaws, but they're still a big improvement over
>>>> .properties for L10n purposes, and they are what the Red Hat L10n team
>>>> mostly uses in current translation workflows (as I understand it).
>>>>
>>>> Anyway, the POT templates for JBoss Tools 3.1, and also the Japanese PO
>>>> files, are currently living in a Mercurial repository here:
>>>> http://bitbucket.org/seanf/jbosstools-l10n/ (Obviously, that's not meant
>>>> to be the permanent location!  And I'm not married to Mercurial, though
>>>> having a DVCS would be nice.)
>>>>
>>>> Ultimately, I suggest these particular translations should probably live
>>>> somewhere like this:
>>>> http://anonsvn.jboss.org/repos/jboss-l10n/jbosstools/3.1 [or maybe
>>>> .../trunk]
>>>>
>>>> But where to host them is the subject of my other post...
>>>>
>>>> Sean.
>


-- 
Sean Flanigan

Senior Software Engineer
Engineering - Internationalisation
Red Hat