[keycloak-dev] i18n/l10n: English text in java code

Stian Thorgersen sthorger at redhat.com
Fri Oct 9 07:57:54 EDT 2015 

Translation team that is

On 9 October 2015 at 13:57, Stian Thorgersen <sthorger at redhat.com> wrote:

> By the way I'm pretty sure our translation theme will take our English
> bundle and translate it to a language. They won't need to start/stop the
> server and open the admin console to check that they've translated all
> keys. They should have a tool/script for that.
>
> On 9 October 2015 at 13:56, Stian Thorgersen <sthorger at redhat.com> wrote:
>
>>
>>
>> On 9 October 2015 at 13:46, Stan Silvert <ssilvert at redhat.com> wrote:
>>
>>> On 10/9/2015 7:32 AM, Stian Thorgersen wrote:
>>>
>>> #1 You're not going to catch all missing keys this way - as I said
>>> there's 2 types, custom defined as well which could be missing from default
>>> bundle
>>>
>>> It catches it at load time.  As it loads each bundle, it checks against
>>> the previously loaded bundle.  That will indeed catch all missing keys in
>>> any bundle you try to test.
>>>
>>> I don't know exactly what you mean by "custom defined".  Somehow a
>>> third-party bundle must be merged with our default bundle.  Unless I
>>> completely misunderstand, the code I wrote will still work.
>>>
>>
>> New keys can be defined by using keys in client descriptions, client
>> names, etc, etc.. These won't be in our message bundle, but would be in the
>> bundle in a customers theme. This is why message bundles inherit from the
>> parent theme. Custom providers and themes can further introduce new keys.
>>
>>
>>>
>>> #2 Rendering the whole bundle useless just because you're missing one
>>> key is just daft
>>>
>>> It is the correct thing to do.  A missing key is like a null pointer.
>>> It deserves a RuntimeException.
>>>
>>
>> No it's not the correct thing to do.
>>
>>
>>>
>>> #3 There will quite likely be separate teams that do translations to
>>> those that do development, which means stack traces and log output is not
>>> the solution
>>>
>>> I don't see what that has to do with anything.  You start with a set of
>>> bundles containing all the correct keys.  Then you translate each bundle.
>>> If you accidentally delete a key then you want to know that right away.
>>> But we should indeed ask the translation team what they want to see.
>>>
>>
>> Sure, go ahead and ask them if they want to look for RuntimeExceptions in
>> the server log.
>>
>>
>>>
>>> #4 Doing a check each time you pull a message bundle to compare with the
>>> base bundle is probably not that expensive, but still pretty daft thing to
>>> do
>>>
>>> You only load each bundle once.  So the check only happens the first
>>> time you request the bundle.
>>>
>>> #5 A proper util that's used to translate bundles is much better - we
>>> can implement a page in the admin console that allows you to validate a
>>> bundle and print out all missing bundles. This is something that would be
>>> more developer friendly and also would be usable by non-developers (aka
>>> people with other language skills than Java)
>>>
>>> We should ask the translation team what they want to see and how they do
>>> their work.  I'm sure that they don't expect a tool to be built into the
>>> product.  None of our other products have that.
>>>
>>
>> I don't just care about our translation team, they will translate the
>> built in keys, but they won't translate the ones introduced by users.
>>
>>
>>
>>>
>>>
>>>
>>> On 9 October 2015 at 13:24, Stan Silvert <ssilvert at redhat.com> wrote:
>>>
>>>> On 10/9/2015 6:21 AM, Marko Strukelj wrote:
>>>>
>>>> And we can always log the missing key situation into server log - that
>>>> should be enough for developers to notice it, and fix it.
>>>>
>>>> This is basically what happens with the code I wrote for the fix:
>>>> https://github.com/keycloak/keycloak/pull/1690
>>>>
>>>> You get an error in the console and then a stack trace on the server.
>>>> The stack trace tells you exactly which key is missing.  But the console
>>>> doesn't crash or anything like that.  You just switch back to your original
>>>> language and everything works fine.
>>>>
>>>>
>>>> On Fri, Oct 9, 2015 at 8:09 AM, Stian Thorgersen <sthorger at redhat.com>
>>>> wrote:
>>>>
>>>>> There's two places where keys can be missing:
>>>>>
>>>>> * In a translation - this can be an honest mistake, or the translation
>>>>> wasn't updated when KC was updated
>>>>> * Custom keys added - for example when keys are used for display names
>>>>> of clients, roles, etc..
>>>>>
>>>>> Manually having to go through all sorts of pages to look for missing
>>>>> keys is very error prone and time consuming, so will not be the best option
>>>>> for developers. In both cases above the correct way to do this would be to
>>>>> have a way to verify a message bundle. We need a tool that can quickly
>>>>> identify if there are missing keys and we could expose that through the
>>>>> admin console. We currently have a student looking at providing a UI for
>>>>> defining locales and she is also going to look at adding some way of
>>>>> identifying if a locale is missing keys and also to easily list only
>>>>> missing keys.
>>>>>
>>>>> For end users as I've said they will have no clue what ???key??? is,
>>>>> and even worse if we throw an exception/error just because a missing key
>>>>> we'll actually break the whole console just because of a missing key. It's
>>>>> a much better option to look for the key in another translation and display
>>>>> that. Chances are they will be able to interpret one or two English words.
>>>>> Certainly higher chance of that then them being able to interpret ???key???.
>>>>>
>>>>>
>>>>> On 9 October 2015 at 07:51, Thomas Raehalme <
>>>>> thomas.raehalme at aitiofinland.com> wrote:
>>>>>
>>>>>> How about returning something noticeable like ???key??? for example?
>>>>>> On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> That's not putting it to rest at all! Throwing a RuntimeException
>>>>>>> and rendering the whole admin console useless just because there's a
>>>>>>> missing key is a horrible idea.
>>>>>>>
>>>>>>> On 8 October 2015 at 20:33, Stan Silvert <ssilvert at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> What if English is the bundle that has a missing key?
>>>>>>>>
>>>>>>>> Let's just put this to rest and solve it once and for all.  The
>>>>>>>> simplest solution I can think of is to just compare keys when a new bundle
>>>>>>>> is loaded.  If any bundle has a missing key or it has key not found in the
>>>>>>>> previous loaded bundle, we throw a RuntimeException.  I can submit a patch
>>>>>>>> for that in just a few minutes.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/8/2015 1:28 PM, Stian Thorgersen wrote:
>>>>>>>>
>>>>>>>> I'm not sure I'm buying into the argument that displaying the key
>>>>>>>> is better for developers. Having English suddenly pop-up in a German
>>>>>>>> translation is just as obvious as a key. Besides as Stan points out you
>>>>>>>> catch missing keys by comparing missing keys between English and German.
>>>>>>>>
>>>>>>>> However, if there is a mistake in a translation then a user may
>>>>>>>> quite likely be able to interpret English text, while a user will not be
>>>>>>>> able to interpret a key. So if a key is missing in a translation (which is
>>>>>>>> obviously a "bug") it's better to display English than to display the key.
>>>>>>>>
>>>>>>>> On 8 October 2015 at 14:13, Stan Silvert <ssilvert at redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On 10/8/2015 12:48 AM, Thomas Raehalme wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger at redhat.com>
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> > With regards to internationalization I have two questions:
>>>>>>>>> >
>>>>>>>>> > * Should we fallback to English messages if a key is missing in
>>>>>>>>> a translation? Alternative is to show key, but that's not going to help
>>>>>>>>> anyone
>>>>>>>>>
>>>>>>>>> A missing key is a bug and showing the message in the default
>>>>>>>>> locale may hide the problem.
>>>>>>>>>
>>>>>>>>> Even though showing the key does not help the end user it helps
>>>>>>>>> the developer and identifies the problem. For this reason I think showing
>>>>>>>>> the key would be a good idea.
>>>>>>>>>
>>>>>>>>> For our bundles, we could catch missing keys at build time.
>>>>>>>>>
>>>>>>>>> Failing that, I agree that displaying the key is better than
>>>>>>>>> falling back to English.  This is especially true right now while we
>>>>>>>>> haven't completed the task of converting everything.  If we fall back to
>>>>>>>>> English we won't know if the problem is a missing key or if the text just
>>>>>>>>> hasn't been converted yet.
>>>>>>>>>
>>>>>>>>> > * Should we change message bundles to UTF-8? Or is ISO 8859-1
>>>>>>>>> going to work for all languages?
>>>>>>>>>
>>>>>>>>> Depends what those all languages are :-)
>>>>>>>>>
>>>>>>>>> I think UTF-8 is the best choice as it will handle practically any
>>>>>>>>> character.
>>>>>>>>>
>>>>>>>>> But if you're referring to Java resource bundles the encoding for
>>>>>>>>> .properties is ISO-8859-1 but there are means to handle any UTF-8 character.
>>>>>>>>>
>>>>>>>>> Yes, an UTF-8 character can be encoded in ISO-8859-1.  Java
>>>>>>>>> provides a native2ascii tool for converting entire files.  The resource
>>>>>>>>> bundle tools in most IDE's do this for you automatically.  So you just edit
>>>>>>>>> as UTF-8 and it saves the bundle as ISO-8859-1.
>>>>>>>>>
>>>>>>>>> We can read our bundles as UTF-8 if we want to do that.  I'd
>>>>>>>>> rather not, because I'm not sure what we might run into down the road with
>>>>>>>>> Java assuming resource bundles are always ISO-8859-1.
>>>>>>>>>
>>>>>>>>> But I'd like to get the perspective of people who have handled
>>>>>>>>> resource bundles in languages that are not fully supported by ISO-8859-1.
>>>>>>>>> Is it too much of a pain to do a conversion or do the tools make the
>>>>>>>>> process seamless?
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>> >
>>>>>>>>> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert at redhat.com>
>>>>>>>>> wrote:
>>>>>>>>> >>
>>>>>>>>> >> Marko brought this to my attention yesterday.  For some things,
>>>>>>>>> we
>>>>>>>>> >> dynamically create UI.  In this case, the java code contains
>>>>>>>>> the English
>>>>>>>>> >> text and it needs to be localized.  Luckily, the solution was
>>>>>>>>> pretty
>>>>>>>>> >> straightforward.  We just replace the English text with a key
>>>>>>>>> into the
>>>>>>>>> >> message bundle.  The html template that displays this text
>>>>>>>>> already pulls
>>>>>>>>> >> from an Angular scope so we just leave that alone and pass it
>>>>>>>>> through
>>>>>>>>> >> the |translate filter.  You do need to also add the
>>>>>>>>> double-colon.
>>>>>>>>> >>
>>>>>>>>> >> One nice side effect is that if the key is not found in the
>>>>>>>>> bundle then
>>>>>>>>> >> the output of the translate filter is the unchanged text.  This
>>>>>>>>> means
>>>>>>>>> >> that any code which has not converted to using bundle keys will
>>>>>>>>> still
>>>>>>>>> >> work as expected.   And, any third-party providers can just
>>>>>>>>> pass in
>>>>>>>>> >> plain text if they don't care about l10n.  If they ever do care
>>>>>>>>> about
>>>>>>>>> >> l10n we will just need to provide a means for them to add
>>>>>>>>> key/value
>>>>>>>>> >> pairs to the resource bundles.
>>>>>>>>> >>
>>>>>>>>> >> Here is an example for anyone who needs to localize English text
>>>>>>>>> >> embedded in java:
>>>>>>>>> >>
>>>>>>>>> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd8833c37be8549
>>>>>>>>> >>
>>>>>>>>> >> Stan
>>>>>>>>> >> _______________________________________________
>>>>>>>>> >> keycloak-dev mailing list
>>>>>>>>> >> keycloak-dev at lists.jboss.org
>>>>>>>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > _______________________________________________
>>>>>>>>> > keycloak-dev mailing list
>>>>>>>>> > keycloak-dev at lists.jboss.org
>>>>>>>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> keycloak-dev mailing list
>>>>> keycloak-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> keycloak-dev mailing list
>>>> keycloak-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151009/6d08be79/attachment-0001.html

More information about the keycloak-dev mailing list