[keycloak-dev] i18n/l10n: English text in java code

Stian Thorgersen sthorger at redhat.com
Fri Oct 9 07:57:42 EDT 2015 

By the way I'm pretty sure our translation theme will take our English
bundle and translate it to a language. They won't need to start/stop the
server and open the admin console to check that they've translated all
keys. They should have a tool/script for that.

On 9 October 2015 at 13:56, Stian Thorgersen <sthorger at redhat.com> wrote:

>
>
> On 9 October 2015 at 13:46, Stan Silvert <ssilvert at redhat.com> wrote:
>
>> On 10/9/2015 7:32 AM, Stian Thorgersen wrote:
>>
>> #1 You're not going to catch all missing keys this way - as I said
>> there's 2 types, custom defined as well which could be missing from default
>> bundle
>>
>> It catches it at load time.  As it loads each bundle, it checks against
>> the previously loaded bundle.  That will indeed catch all missing keys in
>> any bundle you try to test.
>>
>> I don't know exactly what you mean by "custom defined".  Somehow a
>> third-party bundle must be merged with our default bundle.  Unless I
>> completely misunderstand, the code I wrote will still work.
>>
>
> New keys can be defined by using keys in client descriptions, client
> names, etc, etc.. These won't be in our message bundle, but would be in the
> bundle in a customers theme. This is why message bundles inherit from the
> parent theme. Custom providers and themes can further introduce new keys.
>
>
>>
>> #2 Rendering the whole bundle useless just because you're missing one key
>> is just daft
>>
>> It is the correct thing to do.  A missing key is like a null pointer.  It
>> deserves a RuntimeException.
>>
>
> No it's not the correct thing to do.
>
>
>>
>> #3 There will quite likely be separate teams that do translations to
>> those that do development, which means stack traces and log output is not
>> the solution
>>
>> I don't see what that has to do with anything.  You start with a set of
>> bundles containing all the correct keys.  Then you translate each bundle.
>> If you accidentally delete a key then you want to know that right away.
>> But we should indeed ask the translation team what they want to see.
>>
>
> Sure, go ahead and ask them if they want to look for RuntimeExceptions in
> the server log.
>
>
>>
>> #4 Doing a check each time you pull a message bundle to compare with the
>> base bundle is probably not that expensive, but still pretty daft thing to
>> do
>>
>> You only load each bundle once.  So the check only happens the first time
>> you request the bundle.
>>
>> #5 A proper util that's used to translate bundles is much better - we can
>> implement a page in the admin console that allows you to validate a bundle
>> and print out all missing bundles. This is something that would be more
>> developer friendly and also would be usable by non-developers (aka people
>> with other language skills than Java)
>>
>> We should ask the translation team what they want to see and how they do
>> their work.  I'm sure that they don't expect a tool to be built into the
>> product.  None of our other products have that.
>>
>
> I don't just care about our translation team, they will translate the
> built in keys, but they won't translate the ones introduced by users.
>
>
>
>>
>>
>>
>> On 9 October 2015 at 13:24, Stan Silvert <ssilvert at redhat.com> wrote:
>>
>>> On 10/9/2015 6:21 AM, Marko Strukelj wrote:
>>>
>>> And we can always log the missing key situation into server log - that
>>> should be enough for developers to notice it, and fix it.
>>>
>>> This is basically what happens with the code I wrote for the fix:
>>> https://github.com/keycloak/keycloak/pull/1690
>>>
>>> You get an error in the console and then a stack trace on the server.
>>> The stack trace tells you exactly which key is missing.  But the console
>>> doesn't crash or anything like that.  You just switch back to your original
>>> language and everything works fine.
>>>
>>>
>>> On Fri, Oct 9, 2015 at 8:09 AM, Stian Thorgersen <sthorger at redhat.com>
>>> wrote:
>>>
>>>> There's two places where keys can be missing:
>>>>
>>>> * In a translation - this can be an honest mistake, or the translation
>>>> wasn't updated when KC was updated
>>>> * Custom keys added - for example when keys are used for display names
>>>> of clients, roles, etc..
>>>>
>>>> Manually having to go through all sorts of pages to look for missing
>>>> keys is very error prone and time consuming, so will not be the best option
>>>> for developers. In both cases above the correct way to do this would be to
>>>> have a way to verify a message bundle. We need a tool that can quickly
>>>> identify if there are missing keys and we could expose that through the
>>>> admin console. We currently have a student looking at providing a UI for
>>>> defining locales and she is also going to look at adding some way of
>>>> identifying if a locale is missing keys and also to easily list only
>>>> missing keys.
>>>>
>>>> For end users as I've said they will have no clue what ???key??? is,
>>>> and even worse if we throw an exception/error just because a missing key
>>>> we'll actually break the whole console just because of a missing key. It's
>>>> a much better option to look for the key in another translation and display
>>>> that. Chances are they will be able to interpret one or two English words.
>>>> Certainly higher chance of that then them being able to interpret ???key???.
>>>>
>>>>
>>>> On 9 October 2015 at 07:51, Thomas Raehalme <
>>>> thomas.raehalme at aitiofinland.com> wrote:
>>>>
>>>>> How about returning something noticeable like ???key??? for example?
>>>>> On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> That's not putting it to rest at all! Throwing a RuntimeException and
>>>>>> rendering the whole admin console useless just because there's a missing
>>>>>> key is a horrible idea.
>>>>>>
>>>>>> On 8 October 2015 at 20:33, Stan Silvert <ssilvert at redhat.com> wrote:
>>>>>>
>>>>>>> What if English is the bundle that has a missing key?
>>>>>>>
>>>>>>> Let's just put this to rest and solve it once and for all.  The
>>>>>>> simplest solution I can think of is to just compare keys when a new bundle
>>>>>>> is loaded.  If any bundle has a missing key or it has key not found in the
>>>>>>> previous loaded bundle, we throw a RuntimeException.  I can submit a patch
>>>>>>> for that in just a few minutes.
>>>>>>>
>>>>>>>
>>>>>>> On 10/8/2015 1:28 PM, Stian Thorgersen wrote:
>>>>>>>
>>>>>>> I'm not sure I'm buying into the argument that displaying the key is
>>>>>>> better for developers. Having English suddenly pop-up in a German
>>>>>>> translation is just as obvious as a key. Besides as Stan points out you
>>>>>>> catch missing keys by comparing missing keys between English and German.
>>>>>>>
>>>>>>> However, if there is a mistake in a translation then a user may
>>>>>>> quite likely be able to interpret English text, while a user will not be
>>>>>>> able to interpret a key. So if a key is missing in a translation (which is
>>>>>>> obviously a "bug") it's better to display English than to display the key.
>>>>>>>
>>>>>>> On 8 October 2015 at 14:13, Stan Silvert <ssilvert at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> On 10/8/2015 12:48 AM, Thomas Raehalme wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger at redhat.com>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > With regards to internationalization I have two questions:
>>>>>>>> >
>>>>>>>> > * Should we fallback to English messages if a key is missing in a
>>>>>>>> translation? Alternative is to show key, but that's not going to help anyone
>>>>>>>>
>>>>>>>> A missing key is a bug and showing the message in the default
>>>>>>>> locale may hide the problem.
>>>>>>>>
>>>>>>>> Even though showing the key does not help the end user it helps the
>>>>>>>> developer and identifies the problem. For this reason I think showing the
>>>>>>>> key would be a good idea.
>>>>>>>>
>>>>>>>> For our bundles, we could catch missing keys at build time.
>>>>>>>>
>>>>>>>> Failing that, I agree that displaying the key is better than
>>>>>>>> falling back to English.  This is especially true right now while we
>>>>>>>> haven't completed the task of converting everything.  If we fall back to
>>>>>>>> English we won't know if the problem is a missing key or if the text just
>>>>>>>> hasn't been converted yet.
>>>>>>>>
>>>>>>>> > * Should we change message bundles to UTF-8? Or is ISO 8859-1
>>>>>>>> going to work for all languages?
>>>>>>>>
>>>>>>>> Depends what those all languages are :-)
>>>>>>>>
>>>>>>>> I think UTF-8 is the best choice as it will handle practically any
>>>>>>>> character.
>>>>>>>>
>>>>>>>> But if you're referring to Java resource bundles the encoding for
>>>>>>>> .properties is ISO-8859-1 but there are means to handle any UTF-8 character.
>>>>>>>>
>>>>>>>> Yes, an UTF-8 character can be encoded in ISO-8859-1.  Java
>>>>>>>> provides a native2ascii tool for converting entire files.  The resource
>>>>>>>> bundle tools in most IDE's do this for you automatically.  So you just edit
>>>>>>>> as UTF-8 and it saves the bundle as ISO-8859-1.
>>>>>>>>
>>>>>>>> We can read our bundles as UTF-8 if we want to do that.  I'd rather
>>>>>>>> not, because I'm not sure what we might run into down the road with Java
>>>>>>>> assuming resource bundles are always ISO-8859-1.
>>>>>>>>
>>>>>>>> But I'd like to get the perspective of people who have handled
>>>>>>>> resource bundles in languages that are not fully supported by ISO-8859-1.
>>>>>>>> Is it too much of a pain to do a conversion or do the tools make the
>>>>>>>> process seamless?
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>> >
>>>>>>>> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert at redhat.com>
>>>>>>>> wrote:
>>>>>>>> >>
>>>>>>>> >> Marko brought this to my attention yesterday.  For some things,
>>>>>>>> we
>>>>>>>> >> dynamically create UI.  In this case, the java code contains the
>>>>>>>> English
>>>>>>>> >> text and it needs to be localized.  Luckily, the solution was
>>>>>>>> pretty
>>>>>>>> >> straightforward.  We just replace the English text with a key
>>>>>>>> into the
>>>>>>>> >> message bundle.  The html template that displays this text
>>>>>>>> already pulls
>>>>>>>> >> from an Angular scope so we just leave that alone and pass it
>>>>>>>> through
>>>>>>>> >> the |translate filter.  You do need to also add the double-colon.
>>>>>>>> >>
>>>>>>>> >> One nice side effect is that if the key is not found in the
>>>>>>>> bundle then
>>>>>>>> >> the output of the translate filter is the unchanged text.  This
>>>>>>>> means
>>>>>>>> >> that any code which has not converted to using bundle keys will
>>>>>>>> still
>>>>>>>> >> work as expected.   And, any third-party providers can just pass
>>>>>>>> in
>>>>>>>> >> plain text if they don't care about l10n.  If they ever do care
>>>>>>>> about
>>>>>>>> >> l10n we will just need to provide a means for them to add
>>>>>>>> key/value
>>>>>>>> >> pairs to the resource bundles.
>>>>>>>> >>
>>>>>>>> >> Here is an example for anyone who needs to localize English text
>>>>>>>> >> embedded in java:
>>>>>>>> >>
>>>>>>>> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd8833c37be8549
>>>>>>>> >>
>>>>>>>> >> Stan
>>>>>>>> >> _______________________________________________
>>>>>>>> >> keycloak-dev mailing list
>>>>>>>> >> keycloak-dev at lists.jboss.org
>>>>>>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > _______________________________________________
>>>>>>>> > keycloak-dev mailing list
>>>>>>>> > keycloak-dev at lists.jboss.org
>>>>>>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>> _______________________________________________
>>>> keycloak-dev mailing list
>>>> keycloak-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151009/1d0a1062/attachment-0001.html

More information about the keycloak-dev mailing list