[keycloak-dev] i18n/l10n: English text in java code

Fri Oct 9 07:56:26 EDT 2015

On 9 October 2015 at 13:46, Stan Silvert <ssilvert at redhat.com> wrote:

> On 10/9/2015 7:32 AM, Stian Thorgersen wrote:
>
> #1 You're not going to catch all missing keys this way - as I said there's
> 2 types, custom defined as well which could be missing from default bundle
>
> It catches it at load time.  As it loads each bundle, it checks against
> the previously loaded bundle.  That will indeed catch all missing keys in
> any bundle you try to test.
>
> I don't know exactly what you mean by "custom defined".  Somehow a
> third-party bundle must be merged with our default bundle.  Unless I
> completely misunderstand, the code I wrote will still work.
>

New keys can be defined by using keys in client descriptions, client names,
etc, etc.. These won't be in our message bundle, but would be in the bundle
in a customers theme. This is why message bundles inherit from the parent
theme. Custom providers and themes can further introduce new keys.

>
> #2 Rendering the whole bundle useless just because you're missing one key
> is just daft
>
> It is the correct thing to do.  A missing key is like a null pointer.  It
> deserves a RuntimeException.
>

No it's not the correct thing to do.

>
> #3 There will quite likely be separate teams that do translations to those
> that do development, which means stack traces and log output is not the
> solution
>
> I don't see what that has to do with anything.  You start with a set of
> bundles containing all the correct keys.  Then you translate each bundle.
> If you accidentally delete a key then you want to know that right away.
> But we should indeed ask the translation team what they want to see.
>

Sure, go ahead and ask them if they want to look for RuntimeExceptions in
the server log.

>
> #4 Doing a check each time you pull a message bundle to compare with the
> base bundle is probably not that expensive, but still pretty daft thing to
> do
>
> You only load each bundle once.  So the check only happens the first time
> you request the bundle.
>
> #5 A proper util that's used to translate bundles is much better - we can
> implement a page in the admin console that allows you to validate a bundle
> and print out all missing bundles. This is something that would be more
> developer friendly and also would be usable by non-developers (aka people
> with other language skills than Java)
>
> We should ask the translation team what they want to see and how they do
> their work.  I'm sure that they don't expect a tool to be built into the
> product.  None of our other products have that.
>

I don't just care about our translation team, they will translate the built
in keys, but they won't translate the ones introduced by users.

>
>
>
> On 9 October 2015 at 13:24, Stan Silvert <ssilvert at redhat.com> wrote:
>
>> On 10/9/2015 6:21 AM, Marko Strukelj wrote:
>>
>> And we can always log the missing key situation into server log - that
>> should be enough for developers to notice it, and fix it.
>>
>> This is basically what happens with the code I wrote for the fix:
>> https://github.com/keycloak/keycloak/pull/1690
>>
>> You get an error in the console and then a stack trace on the server.
>> The stack trace tells you exactly which key is missing.  But the console
>> doesn't crash or anything like that.  You just switch back to your original
>> language and everything works fine.
>>
>>
>> On Fri, Oct 9, 2015 at 8:09 AM, Stian Thorgersen <sthorger at redhat.com>
>> wrote:
>>
>>> There's two places where keys can be missing:
>>>
>>> * In a translation - this can be an honest mistake, or the translation
>>> wasn't updated when KC was updated
>>> * Custom keys added - for example when keys are used for display names
>>> of clients, roles, etc..
>>>
>>> Manually having to go through all sorts of pages to look for missing
>>> keys is very error prone and time consuming, so will not be the best option
>>> for developers. In both cases above the correct way to do this would be to
>>> have a way to verify a message bundle. We need a tool that can quickly
>>> identify if there are missing keys and we could expose that through the
>>> admin console. We currently have a student looking at providing a UI for
>>> defining locales and she is also going to look at adding some way of
>>> identifying if a locale is missing keys and also to easily list only
>>> missing keys.
>>>
>>> For end users as I've said they will have no clue what ???key??? is, and
>>> even worse if we throw an exception/error just because a missing key we'll
>>> actually break the whole console just because of a missing key. It's a much
>>> better option to look for the key in another translation and display that.
>>> Chances are they will be able to interpret one or two English words.
>>> Certainly higher chance of that then them being able to interpret ???key???.
>>>
>>>
>>> On 9 October 2015 at 07:51, Thomas Raehalme <
>>> thomas.raehalme at aitiofinland.com> wrote:
>>>
>>>> How about returning something noticeable like ???key??? for example?
>>>> On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger at redhat.com> wrote:
>>>>
>>>>> That's not putting it to rest at all! Throwing a RuntimeException and
>>>>> rendering the whole admin console useless just because there's a missing
>>>>> key is a horrible idea.
>>>>>
>>>>> On 8 October 2015 at 20:33, Stan Silvert <ssilvert at redhat.com> wrote:
>>>>>
>>>>>> What if English is the bundle that has a missing key?
>>>>>>
>>>>>> Let's just put this to rest and solve it once and for all.  The
>>>>>> simplest solution I can think of is to just compare keys when a new bundle
>>>>>> is loaded.  If any bundle has a missing key or it has key not found in the
>>>>>> previous loaded bundle, we throw a RuntimeException.  I can submit a patch
>>>>>> for that in just a few minutes.
>>>>>>
>>>>>>
>>>>>> On 10/8/2015 1:28 PM, Stian Thorgersen wrote:
>>>>>>
>>>>>> I'm not sure I'm buying into the argument that displaying the key is
>>>>>> better for developers. Having English suddenly pop-up in a German
>>>>>> translation is just as obvious as a key. Besides as Stan points out you
>>>>>> catch missing keys by comparing missing keys between English and German.
>>>>>>
>>>>>> However, if there is a mistake in a translation then a user may quite
>>>>>> likely be able to interpret English text, while a user will not be able to
>>>>>> interpret a key. So if a key is missing in a translation (which is
>>>>>> obviously a "bug") it's better to display English than to display the key.
>>>>>>
>>>>>> On 8 October 2015 at 14:13, Stan Silvert <ssilvert at redhat.com> wrote:
>>>>>>
>>>>>>> On 10/8/2015 12:48 AM, Thomas Raehalme wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger at redhat.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > With regards to internationalization I have two questions:
>>>>>>> >
>>>>>>> > * Should we fallback to English messages if a key is missing in a
>>>>>>> translation? Alternative is to show key, but that's not going to help anyone
>>>>>>>
>>>>>>> A missing key is a bug and showing the message in the default locale
>>>>>>> may hide the problem.
>>>>>>>
>>>>>>> Even though showing the key does not help the end user it helps the
>>>>>>> developer and identifies the problem. For this reason I think showing the
>>>>>>> key would be a good idea.
>>>>>>>
>>>>>>> For our bundles, we could catch missing keys at build time.
>>>>>>>
>>>>>>> Failing that, I agree that displaying the key is better than falling
>>>>>>> back to English.  This is especially true right now while we haven't
>>>>>>> completed the task of converting everything.  If we fall back to English we
>>>>>>> won't know if the problem is a missing key or if the text just hasn't been
>>>>>>> converted yet.
>>>>>>>
>>>>>>> > * Should we change message bundles to UTF-8? Or is ISO 8859-1
>>>>>>> going to work for all languages?
>>>>>>>
>>>>>>> Depends what those all languages are :-)
>>>>>>>
>>>>>>> I think UTF-8 is the best choice as it will handle practically any
>>>>>>> character.
>>>>>>>
>>>>>>> But if you're referring to Java resource bundles the encoding for
>>>>>>> .properties is ISO-8859-1 but there are means to handle any UTF-8 character.
>>>>>>>
>>>>>>> Yes, an UTF-8 character can be encoded in ISO-8859-1.  Java provides
>>>>>>> a native2ascii tool for converting entire files.  The resource bundle tools
>>>>>>> in most IDE's do this for you automatically.  So you just edit as UTF-8 and
>>>>>>> it saves the bundle as ISO-8859-1.
>>>>>>>
>>>>>>> We can read our bundles as UTF-8 if we want to do that.  I'd rather
>>>>>>> not, because I'm not sure what we might run into down the road with Java
>>>>>>> assuming resource bundles are always ISO-8859-1.
>>>>>>>
>>>>>>> But I'd like to get the perspective of people who have handled
>>>>>>> resource bundles in languages that are not fully supported by ISO-8859-1.
>>>>>>> Is it too much of a pain to do a conversion or do the tools make the
>>>>>>> process seamless?
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Thomas
>>>>>>>
>>>>>>> >
>>>>>>> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert at redhat.com>
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >> Marko brought this to my attention yesterday.  For some things, we
>>>>>>> >> dynamically create UI.  In this case, the java code contains the
>>>>>>> English
>>>>>>> >> text and it needs to be localized.  Luckily, the solution was
>>>>>>> pretty
>>>>>>> >> straightforward.  We just replace the English text with a key
>>>>>>> into the
>>>>>>> >> message bundle.  The html template that displays this text
>>>>>>> already pulls
>>>>>>> >> from an Angular scope so we just leave that alone and pass it
>>>>>>> through
>>>>>>> >> the |translate filter.  You do need to also add the double-colon.
>>>>>>> >>
>>>>>>> >> One nice side effect is that if the key is not found in the
>>>>>>> bundle then
>>>>>>> >> the output of the translate filter is the unchanged text.  This
>>>>>>> means
>>>>>>> >> that any code which has not converted to using bundle keys will
>>>>>>> still
>>>>>>> >> work as expected.   And, any third-party providers can just pass
>>>>>>> in
>>>>>>> >> plain text if they don't care about l10n.  If they ever do care
>>>>>>> about
>>>>>>> >> l10n we will just need to provide a means for them to add
>>>>>>> key/value
>>>>>>> >> pairs to the resource bundles.
>>>>>>> >>
>>>>>>> >> Here is an example for anyone who needs to localize English text
>>>>>>> >> embedded in java:
>>>>>>> >>
>>>>>>> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd8833c37be8549
>>>>>>> >>
>>>>>>> >> Stan
>>>>>>> >> _______________________________________________
>>>>>>> >> keycloak-dev mailing list
>>>>>>> >> keycloak-dev at lists.jboss.org
>>>>>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > _______________________________________________
>>>>>>> > keycloak-dev mailing list
>>>>>>> > keycloak-dev at lists.jboss.org
>>>>>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>
>>
>>
>> _______________________________________________
>> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>>
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151009/efd37548/attachment-0001.html 