Re: [keycloak-dev] i18n/l10n: English text in java code

On 9 October 2015 at 13:46, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: > On 10/9/2015 7:32 AM, Stian Thorgersen wrote: > > #1 You're not going to catch all missing keys this way - as I said > there's 2 types, custom defined as well which could be missing from default > bundle > > It catches it at load time. As it loads each bundle, it checks against > the previously loaded bundle. That will indeed catch all missing keys in > any bundle you try to test. > > I don't know exactly what you mean by "custom defined". Somehow a > third-party bundle must be merged with our default bundle. Unless I > completely misunderstand, the code I wrote will still work. > New keys can be defined by using keys in client descriptions, client names, etc, etc.. These won't be in our message bundle, but would be in the bundle in a customers theme. This is why message bundles inherit from the parent theme. Custom providers and themes can further introduce new keys. > > #2 Rendering the whole bundle useless just because you're missing one key > is just daft > > It is the correct thing to do. A missing key is like a null pointer. It > deserves a RuntimeException. > No it's not the correct thing to do. > > #3 There will quite likely be separate teams that do translations to > those that do development, which means stack traces and log output is not > the solution > > I don't see what that has to do with anything. You start with a set of > bundles containing all the correct keys. Then you translate each bundle. > If you accidentally delete a key then you want to know that right away. > But we should indeed ask the translation team what they want to see. > Sure, go ahead and ask them if they want to look for RuntimeExceptions in the server log. > > #4 Doing a check each time you pull a message bundle to compare with the > base bundle is probably not that expensive, but still pretty daft thing to > do > > You only load each bundle once. So the check only happens the first time > you request the bundle. > > #5 A proper util that's used to translate bundles is much better - we can > implement a page in the admin console that allows you to validate a bundle > and print out all missing bundles. This is something that would be more > developer friendly and also would be usable by non-developers (aka people > with other language skills than Java) > > We should ask the translation team what they want to see and how they do > their work. I'm sure that they don't expect a tool to be built into the > product. None of our other products have that. > I don't just care about our translation team, they will translate the built in keys, but they won't translate the ones introduced by users. > > > > On 9 October 2015 at 13:24, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: > >> On 10/9/2015 6:21 AM, Marko Strukelj wrote: >> >> And we can always log the missing key situation into server log - that >> should be enough for developers to notice it, and fix it. >> >> This is basically what happens with the code I wrote for the fix: >> https://github.com/keycloak/keycloak/pull/1690 >> >> You get an error in the console and then a stack trace on the server. >> The stack trace tells you exactly which key is missing. But the console >> doesn't crash or anything like that. You just switch back to your original >> language and everything works fine. >> >> >> On Fri, Oct 9, 2015 at 8:09 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; >> wrote: >> >>> There's two places where keys can be missing: >>> >>> * In a translation - this can be an honest mistake, or the translation >>> wasn't updated when KC was updated >>> * Custom keys added - for example when keys are used for display names >>> of clients, roles, etc.. >>> >>> Manually having to go through all sorts of pages to look for missing >>> keys is very error prone and time consuming, so will not be the best option >>> for developers. In both cases above the correct way to do this would be to >>> have a way to verify a message bundle. We need a tool that can quickly >>> identify if there are missing keys and we could expose that through the >>> admin console. We currently have a student looking at providing a UI for >>> defining locales and she is also going to look at adding some way of >>> identifying if a locale is missing keys and also to easily list only >>> missing keys. >>> >>> For end users as I've said they will have no clue what ???key??? is, >>> and even worse if we throw an exception/error just because a missing key >>> we'll actually break the whole console just because of a missing key. It's >>> a much better option to look for the key in another translation and display >>> that. Chances are they will be able to interpret one or two English words. >>> Certainly higher chance of that then them being able to interpret ???key???. >>> >>> >>> On 9 October 2015 at 07:51, Thomas Raehalme < >>> thomas.raehalme(a)aitiofinland.com&gt; wrote: >>> >>>> How about returning something noticeable like ???key??? for example? >>>> On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; >>>> wrote: >>>> >>>>> That's not putting it to rest at all! Throwing a RuntimeException and >>>>> rendering the whole admin console useless just because there's a missing >>>>> key is a horrible idea. >>>>> >>>>> On 8 October 2015 at 20:33, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >>>>> >>>>>> What if English is the bundle that has a missing key? >>>>>> >>>>>> Let's just put this to rest and solve it once and for all. The >>>>>> simplest solution I can think of is to just compare keys when a new bundle >>>>>> is loaded. If any bundle has a missing key or it has key not found in the >>>>>> previous loaded bundle, we throw a RuntimeException. I can submit a patch >>>>>> for that in just a few minutes. >>>>>> >>>>>> >>>>>> On 10/8/2015 1:28 PM, Stian Thorgersen wrote: >>>>>> >>>>>> I'm not sure I'm buying into the argument that displaying the key is >>>>>> better for developers. Having English suddenly pop-up in a German >>>>>> translation is just as obvious as a key. Besides as Stan points out you >>>>>> catch missing keys by comparing missing keys between English and German. >>>>>> >>>>>> However, if there is a mistake in a translation then a user may >>>>>> quite likely be able to interpret English text, while a user will not be >>>>>> able to interpret a key. So if a key is missing in a translation (which is >>>>>> obviously a "bug") it's better to display English than to display the key. >>>>>> >>>>>> On 8 October 2015 at 14:13, Stan Silvert <ssilvert(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> On 10/8/2015 12:48 AM, Thomas Raehalme wrote: >>>>>>> >>>>>>> >>>>>>> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> > >>>>>>> > With regards to internationalization I have two questions: >>>>>>> > >>>>>>> > * Should we fallback to English messages if a key is missing in a >>>>>>> translation? Alternative is to show key, but that's not going to help anyone >>>>>>> >>>>>>> A missing key is a bug and showing the message in the default >>>>>>> locale may hide the problem. >>>>>>> >>>>>>> Even though showing the key does not help the end user it helps the >>>>>>> developer and identifies the problem. For this reason I think showing the >>>>>>> key would be a good idea. >>>>>>> >>>>>>> For our bundles, we could catch missing keys at build time. >>>>>>> >>>>>>> Failing that, I agree that displaying the key is better than >>>>>>> falling back to English. This is especially true right now while we >>>>>>> haven't completed the task of converting everything. If we fall back to >>>>>>> English we won't know if the problem is a missing key or if the text just >>>>>>> hasn't been converted yet. >>>>>>> >>>>>>> > * Should we change message bundles to UTF-8? Or is ISO 8859-1 >>>>>>> going to work for all languages? >>>>>>> >>>>>>> Depends what those all languages are :-) >>>>>>> >>>>>>> I think UTF-8 is the best choice as it will handle practically any >>>>>>> character. >>>>>>> >>>>>>> But if you're referring to Java resource bundles the encoding for >>>>>>> .properties is ISO-8859-1 but there are means to handle any UTF-8 character. >>>>>>> >>>>>>> Yes, an UTF-8 character can be encoded in ISO-8859-1. Java >>>>>>> provides a native2ascii tool for converting entire files. The resource >>>>>>> bundle tools in most IDE's do this for you automatically. So you just edit >>>>>>> as UTF-8 and it saves the bundle as ISO-8859-1. >>>>>>> >>>>>>> We can read our bundles as UTF-8 if we want to do that. I'd rather >>>>>>> not, because I'm not sure what we might run into down the road with Java >>>>>>> assuming resource bundles are always ISO-8859-1. >>>>>>> >>>>>>> But I'd like to get the perspective of people who have handled >>>>>>> resource bundles in languages that are not fully supported by ISO-8859-1. >>>>>>> Is it too much of a pain to do a conversion or do the tools make the >>>>>>> process seamless? >>>>>>> >>>>>>> Best regards, >>>>>>> Thomas >>>>>>> >>>>>>> > >>>>>>> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> >> >>>>>>> >> Marko brought this to my attention yesterday. For some things, >>>>>>> we >>>>>>> >> dynamically create UI. In this case, the java code contains the >>>>>>> English >>>>>>> >> text and it needs to be localized. Luckily, the solution was >>>>>>> pretty >>>>>>> >> straightforward. We just replace the English text with a key >>>>>>> into the >>>>>>> >> message bundle. The html template that displays this text >>>>>>> already pulls >>>>>>> >> from an Angular scope so we just leave that alone and pass it >>>>>>> through >>>>>>> >> the |translate filter. You do need to also add the double-colon. >>>>>>> >> >>>>>>> >> One nice side effect is that if the key is not found in the >>>>>>> bundle then >>>>>>> >> the output of the translate filter is the unchanged text. This >>>>>>> means >>>>>>> >> that any code which has not converted to using bundle keys will >>>>>>> still >>>>>>> >> work as expected. And, any third-party providers can just pass >>>>>>> in >>>>>>> >> plain text if they don't care about l10n. If they ever do care >>>>>>> about >>>>>>> >> l10n we will just need to provide a means for them to add >>>>>>> key/value >>>>>>> >> pairs to the resource bundles. >>>>>>> >> >>>>>>> >> Here is an example for anyone who needs to localize English text >>>>>>> >> embedded in java: >>>>>>> >> >>>>>>> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >>>>>>> >> >>>>>>> >> Stan >>>>>>> >> _______________________________________________ >>>>>>> >> keycloak-dev mailing list >>>>>>> >> keycloak-dev(a)lists.jboss.org >>>>>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > _______________________________________________ >>>>>>> > keycloak-dev mailing list >>>>>>> > keycloak-dev(a)lists.jboss.org >>>>>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> >> _______________________________________________ >> keycloak-dev mailing listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > >