Marko brought this to my attention yesterday. For some things, we dynamically create UI. In this case, the java code contains the English text and it needs to be localized. Luckily, the solution was pretty straightforward. We just replace the English text with a key into the message bundle. The html template that displays this text already pulls from an Angular scope so we just leave that alone and pass it through the |translate filter. You do need to also add the double-colon. One nice side effect is that if the key is not found in the bundle then the output of the translate filter is the unchanged text. This means that any code which has not converted to using bundle keys will still work as expected. And, any third-party providers can just pass in plain text if they don't care about l10n. If they ever do care about l10n we will just need to provide a means for them to add key/value pairs to the resource bundles. Here is an example for anyone who needs to localize English text embedded in java: https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... Stan _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev

On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; wrote: > > With regards to internationalization I have two questions: > > * Should we fallback to English messages if a key is missing in a translation? Alternative is to show key, but that's not going to help anyone A missing key is a bug and showing the message in the default locale may hide the problem. Even though showing the key does not help the end user it helps the developer and identifies the problem. For this reason I think showing the key would be a good idea. > * Should we change message bundles to UTF-8? Or is ISO 8859-1 going to work for all languages? Depends what those all languages are :-) I think UTF-8 is the best choice as it will handle practically any character. But if you're referring to Java resource bundles the encoding for .properties is ISO-8859-1 but there are means to handle any UTF-8 character. Best regards, Thomas > > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >> >> Marko brought this to my attention yesterday. For some things, we >> dynamically create UI. In this case, the java code contains the English >> text and it needs to be localized. Luckily, the solution was pretty >> straightforward. We just replace the English text with a key into the >> message bundle. The html template that displays this text already pulls >> from an Angular scope so we just leave that alone and pass it through >> the |translate filter. You do need to also add the double-colon. >> >> One nice side effect is that if the key is not found in the bundle then >> the output of the translate filter is the unchanged text. This means >> that any code which has not converted to using bundle keys will still >> work as expected. And, any third-party providers can just pass in >> plain text if they don't care about l10n. If they ever do care about >> l10n we will just need to provide a means for them to add key/value >> pairs to the resource bundles. >> >> Here is an example for anyone who needs to localize English text >> embedded in java: >> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >> >> Stan >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev

On Thu, Oct 8, 2015 at 7:54 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > Default message bundles are Java properties. For Java developers it may > be expected that they are encoded in ISO-8859-1, but for other people > they probably expect UTF-8. Java property files are nice as the format is > simple (key=value is much nicer and easier to write than { "key" : "value" > }). There's also support for translating property files backed into IDEs. > At the moment if you use Java properties with ResourceBundle the file encoding must be ISO-8859-1. I think Java 9 may introduce support for UTF-8 (http://openjdk.java.net/jeps/226). Best regards, Thomas > > On 8 October 2015 at 06:48, Thomas Raehalme < > thomas.raehalme(a)aitiofinland.com&gt; wrote: > >> >> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; wrote: >> > >> > With regards to internationalization I have two questions: >> > >> > * Should we fallback to English messages if a key is missing in a >> translation? Alternative is to show key, but that's not going to help anyone >> >> A missing key is a bug and showing the message in the default locale may >> hide the problem. >> >> Even though showing the key does not help the end user it helps the >> developer and identifies the problem. For this reason I think showing the >> key would be a good idea. >> >> > * Should we change message bundles to UTF-8? Or is ISO 8859-1 going to >> work for all languages? >> >> Depends what those all languages are :-) >> >> I think UTF-8 is the best choice as it will handle practically any >> character. >> >> But if you're referring to Java resource bundles the encoding for >> .properties is ISO-8859-1 but there are means to handle any UTF-8 character. >> >> Best regards, >> Thomas >> >> > >> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >> >> >> >> Marko brought this to my attention yesterday. For some things, we >> >> dynamically create UI. In this case, the java code contains the >> English >> >> text and it needs to be localized. Luckily, the solution was pretty >> >> straightforward. We just replace the English text with a key into the >> >> message bundle. The html template that displays this text already >> pulls >> >> from an Angular scope so we just leave that alone and pass it through >> >> the |translate filter. You do need to also add the double-colon. >> >> >> >> One nice side effect is that if the key is not found in the bundle >> then >> >> the output of the translate filter is the unchanged text. This means >> >> that any code which has not converted to using bundle keys will still >> >> work as expected. And, any third-party providers can just pass in >> >> plain text if they don't care about l10n. If they ever do care about >> >> l10n we will just need to provide a means for them to add key/value >> >> pairs to the resource bundles. >> >> >> >> Here is an example for anyone who needs to localize English text >> >> embedded in java: >> >> >> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >> >> >> >> Stan >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> > >> > >> > _______________________________________________ >> > keycloak-dev mailing list >> > keycloak-dev(a)lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > -- *Thomas Raehalme* *CTO, teknologiajohtaja* Mobile +358 40 545 0605 *Aitio Finland Oy* Väinönkatu 26 A 40100 JYVÄSKYLÄ, Finland Tel. +358 10 322 0040 www.aitiofinland.com <http://www.codecenter.fi> *Codecenter on nyt Aitio – me kun ei vain koodata!*

On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote: > > With regards to internationalization I have two questions: > > * Should we fallback to English messages if a key is missing in a translation? Alternative is to show key, but that's not going to help anyone A missing key is a bug and showing the message in the default locale may hide the problem. Even though showing the key does not help the end user it helps the developer and identifies the problem. For this reason I think showing the key would be a good idea.

> * Should we change message bundles to UTF-8? Or is ISO 8859-1 going to work for all languages? Depends what those all languages are :-) I think UTF-8 is the best choice as it will handle practically any character. But if you're referring to Java resource bundles the encoding for .properties is ISO-8859-1 but there are means to handle any UTF-8 character.

Best regards, Thomas > > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com <mailto:ssilvert@redhat.com>> wrote: >> >> Marko brought this to my attention yesterday. For some things, we >> dynamically create UI. In this case, the java code contains the English >> text and it needs to be localized. Luckily, the solution was pretty >> straightforward. We just replace the English text with a key into the >> message bundle. The html template that displays this text already pulls >> from an Angular scope so we just leave that alone and pass it through >> the |translate filter. You do need to also add the double-colon. >> >> One nice side effect is that if the key is not found in the bundle then >> the output of the translate filter is the unchanged text. This means >> that any code which has not converted to using bundle keys will still >> work as expected. And, any third-party providers can just pass in >> plain text if they don't care about l10n. If they ever do care about >> l10n we will just need to provide a means for them to add key/value >> pairs to the resource bundles. >> >> Here is an example for anyone who needs to localize English text >> embedded in java: >> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >> >> Stan >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/keycloak-dev

I'm not sure I'm buying into the argument that displaying the key is better for developers. Having English suddenly pop-up in a German translation is just as obvious as a key. Besides as Stan points out you catch missing keys by comparing missing keys between English and German. However, if there is a mistake in a translation then a user may quite likely be able to interpret English text, while a user will not be able to interpret a key. So if a key is missing in a translation (which is obviously a "bug") it's better to display English than to display the key. On 8 October 2015 at 14:13, Stan Silvert <ssilvert(a)redhat.com <mailto:ssilvert@redhat.com>> wrote: On 10/8/2015 12:48 AM, Thomas Raehalme wrote: > > > On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com > <mailto:sthorger@redhat.com>> wrote: > > > > With regards to internationalization I have two questions: > > > > * Should we fallback to English messages if a key is missing in > a translation? Alternative is to show key, but that's not going > to help anyone > > A missing key is a bug and showing the message in the default > locale may hide the problem. > > Even though showing the key does not help the end user it helps > the developer and identifies the problem. For this reason I think > showing the key would be a good idea. > For our bundles, we could catch missing keys at build time. Failing that, I agree that displaying the key is better than falling back to English. This is especially true right now while we haven't completed the task of converting everything. If we fall back to English we won't know if the problem is a missing key or if the text just hasn't been converted yet. > > * Should we change message bundles to UTF-8? Or is ISO 8859-1 going to work for all languages? > > Depends what those all languages are :-) > > I think UTF-8 is the best choice as it will handle practically > any character. > > But if you're referring to Java resource bundles the encoding for > .properties is ISO-8859-1 but there are means to handle any UTF-8 > character. > Yes, an UTF-8 character can be encoded in ISO-8859-1. Java provides a native2ascii tool for converting entire files. The resource bundle tools in most IDE's do this for you automatically. So you just edit as UTF-8 and it saves the bundle as ISO-8859-1. We can read our bundles as UTF-8 if we want to do that. I'd rather not, because I'm not sure what we might run into down the road with Java assuming resource bundles are always ISO-8859-1. But I'd like to get the perspective of people who have handled resource bundles in languages that are not fully supported by ISO-8859-1. Is it too much of a pain to do a conversion or do the tools make the process seamless? > Best regards, > Thomas > > > > > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com > <mailto:ssilvert@redhat.com>> wrote: > >> > >> Marko brought this to my attention yesterday. For some things, we > >> dynamically create UI. In this case, the java code contains > the English > >> text and it needs to be localized. Luckily, the solution was > pretty > >> straightforward. We just replace the English text with a key > into the > >> message bundle. The html template that displays this text > already pulls > >> from an Angular scope so we just leave that alone and pass it > through > >> the |translate filter. You do need to also add the double-colon. > >> > >> One nice side effect is that if the key is not found in the > bundle then > >> the output of the translate filter is the unchanged text. > This means > >> that any code which has not converted to using bundle keys > will still > >> work as expected. And, any third-party providers can just > pass in > >> plain text if they don't care about l10n. If they ever do > care about > >> l10n we will just need to provide a means for them to add > key/value > >> pairs to the resource bundles. > >> > >> Here is an example for anyone who needs to localize English text > >> embedded in java: > >> > https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... > >> > >> Stan > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> > > https://lists.jboss.org/mailman/listinfo/keycloak-dev >

That's not putting it to rest at all! Throwing a RuntimeException and rendering the whole admin console useless just because there's a missing key is a horrible idea. On 8 October 2015 at 20:33, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: > What if English is the bundle that has a missing key? > > Let's just put this to rest and solve it once and for all. The simplest > solution I can think of is to just compare keys when a new bundle is > loaded. If any bundle has a missing key or it has key not found in the > previous loaded bundle, we throw a RuntimeException. I can submit a patch > for that in just a few minutes. > > > On 10/8/2015 1:28 PM, Stian Thorgersen wrote: > > I'm not sure I'm buying into the argument that displaying the key is > better for developers. Having English suddenly pop-up in a German > translation is just as obvious as a key. Besides as Stan points out you > catch missing keys by comparing missing keys between English and German. > > However, if there is a mistake in a translation then a user may quite > likely be able to interpret English text, while a user will not be able to > interpret a key. So if a key is missing in a translation (which is > obviously a "bug") it's better to display English than to display the key. > > On 8 October 2015 at 14:13, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: > >> On 10/8/2015 12:48 AM, Thomas Raehalme wrote: >> >> >> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; wrote: >> > >> > With regards to internationalization I have two questions: >> > >> > * Should we fallback to English messages if a key is missing in a >> translation? Alternative is to show key, but that's not going to help anyone >> >> A missing key is a bug and showing the message in the default locale may >> hide the problem. >> >> Even though showing the key does not help the end user it helps the >> developer and identifies the problem. For this reason I think showing the >> key would be a good idea. >> >> For our bundles, we could catch missing keys at build time. >> >> Failing that, I agree that displaying the key is better than falling >> back to English. This is especially true right now while we haven't >> completed the task of converting everything. If we fall back to English we >> won't know if the problem is a missing key or if the text just hasn't been >> converted yet. >> >> > * Should we change message bundles to UTF-8? Or is ISO 8859-1 going to >> work for all languages? >> >> Depends what those all languages are :-) >> >> I think UTF-8 is the best choice as it will handle practically any >> character. >> >> But if you're referring to Java resource bundles the encoding for >> .properties is ISO-8859-1 but there are means to handle any UTF-8 character. >> >> Yes, an UTF-8 character can be encoded in ISO-8859-1. Java provides a >> native2ascii tool for converting entire files. The resource bundle tools >> in most IDE's do this for you automatically. So you just edit as UTF-8 and >> it saves the bundle as ISO-8859-1. >> >> We can read our bundles as UTF-8 if we want to do that. I'd rather not, >> because I'm not sure what we might run into down the road with Java >> assuming resource bundles are always ISO-8859-1. >> >> But I'd like to get the perspective of people who have handled resource >> bundles in languages that are not fully supported by ISO-8859-1. Is it too >> much of a pain to do a conversion or do the tools make the process >> seamless? >> >> Best regards, >> Thomas >> >> > >> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >> >> >> >> Marko brought this to my attention yesterday. For some things, we >> >> dynamically create UI. In this case, the java code contains the >> English >> >> text and it needs to be localized. Luckily, the solution was pretty >> >> straightforward. We just replace the English text with a key into the >> >> message bundle. The html template that displays this text already >> pulls >> >> from an Angular scope so we just leave that alone and pass it through >> >> the |translate filter. You do need to also add the double-colon. >> >> >> >> One nice side effect is that if the key is not found in the bundle >> then >> >> the output of the translate filter is the unchanged text. This means >> >> that any code which has not converted to using bundle keys will still >> >> work as expected. And, any third-party providers can just pass in >> >> plain text if they don't care about l10n. If they ever do care about >> >> l10n we will just need to provide a means for them to add key/value >> >> pairs to the resource bundles. >> >> >> >> Here is an example for anyone who needs to localize English text >> >> embedded in java: >> >> >> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >> >> >> >> Stan >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> > >> > >> > _______________________________________________ >> > keycloak-dev mailing list >> > keycloak-dev(a)lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > >

There's two places where keys can be missing: * In a translation - this can be an honest mistake, or the translation wasn't updated when KC was updated * Custom keys added - for example when keys are used for display names of clients, roles, etc.. Manually having to go through all sorts of pages to look for missing keys is very error prone and time consuming, so will not be the best option for developers. In both cases above the correct way to do this would be to have a way to verify a message bundle. We need a tool that can quickly identify if there are missing keys and we could expose that through the admin console. We currently have a student looking at providing a UI for defining locales and she is also going to look at adding some way of identifying if a locale is missing keys and also to easily list only missing keys. For end users as I've said they will have no clue what ???key??? is, and even worse if we throw an exception/error just because a missing key we'll actually break the whole console just because of a missing key. It's a much better option to look for the key in another translation and display that. Chances are they will be able to interpret one or two English words. Certainly higher chance of that then them being able to interpret ???key???. On 9 October 2015 at 07:51, Thomas Raehalme < thomas.raehalme(a)aitiofinland.com&gt; wrote: > How about returning something noticeable like ???key??? for example? > On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; wrote: > >> That's not putting it to rest at all! Throwing a RuntimeException and >> rendering the whole admin console useless just because there's a missing >> key is a horrible idea. >> >> On 8 October 2015 at 20:33, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >> >>> What if English is the bundle that has a missing key? >>> >>> Let's just put this to rest and solve it once and for all. The >>> simplest solution I can think of is to just compare keys when a new bundle >>> is loaded. If any bundle has a missing key or it has key not found in the >>> previous loaded bundle, we throw a RuntimeException. I can submit a patch >>> for that in just a few minutes. >>> >>> >>> On 10/8/2015 1:28 PM, Stian Thorgersen wrote: >>> >>> I'm not sure I'm buying into the argument that displaying the key is >>> better for developers. Having English suddenly pop-up in a German >>> translation is just as obvious as a key. Besides as Stan points out you >>> catch missing keys by comparing missing keys between English and German. >>> >>> However, if there is a mistake in a translation then a user may quite >>> likely be able to interpret English text, while a user will not be able to >>> interpret a key. So if a key is missing in a translation (which is >>> obviously a "bug") it's better to display English than to display the key. >>> >>> On 8 October 2015 at 14:13, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >>> >>>> On 10/8/2015 12:48 AM, Thomas Raehalme wrote: >>>> >>>> >>>> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; >>>> wrote: >>>> > >>>> > With regards to internationalization I have two questions: >>>> > >>>> > * Should we fallback to English messages if a key is missing in a >>>> translation? Alternative is to show key, but that's not going to help anyone >>>> >>>> A missing key is a bug and showing the message in the default locale >>>> may hide the problem. >>>> >>>> Even though showing the key does not help the end user it helps the >>>> developer and identifies the problem. For this reason I think showing the >>>> key would be a good idea. >>>> >>>> For our bundles, we could catch missing keys at build time. >>>> >>>> Failing that, I agree that displaying the key is better than falling >>>> back to English. This is especially true right now while we haven't >>>> completed the task of converting everything. If we fall back to English we >>>> won't know if the problem is a missing key or if the text just hasn't been >>>> converted yet. >>>> >>>> > * Should we change message bundles to UTF-8? Or is ISO 8859-1 going >>>> to work for all languages? >>>> >>>> Depends what those all languages are :-) >>>> >>>> I think UTF-8 is the best choice as it will handle practically any >>>> character. >>>> >>>> But if you're referring to Java resource bundles the encoding for >>>> .properties is ISO-8859-1 but there are means to handle any UTF-8 character. >>>> >>>> Yes, an UTF-8 character can be encoded in ISO-8859-1. Java provides a >>>> native2ascii tool for converting entire files. The resource bundle tools >>>> in most IDE's do this for you automatically. So you just edit as UTF-8 and >>>> it saves the bundle as ISO-8859-1. >>>> >>>> We can read our bundles as UTF-8 if we want to do that. I'd rather >>>> not, because I'm not sure what we might run into down the road with Java >>>> assuming resource bundles are always ISO-8859-1. >>>> >>>> But I'd like to get the perspective of people who have handled >>>> resource bundles in languages that are not fully supported by ISO-8859-1. >>>> Is it too much of a pain to do a conversion or do the tools make the >>>> process seamless? >>>> >>>> Best regards, >>>> Thomas >>>> >>>> > >>>> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com&gt; >>>> wrote: >>>> >> >>>> >> Marko brought this to my attention yesterday. For some things, we >>>> >> dynamically create UI. In this case, the java code contains the >>>> English >>>> >> text and it needs to be localized. Luckily, the solution was pretty >>>> >> straightforward. We just replace the English text with a key into >>>> the >>>> >> message bundle. The html template that displays this text already >>>> pulls >>>> >> from an Angular scope so we just leave that alone and pass it >>>> through >>>> >> the |translate filter. You do need to also add the double-colon. >>>> >> >>>> >> One nice side effect is that if the key is not found in the bundle >>>> then >>>> >> the output of the translate filter is the unchanged text. This >>>> means >>>> >> that any code which has not converted to using bundle keys will >>>> still >>>> >> work as expected. And, any third-party providers can just pass in >>>> >> plain text if they don't care about l10n. If they ever do care >>>> about >>>> >> l10n we will just need to provide a means for them to add key/value >>>> >> pairs to the resource bundles. >>>> >> >>>> >> Here is an example for anyone who needs to localize English text >>>> >> embedded in java: >>>> >> >>>> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >>>> >> >>>> >> Stan >>>> >> _______________________________________________ >>>> >> keycloak-dev mailing list >>>> >> keycloak-dev(a)lists.jboss.org >>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> > >>>> > >>>> > >>>> > _______________________________________________ >>>> > keycloak-dev mailing list >>>> > keycloak-dev(a)lists.jboss.org >>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> >>>> >>> >>> >> _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev

On 10/9/2015 6:21 AM, Marko Strukelj wrote: And we can always log the missing key situation into server log - that should be enough for developers to notice it, and fix it. This is basically what happens with the code I wrote for the fix: https://github.com/keycloak/keycloak/pull/1690 You get an error in the console and then a stack trace on the server. The stack trace tells you exactly which key is missing. But the console doesn't crash or anything like that. You just switch back to your original language and everything works fine. On Fri, Oct 9, 2015 at 8:09 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > There's two places where keys can be missing: > > * In a translation - this can be an honest mistake, or the translation > wasn't updated when KC was updated > * Custom keys added - for example when keys are used for display names of > clients, roles, etc.. > > Manually having to go through all sorts of pages to look for missing keys > is very error prone and time consuming, so will not be the best option for > developers. In both cases above the correct way to do this would be to have > a way to verify a message bundle. We need a tool that can quickly identify > if there are missing keys and we could expose that through the admin > console. We currently have a student looking at providing a UI for defining > locales and she is also going to look at adding some way of identifying if > a locale is missing keys and also to easily list only missing keys. > > For end users as I've said they will have no clue what ???key??? is, and > even worse if we throw an exception/error just because a missing key we'll > actually break the whole console just because of a missing key. It's a much > better option to look for the key in another translation and display that. > Chances are they will be able to interpret one or two English words. > Certainly higher chance of that then them being able to interpret ???key???. > > > On 9 October 2015 at 07:51, Thomas Raehalme < > thomas.raehalme(a)aitiofinland.com&gt; wrote: > >> How about returning something noticeable like ???key??? for example? >> On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; wrote: >> >>> That's not putting it to rest at all! Throwing a RuntimeException and >>> rendering the whole admin console useless just because there's a missing >>> key is a horrible idea. >>> >>> On 8 October 2015 at 20:33, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >>> >>>> What if English is the bundle that has a missing key? >>>> >>>> Let's just put this to rest and solve it once and for all. The >>>> simplest solution I can think of is to just compare keys when a new bundle >>>> is loaded. If any bundle has a missing key or it has key not found in the >>>> previous loaded bundle, we throw a RuntimeException. I can submit a patch >>>> for that in just a few minutes. >>>> >>>> >>>> On 10/8/2015 1:28 PM, Stian Thorgersen wrote: >>>> >>>> I'm not sure I'm buying into the argument that displaying the key is >>>> better for developers. Having English suddenly pop-up in a German >>>> translation is just as obvious as a key. Besides as Stan points out you >>>> catch missing keys by comparing missing keys between English and German. >>>> >>>> However, if there is a mistake in a translation then a user may quite >>>> likely be able to interpret English text, while a user will not be able to >>>> interpret a key. So if a key is missing in a translation (which is >>>> obviously a "bug") it's better to display English than to display the key. >>>> >>>> On 8 October 2015 at 14:13, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >>>> >>>>> On 10/8/2015 12:48 AM, Thomas Raehalme wrote: >>>>> >>>>> >>>>> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; >>>>> wrote: >>>>> > >>>>> > With regards to internationalization I have two questions: >>>>> > >>>>> > * Should we fallback to English messages if a key is missing in a >>>>> translation? Alternative is to show key, but that's not going to help anyone >>>>> >>>>> A missing key is a bug and showing the message in the default locale >>>>> may hide the problem. >>>>> >>>>> Even though showing the key does not help the end user it helps the >>>>> developer and identifies the problem. For this reason I think showing the >>>>> key would be a good idea. >>>>> >>>>> For our bundles, we could catch missing keys at build time. >>>>> >>>>> Failing that, I agree that displaying the key is better than falling >>>>> back to English. This is especially true right now while we haven't >>>>> completed the task of converting everything. If we fall back to English we >>>>> won't know if the problem is a missing key or if the text just hasn't been >>>>> converted yet. >>>>> >>>>> > * Should we change message bundles to UTF-8? Or is ISO 8859-1 going >>>>> to work for all languages? >>>>> >>>>> Depends what those all languages are :-) >>>>> >>>>> I think UTF-8 is the best choice as it will handle practically any >>>>> character. >>>>> >>>>> But if you're referring to Java resource bundles the encoding for >>>>> .properties is ISO-8859-1 but there are means to handle any UTF-8 character. >>>>> >>>>> Yes, an UTF-8 character can be encoded in ISO-8859-1. Java provides >>>>> a native2ascii tool for converting entire files. The resource bundle tools >>>>> in most IDE's do this for you automatically. So you just edit as UTF-8 and >>>>> it saves the bundle as ISO-8859-1. >>>>> >>>>> We can read our bundles as UTF-8 if we want to do that. I'd rather >>>>> not, because I'm not sure what we might run into down the road with Java >>>>> assuming resource bundles are always ISO-8859-1. >>>>> >>>>> But I'd like to get the perspective of people who have handled >>>>> resource bundles in languages that are not fully supported by ISO-8859-1. >>>>> Is it too much of a pain to do a conversion or do the tools make the >>>>> process seamless? >>>>> >>>>> Best regards, >>>>> Thomas >>>>> >>>>> > >>>>> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com&gt; >>>>> wrote: >>>>> >> >>>>> >> Marko brought this to my attention yesterday. For some things, we >>>>> >> dynamically create UI. In this case, the java code contains the >>>>> English >>>>> >> text and it needs to be localized. Luckily, the solution was >>>>> pretty >>>>> >> straightforward. We just replace the English text with a key into >>>>> the >>>>> >> message bundle. The html template that displays this text already >>>>> pulls >>>>> >> from an Angular scope so we just leave that alone and pass it >>>>> through >>>>> >> the |translate filter. You do need to also add the double-colon. >>>>> >> >>>>> >> One nice side effect is that if the key is not found in the bundle >>>>> then >>>>> >> the output of the translate filter is the unchanged text. This >>>>> means >>>>> >> that any code which has not converted to using bundle keys will >>>>> still >>>>> >> work as expected. And, any third-party providers can just pass in >>>>> >> plain text if they don't care about l10n. If they ever do care >>>>> about >>>>> >> l10n we will just need to provide a means for them to add key/value >>>>> >> pairs to the resource bundles. >>>>> >> >>>>> >> Here is an example for anyone who needs to localize English text >>>>> >> embedded in java: >>>>> >> >>>>> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >>>>> >> >>>>> >> Stan >>>>> >> _______________________________________________ >>>>> >> keycloak-dev mailing list >>>>> >> keycloak-dev(a)lists.jboss.org >>>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> > >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > keycloak-dev mailing list >>>>> > keycloak-dev(a)lists.jboss.org >>>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>>> >>>>> >>>> >>>> >>> > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ keycloak-dev mailing listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev

On 10/9/2015 7:32 AM, Stian Thorgersen wrote: #1 You're not going to catch all missing keys this way - as I said there's 2 types, custom defined as well which could be missing from default bundle It catches it at load time. As it loads each bundle, it checks against the previously loaded bundle. That will indeed catch all missing keys in any bundle you try to test. I don't know exactly what you mean by "custom defined". Somehow a third-party bundle must be merged with our default bundle. Unless I completely misunderstand, the code I wrote will still work.

#2 Rendering the whole bundle useless just because you're missing one key is just daft It is the correct thing to do. A missing key is like a null pointer. It deserves a RuntimeException.

#3 There will quite likely be separate teams that do translations to those that do development, which means stack traces and log output is not the solution I don't see what that has to do with anything. You start with a set of bundles containing all the correct keys. Then you translate each bundle. If you accidentally delete a key then you want to know that right away. But we should indeed ask the translation team what they want to see.

#4 Doing a check each time you pull a message bundle to compare with the base bundle is probably not that expensive, but still pretty daft thing to do You only load each bundle once. So the check only happens the first time you request the bundle. #5 A proper util that's used to translate bundles is much better - we can implement a page in the admin console that allows you to validate a bundle and print out all missing bundles. This is something that would be more developer friendly and also would be usable by non-developers (aka people with other language skills than Java) We should ask the translation team what they want to see and how they do their work. I'm sure that they don't expect a tool to be built into the product. None of our other products have that.

On 9 October 2015 at 13:24, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: > On 10/9/2015 6:21 AM, Marko Strukelj wrote: > > And we can always log the missing key situation into server log - that > should be enough for developers to notice it, and fix it. > > This is basically what happens with the code I wrote for the fix: > https://github.com/keycloak/keycloak/pull/1690 > > You get an error in the console and then a stack trace on the server. > The stack trace tells you exactly which key is missing. But the console > doesn't crash or anything like that. You just switch back to your original > language and everything works fine. > > > On Fri, Oct 9, 2015 at 8:09 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; > wrote: > >> There's two places where keys can be missing: >> >> * In a translation - this can be an honest mistake, or the translation >> wasn't updated when KC was updated >> * Custom keys added - for example when keys are used for display names >> of clients, roles, etc.. >> >> Manually having to go through all sorts of pages to look for missing >> keys is very error prone and time consuming, so will not be the best option >> for developers. In both cases above the correct way to do this would be to >> have a way to verify a message bundle. We need a tool that can quickly >> identify if there are missing keys and we could expose that through the >> admin console. We currently have a student looking at providing a UI for >> defining locales and she is also going to look at adding some way of >> identifying if a locale is missing keys and also to easily list only >> missing keys. >> >> For end users as I've said they will have no clue what ???key??? is, and >> even worse if we throw an exception/error just because a missing key we'll >> actually break the whole console just because of a missing key. It's a much >> better option to look for the key in another translation and display that. >> Chances are they will be able to interpret one or two English words. >> Certainly higher chance of that then them being able to interpret ???key???. >> >> >> On 9 October 2015 at 07:51, Thomas Raehalme < >> thomas.raehalme(a)aitiofinland.com&gt; wrote: >> >>> How about returning something noticeable like ???key??? for example? >>> On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; wrote: >>> >>>> That's not putting it to rest at all! Throwing a RuntimeException and >>>> rendering the whole admin console useless just because there's a missing >>>> key is a horrible idea. >>>> >>>> On 8 October 2015 at 20:33, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >>>> >>>>> What if English is the bundle that has a missing key? >>>>> >>>>> Let's just put this to rest and solve it once and for all. The >>>>> simplest solution I can think of is to just compare keys when a new bundle >>>>> is loaded. If any bundle has a missing key or it has key not found in the >>>>> previous loaded bundle, we throw a RuntimeException. I can submit a patch >>>>> for that in just a few minutes. >>>>> >>>>> >>>>> On 10/8/2015 1:28 PM, Stian Thorgersen wrote: >>>>> >>>>> I'm not sure I'm buying into the argument that displaying the key is >>>>> better for developers. Having English suddenly pop-up in a German >>>>> translation is just as obvious as a key. Besides as Stan points out you >>>>> catch missing keys by comparing missing keys between English and German. >>>>> >>>>> However, if there is a mistake in a translation then a user may quite >>>>> likely be able to interpret English text, while a user will not be able to >>>>> interpret a key. So if a key is missing in a translation (which is >>>>> obviously a "bug") it's better to display English than to display the key. >>>>> >>>>> On 8 October 2015 at 14:13, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >>>>> >>>>>> On 10/8/2015 12:48 AM, Thomas Raehalme wrote: >>>>>> >>>>>> >>>>>> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; >>>>>> wrote: >>>>>> > >>>>>> > With regards to internationalization I have two questions: >>>>>> > >>>>>> > * Should we fallback to English messages if a key is missing in a >>>>>> translation? Alternative is to show key, but that's not going to help anyone >>>>>> >>>>>> A missing key is a bug and showing the message in the default locale >>>>>> may hide the problem. >>>>>> >>>>>> Even though showing the key does not help the end user it helps the >>>>>> developer and identifies the problem. For this reason I think showing the >>>>>> key would be a good idea. >>>>>> >>>>>> For our bundles, we could catch missing keys at build time. >>>>>> >>>>>> Failing that, I agree that displaying the key is better than falling >>>>>> back to English. This is especially true right now while we haven't >>>>>> completed the task of converting everything. If we fall back to English we >>>>>> won't know if the problem is a missing key or if the text just hasn't been >>>>>> converted yet. >>>>>> >>>>>> > * Should we change message bundles to UTF-8? Or is ISO 8859-1 >>>>>> going to work for all languages? >>>>>> >>>>>> Depends what those all languages are :-) >>>>>> >>>>>> I think UTF-8 is the best choice as it will handle practically any >>>>>> character. >>>>>> >>>>>> But if you're referring to Java resource bundles the encoding for >>>>>> .properties is ISO-8859-1 but there are means to handle any UTF-8 character. >>>>>> >>>>>> Yes, an UTF-8 character can be encoded in ISO-8859-1. Java provides >>>>>> a native2ascii tool for converting entire files. The resource bundle tools >>>>>> in most IDE's do this for you automatically. So you just edit as UTF-8 and >>>>>> it saves the bundle as ISO-8859-1. >>>>>> >>>>>> We can read our bundles as UTF-8 if we want to do that. I'd rather >>>>>> not, because I'm not sure what we might run into down the road with Java >>>>>> assuming resource bundles are always ISO-8859-1. >>>>>> >>>>>> But I'd like to get the perspective of people who have handled >>>>>> resource bundles in languages that are not fully supported by ISO-8859-1. >>>>>> Is it too much of a pain to do a conversion or do the tools make the >>>>>> process seamless? >>>>>> >>>>>> Best regards, >>>>>> Thomas >>>>>> >>>>>> > >>>>>> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com&gt; >>>>>> wrote: >>>>>> >> >>>>>> >> Marko brought this to my attention yesterday. For some things, we >>>>>> >> dynamically create UI. In this case, the java code contains the >>>>>> English >>>>>> >> text and it needs to be localized. Luckily, the solution was >>>>>> pretty >>>>>> >> straightforward. We just replace the English text with a key >>>>>> into the >>>>>> >> message bundle. The html template that displays this text >>>>>> already pulls >>>>>> >> from an Angular scope so we just leave that alone and pass it >>>>>> through >>>>>> >> the |translate filter. You do need to also add the double-colon. >>>>>> >> >>>>>> >> One nice side effect is that if the key is not found in the >>>>>> bundle then >>>>>> >> the output of the translate filter is the unchanged text. This >>>>>> means >>>>>> >> that any code which has not converted to using bundle keys will >>>>>> still >>>>>> >> work as expected. And, any third-party providers can just pass >>>>>> in >>>>>> >> plain text if they don't care about l10n. If they ever do care >>>>>> about >>>>>> >> l10n we will just need to provide a means for them to add >>>>>> key/value >>>>>> >> pairs to the resource bundles. >>>>>> >> >>>>>> >> Here is an example for anyone who needs to localize English text >>>>>> >> embedded in java: >>>>>> >> >>>>>> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >>>>>> >> >>>>>> >> Stan >>>>>> >> _______________________________________________ >>>>>> >> keycloak-dev mailing list >>>>>> >> keycloak-dev(a)lists.jboss.org >>>>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> > >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > keycloak-dev mailing list >>>>>> > keycloak-dev(a)lists.jboss.org >>>>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev >

By the way I'm pretty sure our translation theme will take our English bundle and translate it to a language. They won't need to start/stop the server and open the admin console to check that they've translated all keys. They should have a tool/script for that. On 9 October 2015 at 13:56, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > > > On 9 October 2015 at 13:46, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: > >> On 10/9/2015 7:32 AM, Stian Thorgersen wrote: >> >> #1 You're not going to catch all missing keys this way - as I said >> there's 2 types, custom defined as well which could be missing from default >> bundle >> >> It catches it at load time. As it loads each bundle, it checks against >> the previously loaded bundle. That will indeed catch all missing keys in >> any bundle you try to test. >> >> I don't know exactly what you mean by "custom defined". Somehow a >> third-party bundle must be merged with our default bundle. Unless I >> completely misunderstand, the code I wrote will still work. >> > > New keys can be defined by using keys in client descriptions, client > names, etc, etc.. These won't be in our message bundle, but would be in the > bundle in a customers theme. This is why message bundles inherit from the > parent theme. Custom providers and themes can further introduce new keys. > > >> >> #2 Rendering the whole bundle useless just because you're missing one >> key is just daft >> >> It is the correct thing to do. A missing key is like a null pointer. >> It deserves a RuntimeException. >> > > No it's not the correct thing to do. > > >> >> #3 There will quite likely be separate teams that do translations to >> those that do development, which means stack traces and log output is not >> the solution >> >> I don't see what that has to do with anything. You start with a set of >> bundles containing all the correct keys. Then you translate each bundle. >> If you accidentally delete a key then you want to know that right away. >> But we should indeed ask the translation team what they want to see. >> > > Sure, go ahead and ask them if they want to look for RuntimeExceptions in > the server log. > > >> >> #4 Doing a check each time you pull a message bundle to compare with the >> base bundle is probably not that expensive, but still pretty daft thing to >> do >> >> You only load each bundle once. So the check only happens the first >> time you request the bundle. >> >> #5 A proper util that's used to translate bundles is much better - we >> can implement a page in the admin console that allows you to validate a >> bundle and print out all missing bundles. This is something that would be >> more developer friendly and also would be usable by non-developers (aka >> people with other language skills than Java) >> >> We should ask the translation team what they want to see and how they do >> their work. I'm sure that they don't expect a tool to be built into the >> product. None of our other products have that. >> > > I don't just care about our translation team, they will translate the > built in keys, but they won't translate the ones introduced by users. > > > >> >> >> >> On 9 October 2015 at 13:24, Stan Silvert <ssilvert(a)redhat.com&gt; wrote: >> >>> On 10/9/2015 6:21 AM, Marko Strukelj wrote: >>> >>> And we can always log the missing key situation into server log - that >>> should be enough for developers to notice it, and fix it. >>> >>> This is basically what happens with the code I wrote for the fix: >>> https://github.com/keycloak/keycloak/pull/1690 >>> >>> You get an error in the console and then a stack trace on the server. >>> The stack trace tells you exactly which key is missing. But the console >>> doesn't crash or anything like that. You just switch back to your original >>> language and everything works fine. >>> >>> >>> On Fri, Oct 9, 2015 at 8:09 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; >>> wrote: >>> >>>> There's two places where keys can be missing: >>>> >>>> * In a translation - this can be an honest mistake, or the translation >>>> wasn't updated when KC was updated >>>> * Custom keys added - for example when keys are used for display names >>>> of clients, roles, etc.. >>>> >>>> Manually having to go through all sorts of pages to look for missing >>>> keys is very error prone and time consuming, so will not be the best option >>>> for developers. In both cases above the correct way to do this would be to >>>> have a way to verify a message bundle. We need a tool that can quickly >>>> identify if there are missing keys and we could expose that through the >>>> admin console. We currently have a student looking at providing a UI for >>>> defining locales and she is also going to look at adding some way of >>>> identifying if a locale is missing keys and also to easily list only >>>> missing keys. >>>> >>>> For end users as I've said they will have no clue what ???key??? is, >>>> and even worse if we throw an exception/error just because a missing key >>>> we'll actually break the whole console just because of a missing key. It's >>>> a much better option to look for the key in another translation and display >>>> that. Chances are they will be able to interpret one or two English words. >>>> Certainly higher chance of that then them being able to interpret ???key???. >>>> >>>> >>>> On 9 October 2015 at 07:51, Thomas Raehalme < >>>> thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>> >>>>> How about returning something noticeable like ???key??? for example? >>>>> On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> That's not putting it to rest at all! Throwing a RuntimeException >>>>>> and rendering the whole admin console useless just because there's a >>>>>> missing key is a horrible idea. >>>>>> >>>>>> On 8 October 2015 at 20:33, Stan Silvert <ssilvert(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> What if English is the bundle that has a missing key? >>>>>>> >>>>>>> Let's just put this to rest and solve it once and for all. The >>>>>>> simplest solution I can think of is to just compare keys when a new bundle >>>>>>> is loaded. If any bundle has a missing key or it has key not found in the >>>>>>> previous loaded bundle, we throw a RuntimeException. I can submit a patch >>>>>>> for that in just a few minutes. >>>>>>> >>>>>>> >>>>>>> On 10/8/2015 1:28 PM, Stian Thorgersen wrote: >>>>>>> >>>>>>> I'm not sure I'm buying into the argument that displaying the key >>>>>>> is better for developers. Having English suddenly pop-up in a German >>>>>>> translation is just as obvious as a key. Besides as Stan points out you >>>>>>> catch missing keys by comparing missing keys between English and German. >>>>>>> >>>>>>> However, if there is a mistake in a translation then a user may >>>>>>> quite likely be able to interpret English text, while a user will not be >>>>>>> able to interpret a key. So if a key is missing in a translation (which is >>>>>>> obviously a "bug") it's better to display English than to display the key. >>>>>>> >>>>>>> On 8 October 2015 at 14:13, Stan Silvert <ssilvert(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> >>>>>>>> On 10/8/2015 12:48 AM, Thomas Raehalme wrote: >>>>>>>> >>>>>>>> >>>>>>>> On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com&gt; >>>>>>>> wrote: >>>>>>>> > >>>>>>>> > With regards to internationalization I have two questions: >>>>>>>> > >>>>>>>> > * Should we fallback to English messages if a key is missing in >>>>>>>> a translation? Alternative is to show key, but that's not going to help >>>>>>>> anyone >>>>>>>> >>>>>>>> A missing key is a bug and showing the message in the default >>>>>>>> locale may hide the problem. >>>>>>>> >>>>>>>> Even though showing the key does not help the end user it helps >>>>>>>> the developer and identifies the problem. For this reason I think showing >>>>>>>> the key would be a good idea. >>>>>>>> >>>>>>>> For our bundles, we could catch missing keys at build time. >>>>>>>> >>>>>>>> Failing that, I agree that displaying the key is better than >>>>>>>> falling back to English. This is especially true right now while we >>>>>>>> haven't completed the task of converting everything. If we fall back to >>>>>>>> English we won't know if the problem is a missing key or if the text just >>>>>>>> hasn't been converted yet. >>>>>>>> >>>>>>>> > * Should we change message bundles to UTF-8? Or is ISO 8859-1 >>>>>>>> going to work for all languages? >>>>>>>> >>>>>>>> Depends what those all languages are :-) >>>>>>>> >>>>>>>> I think UTF-8 is the best choice as it will handle practically any >>>>>>>> character. >>>>>>>> >>>>>>>> But if you're referring to Java resource bundles the encoding for >>>>>>>> .properties is ISO-8859-1 but there are means to handle any UTF-8 character. >>>>>>>> >>>>>>>> Yes, an UTF-8 character can be encoded in ISO-8859-1. Java >>>>>>>> provides a native2ascii tool for converting entire files. The resource >>>>>>>> bundle tools in most IDE's do this for you automatically. So you just edit >>>>>>>> as UTF-8 and it saves the bundle as ISO-8859-1. >>>>>>>> >>>>>>>> We can read our bundles as UTF-8 if we want to do that. I'd >>>>>>>> rather not, because I'm not sure what we might run into down the road with >>>>>>>> Java assuming resource bundles are always ISO-8859-1. >>>>>>>> >>>>>>>> But I'd like to get the perspective of people who have handled >>>>>>>> resource bundles in languages that are not fully supported by ISO-8859-1. >>>>>>>> Is it too much of a pain to do a conversion or do the tools make the >>>>>>>> process seamless? >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Thomas >>>>>>>> >>>>>>>> > >>>>>>>> > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com&gt; >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >> Marko brought this to my attention yesterday. For some things, >>>>>>>> we >>>>>>>> >> dynamically create UI. In this case, the java code contains >>>>>>>> the English >>>>>>>> >> text and it needs to be localized. Luckily, the solution was >>>>>>>> pretty >>>>>>>> >> straightforward. We just replace the English text with a key >>>>>>>> into the >>>>>>>> >> message bundle. The html template that displays this text >>>>>>>> already pulls >>>>>>>> >> from an Angular scope so we just leave that alone and pass it >>>>>>>> through >>>>>>>> >> the |translate filter. You do need to also add the >>>>>>>> double-colon. >>>>>>>> >> >>>>>>>> >> One nice side effect is that if the key is not found in the >>>>>>>> bundle then >>>>>>>> >> the output of the translate filter is the unchanged text. This >>>>>>>> means >>>>>>>> >> that any code which has not converted to using bundle keys will >>>>>>>> still >>>>>>>> >> work as expected. And, any third-party providers can just >>>>>>>> pass in >>>>>>>> >> plain text if they don't care about l10n. If they ever do care >>>>>>>> about >>>>>>>> >> l10n we will just need to provide a means for them to add >>>>>>>> key/value >>>>>>>> >> pairs to the resource bundles. >>>>>>>> >> >>>>>>>> >> Here is an example for anyone who needs to localize English text >>>>>>>> >> embedded in java: >>>>>>>> >> >>>>>>>> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >>>>>>>> >> >>>>>>>> >> Stan >>>>>>>> >> _______________________________________________ >>>>>>>> >> keycloak-dev mailing list >>>>>>>> >> keycloak-dev(a)lists.jboss.org >>>>>>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > _______________________________________________ >>>>>>>> > keycloak-dev mailing list >>>>>>>> > keycloak-dev(a)lists.jboss.org >>>>>>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> >

The translation team will expect the key/value pairs to be in a po file. Developers push this to Zanata. The translation is done by the globalization team in Zanata. The translated po files are then incorporated into a build. The I18N QE team takes the build which has the translated po file and does 2 things. 1. They verify that the translated string appears as expected. 2. They verify that the translation is appropriate in the context in which it is presented. Hope this helps, Debi ------------------------------------------------------------------------ *From: *"Stian Thorgersen" <sthorger(a)redhat.com&gt; *To: *"Stian Thorgersen" <stian(a)redhat.com&gt; *Cc: *"keycloak-dev" <keycloak-dev(a)lists.jboss.org&gt; *Sent: *Friday, October 9, 2015 7:57:54 AM *Subject: *Re: [keycloak-dev] i18n/l10n: English text in java code Translation team that is On 9 October 2015 at 13:57, Stian Thorgersen <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote: By the way I'm pretty sure our translation theme will take our English bundle and translate it to a language. They won't need to start/stop the server and open the admin console to check that they've translated all keys. They should have a tool/script for that. On 9 October 2015 at 13:56, Stian Thorgersen <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote: On 9 October 2015 at 13:46, Stan Silvert <ssilvert(a)redhat.com <mailto:ssilvert@redhat.com>> wrote: On 10/9/2015 7:32 AM, Stian Thorgersen wrote: #1 You're not going to catch all missing keys this way - as I said there's 2 types, custom defined as well which could be missing from default bundle It catches it at load time. As it loads each bundle, it checks against the previously loaded bundle. That will indeed catch all missing keys in any bundle you try to test. I don't know exactly what you mean by "custom defined". Somehow a third-party bundle must be merged with our default bundle. Unless I completely misunderstand, the code I wrote will still work. New keys can be defined by using keys in client descriptions, client names, etc, etc.. These won't be in our message bundle, but would be in the bundle in a customers theme. This is why message bundles inherit from the parent theme. Custom providers and themes can further introduce new keys. #2 Rendering the whole bundle useless just because you're missing one key is just daft It is the correct thing to do. A missing key is like a null pointer. It deserves a RuntimeException. No it's not the correct thing to do. #3 There will quite likely be separate teams that do translations to those that do development, which means stack traces and log output is not the solution I don't see what that has to do with anything. You start with a set of bundles containing all the correct keys. Then you translate each bundle. If you accidentally delete a key then you want to know that right away. But we should indeed ask the translation team what they want to see. Sure, go ahead and ask them if they want to look for RuntimeExceptions in the server log. #4 Doing a check each time you pull a message bundle to compare with the base bundle is probably not that expensive, but still pretty daft thing to do You only load each bundle once. So the check only happens the first time you request the bundle. #5 A proper util that's used to translate bundles is much better - we can implement a page in the admin console that allows you to validate a bundle and print out all missing bundles. This is something that would be more developer friendly and also would be usable by non-developers (aka people with other language skills than Java) We should ask the translation team what they want to see and how they do their work. I'm sure that they don't expect a tool to be built into the product. None of our other products have that. I don't just care about our translation team, they will translate the built in keys, but they won't translate the ones introduced by users. On 9 October 2015 at 13:24, Stan Silvert <ssilvert(a)redhat.com <mailto:ssilvert@redhat.com>> wrote: On 10/9/2015 6:21 AM, Marko Strukelj wrote: And we can always log the missing key situation into server log - that should be enough for developers to notice it, and fix it. This is basically what happens with the code I wrote for the fix: https://github.com/keycloak/keycloak/pull/1690 You get an error in the console and then a stack trace on the server. The stack trace tells you exactly which key is missing. But the console doesn't crash or anything like that. You just switch back to your original language and everything works fine. On Fri, Oct 9, 2015 at 8:09 AM, Stian Thorgersen <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote: There's two places where keys can be missing: * In a translation - this can be an honest mistake, or the translation wasn't updated when KC was updated * Custom keys added - for example when keys are used for display names of clients, roles, etc.. Manually having to go through all sorts of pages to look for missing keys is very error prone and time consuming, so will not be the best option for developers. In both cases above the correct way to do this would be to have a way to verify a message bundle. We need a tool that can quickly identify if there are missing keys and we could expose that through the admin console. We currently have a student looking at providing a UI for defining locales and she is also going to look at adding some way of identifying if a locale is missing keys and also to easily list only missing keys. For end users as I've said they will have no clue what ???key??? is, and even worse if we throw an exception/error just because a missing key we'll actually break the whole console just because of a missing key. It's a much better option to look for the key in another translation and display that. Chances are they will be able to interpret one or two English words. Certainly higher chance of that then them being able to interpret ???key???. On 9 October 2015 at 07:51, Thomas Raehalme <thomas.raehalme(a)aitiofinland.com <mailto:thomas.raehalme@aitiofinland.com>> wrote: How about returning something noticeable like ???key??? for example? On Oct 9, 2015 8:10 AM, "Stian Thorgersen" <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote: That's not putting it to rest at all! Throwing a RuntimeException and rendering the whole admin console useless just because there's a missing key is a horrible idea. On 8 October 2015 at 20:33, Stan Silvert <ssilvert(a)redhat.com <mailto:ssilvert@redhat.com>> wrote: What if English is the bundle that has a missing key? Let's just put this to rest and solve it once and for all. The simplest solution I can think of is to just compare keys when a new bundle is loaded. If any bundle has a missing key or it has key not found in the previous loaded bundle, we throw a RuntimeException. I can submit a patch for that in just a few minutes. On 10/8/2015 1:28 PM, Stian Thorgersen wrote: I'm not sure I'm buying into the argument that displaying the key is better for developers. Having English suddenly pop-up in a German translation is just as obvious as a key. Besides as Stan points out you catch missing keys by comparing missing keys between English and German. However, if there is a mistake in a translation then a user may quite likely be able to interpret English text, while a user will not be able to interpret a key. So if a key is missing in a translation (which is obviously a "bug") it's better to display English than to display the key. On 8 October 2015 at 14:13, Stan Silvert <ssilvert(a)redhat.com <mailto:ssilvert@redhat.com>> wrote: On 10/8/2015 12:48 AM, Thomas Raehalme wrote: On Oct 8, 2015 6:53 AM, "Stian Thorgersen" <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote: > > With regards to internationalization I have two questions: > > * Should we fallback to English messages if a key is missing in a translation? Alternative is to show key, but that's not going to help anyone A missing key is a bug and showing the message in the default locale may hide the problem. Even though showing the key does not help the end user it helps the developer and identifies the problem. For this reason I think showing the key would be a good idea. For our bundles, we could catch missing keys at build time. Failing that, I agree that displaying the key is better than falling back to English. This is especially true right now while we haven't completed the task of converting everything. If we fall back to English we won't know if the problem is a missing key or if the text just hasn't been converted yet. > * Should we change message bundles to UTF-8? Or is ISO 8859-1 going to work for all languages? Depends what those all languages are :-) I think UTF-8 is the best choice as it will handle practically any character. But if you're referring to Java resource bundles the encoding for .properties is ISO-8859-1 but there are means to handle any UTF-8 character. Yes, an UTF-8 character can be encoded in ISO-8859-1. Java provides a native2ascii tool for converting entire files. The resource bundle tools in most IDE's do this for you automatically. So you just edit as UTF-8 and it saves the bundle as ISO-8859-1. We can read our bundles as UTF-8 if we want to do that. I'd rather not, because I'm not sure what we might run into down the road with Java assuming resource bundles are always ISO-8859-1. But I'd like to get the perspective of people who have handled resource bundles in languages that are not fully supported by ISO-8859-1. Is it too much of a pain to do a conversion or do the tools make the process seamless? Best regards, Thomas > > On 7 October 2015 at 18:42, Stan Silvert <ssilvert(a)redhat.com <mailto:ssilvert@redhat.com>> wrote: >> >> Marko brought this to my attention yesterday. For some things, we >> dynamically create UI. In this case, the java code contains the English >> text and it needs to be localized. Luckily, the solution was pretty >> straightforward. We just replace the English text with a key into the >> message bundle. The html template that displays this text already pulls >> from an Angular scope so we just leave that alone and pass it through >> the |translate filter. You do need to also add the double-colon. >> >> One nice side effect is that if the key is not found in the bundle then >> the output of the translate filter is the unchanged text. This means >> that any code which has not converted to using bundle keys will still >> work as expected. And, any third-party providers can just pass in >> plain text if they don't care about l10n. If they ever do care about >> l10n we will just need to provide a means for them to add key/value >> pairs to the resource bundles. >> >> Here is an example for anyone who needs to localize English text >> embedded in java: >> https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd88... >> >> Stan >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -- Debi Rieden Program Manager for JBoss Data Virtualization and JBoss Identity (Keycloak) Westford (3S391) http://westford-map.itos.redhat.com/index.html#3S391 Office: 978-392-1023; Cell 617-669-2934 drieden(a)redhat.com #program, #boston _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev