It makes sense, maybe file a Jira associated with: https://issues.jboss.org/browse/KEYCLOAK-3259 ? On 2016-07-04, Hiroyuki Wada wrote: > Hello all, > > I am trying to translate all base theme messages to my country > language, Japanese. And I'd like to contribute them. Before that work, > I'd like to propose about the files encoding. > > Currently, the message files (*.properties) are loaded with ISO-8859-1 > encoding. Therefore, it is necessary to convert the files by > 'native2ascii' command beforehand. However we can directly read the > property files with UTF-8 encoding in java 1.6 or later because > 'java.util.Properties#load(java.io.Reader)' method was introduced as > below. > > http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... > > So, my proposal is supporting the message files with UTF-8 encoding. I > believe that it's very developers/customers friendly. In addition, we > can easily review the translated messages on the github pull request > view and so on. What do you think? > > If it's ok, I'll create a JIRA issue and create a pull request. > > Regards, > > -- > Hiroyuki Wada, > Developer, > Nomura Research Institute, Ltd. > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -- abstractj PGP: 0x84DC9914 _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev

Thanks for your comment. > If we want to change to utf-8 we'd still need to support iso.. for backwards compatibility. If we change to UTF-8, we can still read unicode codepoint like '\u00e8'. There is an incompatibility when non-ascii characters are used in message properties. The non-ascii characters are 0xA0 - 0xFF codes (please refer codepage layout: https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) I think the non-ascii characters might be used in French messages like 'à' so I agree to support ISO-8859-1 for backwards compatibility. To support this, I think we can add a property like "messageEncoding" in keycloak-server.json as below. Is it a good idea? "theme": { "staticMaxAge": 2592000, "cacheTemplates": true, "cacheThemes": true, "messageEncoding": "ISO-8859-1", "folder": { "dir": "${jboss.home.dir}/themes" } }, Regards, On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > We have in the past discussed this and decided to stick with ISO-8859-1. > That was probably not the best idea though. If we want to change to utf-8 > we'd still need to support iso.. for backwards compatibility. > > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; wrote: >> >> It makes sense, maybe file a Jira associated with: >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >> >> On 2016-07-04, Hiroyuki Wada wrote: >> > Hello all, >> > >> > I am trying to translate all base theme messages to my country >> > language, Japanese. And I'd like to contribute them. Before that work, >> > I'd like to propose about the files encoding. >> > >> > Currently, the message files (*.properties) are loaded with ISO-8859-1 >> > encoding. Therefore, it is necessary to convert the files by >> > 'native2ascii' command beforehand. However we can directly read the >> > property files with UTF-8 encoding in java 1.6 or later because >> > 'java.util.Properties#load(java.io.Reader)' method was introduced as >> > below. >> > >> > >> > http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >> > >> > So, my proposal is supporting the message files with UTF-8 encoding. I >> > believe that it's very developers/customers friendly. In addition, we >> > can easily review the translated messages on the github pull request >> > view and so on. What do you think? >> > >> > If it's ok, I'll create a JIRA issue and create a pull request. >> > >> > Regards, >> > >> > -- >> > Hiroyuki Wada, >> > Developer, >> > Nomura Research Institute, Ltd. >> > _______________________________________________ >> > keycloak-dev mailing list >> > keycloak-dev(a)lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> -- >> >> abstractj >> PGP: 0x84DC9914 >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >

Please correct me if I am wrong, but I have been under the impression that Java .properties files should always use encoding ISO-8859-1. Characters not present in ISO-8859-1 can be written in \uxxxx. Won't it make things confusing to developers if another encoding is used here instead? https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... If alternate encodings are desired how about supporting the XML format of Properties? Best regards, Thomas On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > Both iso-8859-1 and utf-8 message bundles should be able to co-exist. > > We can allow specifying the encoding in a comment on the first line like > this: > > # encoding=utf-8 > key=value > > # encoding=iso-8859-1 > key=value > > If the first line in the file doesn't contain the comment with the > encoding then we should default to iso-8859-1 for backwards compatibility > > On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > >> Thanks for your comment. >> >> > If we want to change to utf-8 we'd still need to support iso.. for >> backwards compatibility. >> >> If we change to UTF-8, we can still read unicode codepoint like '\u00e8'. >> There is an incompatibility when non-ascii characters are used in >> message properties. >> The non-ascii characters are 0xA0 - 0xFF codes (please refer codepage >> layout: https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >> >> I think the non-ascii characters might be used in French messages like >> 'à' so I agree to support ISO-8859-1 for backwards compatibility. >> To support this, I think we can add a property like "messageEncoding" >> in keycloak-server.json as below. Is it a good idea? >> >> "theme": { >> "staticMaxAge": 2592000, >> "cacheTemplates": true, >> "cacheThemes": true, >> "messageEncoding": "ISO-8859-1", >> "folder": { >> "dir": "${jboss.home.dir}/themes" >> } >> }, >> >> Regards, >> >> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen <sthorger(a)redhat.com&gt; >> wrote: >> > We have in the past discussed this and decided to stick with >> ISO-8859-1. >> > That was probably not the best idea though. If we want to change to >> utf-8 >> > we'd still need to support iso.. for backwards compatibility. >> > >> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; wrote: >> >> >> >> It makes sense, maybe file a Jira associated with: >> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >> >> >> >> On 2016-07-04, Hiroyuki Wada wrote: >> >> > Hello all, >> >> > >> >> > I am trying to translate all base theme messages to my country >> >> > language, Japanese. And I'd like to contribute them. Before that >> work, >> >> > I'd like to propose about the files encoding. >> >> > >> >> > Currently, the message files (*.properties) are loaded with >> ISO-8859-1 >> >> > encoding. Therefore, it is necessary to convert the files by >> >> > 'native2ascii' command beforehand. However we can directly read the >> >> > property files with UTF-8 encoding in java 1.6 or later because >> >> > 'java.util.Properties#load(java.io.Reader)' method was introduced as >> >> > below. >> >> > >> >> > >> >> > >> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >> >> > >> >> > So, my proposal is supporting the message files with UTF-8 >> encoding. I >> >> > believe that it's very developers/customers friendly. In addition, >> we >> >> > can easily review the translated messages on the github pull request >> >> > view and so on. What do you think? >> >> > >> >> > If it's ok, I'll create a JIRA issue and create a pull request. >> >> > >> >> > Regards, >> >> > >> >> > -- >> >> > Hiroyuki Wada, >> >> > Developer, >> >> > Nomura Research Institute, Ltd. >> >> > _______________________________________________ >> >> > keycloak-dev mailing list >> >> > keycloak-dev(a)lists.jboss.org >> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> -- >> >> >> >> abstractj >> >> PGP: 0x84DC9914 >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> > >> > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev >

Actually, just thought of a con for encoding header in properties file. IDEs would not honor it and would in most cases believe the file is iso-8859-1. On 6 July 2016 at 08:41, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > Java properties file does by default use iso-8859-1, but that's far from > elegant for some languages. I don't see any issues with supporting utf-8 > with a header like I suggested. It would be fairly easy to implement. Java > does use utf-8 by default for XML property files, but I don't think that's > a very nice format to edit the messages in. > > On 6 July 2016 at 08:32, Thomas Raehalme < > thomas.raehalme(a)aitiofinland.com&gt; wrote: > >> Please correct me if I am wrong, but I have been under the impression >> that Java .properties files should always use encoding ISO-8859-1. >> Characters not present in ISO-8859-1 can be written in \uxxxx. Won't it >> make things confusing to developers if another encoding is used here >> instead? >> >> >> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >> >> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >> >> If alternate encodings are desired how about supporting the XML format >> of Properties? >> >> Best regards, >> Thomas >> >> >> On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; >> wrote: >> >>> Both iso-8859-1 and utf-8 message bundles should be able to co-exist. >>> >>> We can allow specifying the encoding in a comment on the first line >>> like this: >>> >>> # encoding=utf-8 >>> key=value >>> >>> # encoding=iso-8859-1 >>> key=value >>> >>> If the first line in the file doesn't contain the comment with the >>> encoding then we should default to iso-8859-1 for backwards compatibility >>> >>> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>> >>>> Thanks for your comment. >>>> >>>> > If we want to change to utf-8 we'd still need to support iso.. for >>>> backwards compatibility. >>>> >>>> If we change to UTF-8, we can still read unicode codepoint like >>>> '\u00e8'. >>>> There is an incompatibility when non-ascii characters are used in >>>> message properties. >>>> The non-ascii characters are 0xA0 - 0xFF codes (please refer codepage >>>> layout: https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >>>> >>>> I think the non-ascii characters might be used in French messages like >>>> 'à' so I agree to support ISO-8859-1 for backwards compatibility. >>>> To support this, I think we can add a property like "messageEncoding" >>>> in keycloak-server.json as below. Is it a good idea? >>>> >>>> "theme": { >>>> "staticMaxAge": 2592000, >>>> "cacheTemplates": true, >>>> "cacheThemes": true, >>>> "messageEncoding": "ISO-8859-1", >>>> "folder": { >>>> "dir": "${jboss.home.dir}/themes" >>>> } >>>> }, >>>> >>>> Regards, >>>> >>>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen <sthorger(a)redhat.com&gt; >>>> wrote: >>>> > We have in the past discussed this and decided to stick with >>>> ISO-8859-1. >>>> > That was probably not the best idea though. If we want to change to >>>> utf-8 >>>> > we'd still need to support iso.. for backwards compatibility. >>>> > >>>> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; wrote: >>>> >> >>>> >> It makes sense, maybe file a Jira associated with: >>>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >>>> >> >>>> >> On 2016-07-04, Hiroyuki Wada wrote: >>>> >> > Hello all, >>>> >> > >>>> >> > I am trying to translate all base theme messages to my country >>>> >> > language, Japanese. And I'd like to contribute them. Before that >>>> work, >>>> >> > I'd like to propose about the files encoding. >>>> >> > >>>> >> > Currently, the message files (*.properties) are loaded with >>>> ISO-8859-1 >>>> >> > encoding. Therefore, it is necessary to convert the files by >>>> >> > 'native2ascii' command beforehand. However we can directly read >>>> the >>>> >> > property files with UTF-8 encoding in java 1.6 or later because >>>> >> > 'java.util.Properties#load(java.io.Reader)' method was introduced >>>> as >>>> >> > below. >>>> >> > >>>> >> > >>>> >> > >>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >>>> >> > >>>> >> > So, my proposal is supporting the message files with UTF-8 >>>> encoding. I >>>> >> > believe that it's very developers/customers friendly. In >>>> addition, we >>>> >> > can easily review the translated messages on the github pull >>>> request >>>> >> > view and so on. What do you think? >>>> >> > >>>> >> > If it's ok, I'll create a JIRA issue and create a pull request. >>>> >> > >>>> >> > Regards, >>>> >> > >>>> >> > -- >>>> >> > Hiroyuki Wada, >>>> >> > Developer, >>>> >> > Nomura Research Institute, Ltd. >>>> >> > _______________________________________________ >>>> >> > keycloak-dev mailing list >>>> >> > keycloak-dev(a)lists.jboss.org >>>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> >>>> >> -- >>>> >> >>>> >> abstractj >>>> >> PGP: 0x84DC9914 >>>> >> _______________________________________________ >>>> >> keycloak-dev mailing list >>>> >> keycloak-dev(a)lists.jboss.org >>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> > >>>> > >>>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> >

On 6 July 2016 at 09:07, Thomas Raehalme <thomas.raehalme(a)aitiofinland.com <mailto:thomas.raehalme@aitiofinland.com>> wrote: I didn't say using ISO-8859-1 with .properties was elegant, just the way it is :-) The fact that it's the default doesn't mean it has to be the only way ;)

Please correct me if I am wrong, but I have been under the impression that Java .properties files should always use encoding ISO-8859-1. Characters not present in ISO-8859-1 can be written in \uxxxx. Won't it make things confusing to developers if another encoding is used here instead? https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... If alternate encodings are desired how about supporting the XML format of Properties? Best regards, Thomas On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > > Both iso-8859-1 and utf-8 message bundles should be able to co-exist. > > We can allow specifying the encoding in a comment on the first line like > this: > > # encoding=utf-8 > key=value > > # encoding=iso-8859-1 > key=value > > If the first line in the file doesn't contain the comment with the > encoding then we should default to iso-8859-1 for backwards compatibility > > On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >> >> Thanks for your comment. >> >> > If we want to change to utf-8 we'd still need to support iso.. for >> > backwards compatibility. >> >> If we change to UTF-8, we can still read unicode codepoint like '\u00e8'. >> There is an incompatibility when non-ascii characters are used in >> message properties. >> The non-ascii characters are 0xA0 - 0xFF codes (please refer codepage >> layout: https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >> >> I think the non-ascii characters might be used in French messages like >> 'à' so I agree to support ISO-8859-1 for backwards compatibility. >> To support this, I think we can add a property like "messageEncoding" >> in keycloak-server.json as below. Is it a good idea? >> >> "theme": { >> "staticMaxAge": 2592000, >> "cacheTemplates": true, >> "cacheThemes": true, >> "messageEncoding": "ISO-8859-1", >> "folder": { >> "dir": "${jboss.home.dir}/themes" >> } >> }, >> >> Regards, >> >> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen <sthorger(a)redhat.com&gt; >> wrote: >> > We have in the past discussed this and decided to stick with >> > ISO-8859-1. >> > That was probably not the best idea though. If we want to change to >> > utf-8 >> > we'd still need to support iso.. for backwards compatibility. >> > >> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; wrote: >> >> >> >> It makes sense, maybe file a Jira associated with: >> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >> >> >> >> On 2016-07-04, Hiroyuki Wada wrote: >> >> > Hello all, >> >> > >> >> > I am trying to translate all base theme messages to my country >> >> > language, Japanese. And I'd like to contribute them. Before that >> >> > work, >> >> > I'd like to propose about the files encoding. >> >> > >> >> > Currently, the message files (*.properties) are loaded with >> >> > ISO-8859-1 >> >> > encoding. Therefore, it is necessary to convert the files by >> >> > 'native2ascii' command beforehand. However we can directly read the >> >> > property files with UTF-8 encoding in java 1.6 or later because >> >> > 'java.util.Properties#load(java.io.Reader)' method was introduced as >> >> > below. >> >> > >> >> > >> >> > >> >> > http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >> >> > >> >> > So, my proposal is supporting the message files with UTF-8 encoding. >> >> > I >> >> > believe that it's very developers/customers friendly. In addition, >> >> > we >> >> > can easily review the translated messages on the github pull request >> >> > view and so on. What do you think? >> >> > >> >> > If it's ok, I'll create a JIRA issue and create a pull request. >> >> > >> >> > Regards, >> >> > >> >> > -- >> >> > Hiroyuki Wada, >> >> > Developer, >> >> > Nomura Research Institute, Ltd. >> >> > _______________________________________________ >> >> > keycloak-dev mailing list >> >> > keycloak-dev(a)lists.jboss.org >> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> -- >> >> >> >> abstractj >> >> PGP: 0x84DC9914 >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev

From the javadoc of java.util.ResourceBundle the encoding for .properties is still expected to be ISO-8859-1: > Constructing a PropertyResourceBundle instance from an InputStream > requires that the input stream be encoded in ISO-8859-1. In that case, > characters that cannot be represented in ISO-8859-1 encoding must be > represented by Unicode Escapes as defined in section 3.3 of The Java™ > Language Specification whereas the other constructor which takes a Reader > does not have that limitation.

https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... I understand that Spring allows one to read .properties using any encoding, but you need to specify the encoding in Spring configuration external to the .properties file. In my opinion it would cause unnecessary confusion amongst developers to use any other encoding than the one defined by the official documentation for Properties. Best regards, Thomas On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > > I think it was true before Java 1.6 because Java standard library > (java.util.ResourceBundle or java.util.Properties) only supported > ISO-8859-1 encoding file. > But they support any encoding after Java 1.6. There are some > frameworks which can read .properties with any encoding. For example, > Spring Framework can read .properties with any encoding. I think it's > very useful to read UTF-8 directly for multibyte language country. > > On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme > <thomas.raehalme(a)aitiofinland.com&gt; wrote: > > Please correct me if I am wrong, but I have been under the impression > > that > > Java .properties files should always use encoding ISO-8859-1. Characters > > not > > present in ISO-8859-1 can be written in \uxxxx. Won't it make things > > confusing to developers if another encoding is used here instead? > > > > > > https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... > > > > https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... > > > > If alternate encodings are desired how about supporting the XML format > > of > > Properties? > > > > Best regards, > > Thomas > > > > > > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; > > wrote: > >> > >> Both iso-8859-1 and utf-8 message bundles should be able to co-exist. > >> > >> We can allow specifying the encoding in a comment on the first line > >> like > >> this: > >> > >> # encoding=utf-8 > >> key=value > >> > >> # encoding=iso-8859-1 > >> key=value > >> > >> If the first line in the file doesn't contain the comment with the > >> encoding then we should default to iso-8859-1 for backwards > >> compatibility > >> > >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > >>> > >>> Thanks for your comment. > >>> > >>> > If we want to change to utf-8 we'd still need to support iso.. for > >>> > backwards compatibility. > >>> > >>> If we change to UTF-8, we can still read unicode codepoint like > >>> '\u00e8'. > >>> There is an incompatibility when non-ascii characters are used in > >>> message properties. > >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer codepage > >>> layout: https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) > >>> > >>> I think the non-ascii characters might be used in French messages like > >>> 'à' so I agree to support ISO-8859-1 for backwards compatibility. > >>> To support this, I think we can add a property like "messageEncoding" > >>> in keycloak-server.json as below. Is it a good idea? > >>> > >>> "theme": { > >>> "staticMaxAge": 2592000, > >>> "cacheTemplates": true, > >>> "cacheThemes": true, > >>> "messageEncoding": "ISO-8859-1", > >>> "folder": { > >>> "dir": "${jboss.home.dir}/themes" > >>> } > >>> }, > >>> > >>> Regards, > >>> > >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen > >>> <sthorger(a)redhat.com&gt; > >>> wrote: > >>> > We have in the past discussed this and decided to stick with > >>> > ISO-8859-1. > >>> > That was probably not the best idea though. If we want to change to > >>> > utf-8 > >>> > we'd still need to support iso.. for backwards compatibility. > >>> > > >>> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; wrote: > >>> >> > >>> >> It makes sense, maybe file a Jira associated with: > >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? > >>> >> > >>> >> On 2016-07-04, Hiroyuki Wada wrote: > >>> >> > Hello all, > >>> >> > > >>> >> > I am trying to translate all base theme messages to my country > >>> >> > language, Japanese. And I'd like to contribute them. Before that > >>> >> > work, > >>> >> > I'd like to propose about the files encoding. > >>> >> > > >>> >> > Currently, the message files (*.properties) are loaded with > >>> >> > ISO-8859-1 > >>> >> > encoding. Therefore, it is necessary to convert the files by > >>> >> > 'native2ascii' command beforehand. However we can directly read > >>> >> > the > >>> >> > property files with UTF-8 encoding in java 1.6 or later because > >>> >> > 'java.util.Properties#load(java.io.Reader)' method was introduced > >>> >> > as > >>> >> > below. > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... > >>> >> > > >>> >> > So, my proposal is supporting the message files with UTF-8 > >>> >> > encoding. > >>> >> > I > >>> >> > believe that it's very developers/customers friendly. In > >>> >> > addition, > >>> >> > we > >>> >> > can easily review the translated messages on the github pull > >>> >> > request > >>> >> > view and so on. What do you think? > >>> >> > > >>> >> > If it's ok, I'll create a JIRA issue and create a pull request. > >>> >> > > >>> >> > Regards, > >>> >> > > >>> >> > -- > >>> >> > Hiroyuki Wada, > >>> >> > Developer, > >>> >> > Nomura Research Institute, Ltd. > >>> >> > _______________________________________________ > >>> >> > keycloak-dev mailing list > >>> >> > keycloak-dev(a)lists.jboss.org > >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> >> > >>> >> -- > >>> >> > >>> >> abstractj > >>> >> PGP: 0x84DC9914 > >>> >> _______________________________________________ > >>> >> keycloak-dev mailing list > >>> >> keycloak-dev(a)lists.jboss.org > >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> > > >>> > > >> > >> > >> > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev(a)lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > >

Let's go for it. Please create a JIRA and send PR. Please just use "# encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter is just weird if you ask me. Thomas: It's optional, defaults to iso-8859-1 and the header will clearly state that a file uses a different encoding, so I disagree that there will be any confusion. On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme > <thomas.raehalme(a)aitiofinland.com&gt; wrote: > > From the javadoc of java.util.ResourceBundle the encoding for > .properties is > > still expected to be ISO-8859-1: > > > >> Constructing a PropertyResourceBundle instance from an InputStream > >> requires that the input stream be encoded in ISO-8859-1. In that case, > >> characters that cannot be represented in ISO-8859-1 encoding must be > >> represented by Unicode Escapes as defined in section 3.3 of The Java™ > >> Language Specification whereas the other constructor which takes a > Reader > >> does not have that limitation. > > It's a case of using constructor which takes an InputStream. > It says we can use any encoding when we use the other constructor > which takes a Reader. > > I understand your concern. So what do you think supporting utf-8 with > a header like Stian suggested? > I think it can avoid confusion because the encoding is noted in the > file itself... > > Regards, > > > > https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... > > > > I understand that Spring allows one to read .properties using any > encoding, > > but you need to specify the encoding in Spring configuration external > to the > > .properties file. > > > > In my opinion it would cause unnecessary confusion amongst developers > to use > > any other encoding than the one defined by the official documentation > for > > Properties. > > > > Best regards, > > Thomas > > > > > > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <wadahiro(a)gmail.com&gt; > wrote: > >> > >> I think it was true before Java 1.6 because Java standard library > >> (java.util.ResourceBundle or java.util.Properties) only supported > >> ISO-8859-1 encoding file. > >> But they support any encoding after Java 1.6. There are some > >> frameworks which can read .properties with any encoding. For example, > >> Spring Framework can read .properties with any encoding. I think it's > >> very useful to read UTF-8 directly for multibyte language country. > >> > >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme > >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: > >> > Please correct me if I am wrong, but I have been under the impression > >> > that > >> > Java .properties files should always use encoding ISO-8859-1. > Characters > >> > not > >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make things > >> > confusing to developers if another encoding is used here instead? > >> > > >> > > >> > > https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... > >> > > >> > > https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... > >> > > >> > If alternate encodings are desired how about supporting the XML > format > >> > of > >> > Properties? > >> > > >> > Best regards, > >> > Thomas > >> > > >> > > >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < > sthorger(a)redhat.com&gt; > >> > wrote: > >> >> > >> >> Both iso-8859-1 and utf-8 message bundles should be able to > co-exist. > >> >> > >> >> We can allow specifying the encoding in a comment on the first line > >> >> like > >> >> this: > >> >> > >> >> # encoding=utf-8 > >> >> key=value > >> >> > >> >> # encoding=iso-8859-1 > >> >> key=value > >> >> > >> >> If the first line in the file doesn't contain the comment with the > >> >> encoding then we should default to iso-8859-1 for backwards > >> >> compatibility > >> >> > >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > >> >>> > >> >>> Thanks for your comment. > >> >>> > >> >>> > If we want to change to utf-8 we'd still need to support iso.. > for > >> >>> > backwards compatibility. > >> >>> > >> >>> If we change to UTF-8, we can still read unicode codepoint like > >> >>> '\u00e8'. > >> >>> There is an incompatibility when non-ascii characters are used in > >> >>> message properties. > >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer > codepage > >> >>> layout: > https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) > >> >>> > >> >>> I think the non-ascii characters might be used in French messages > like > >> >>> 'à' so I agree to support ISO-8859-1 for backwards compatibility. > >> >>> To support this, I think we can add a property like > "messageEncoding" > >> >>> in keycloak-server.json as below. Is it a good idea? > >> >>> > >> >>> "theme": { > >> >>> "staticMaxAge": 2592000, > >> >>> "cacheTemplates": true, > >> >>> "cacheThemes": true, > >> >>> "messageEncoding": "ISO-8859-1", > >> >>> "folder": { > >> >>> "dir": "${jboss.home.dir}/themes" > >> >>> } > >> >>> }, > >> >>> > >> >>> Regards, > >> >>> > >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen > >> >>> <sthorger(a)redhat.com&gt; > >> >>> wrote: > >> >>> > We have in the past discussed this and decided to stick with > >> >>> > ISO-8859-1. > >> >>> > That was probably not the best idea though. If we want to change > to > >> >>> > utf-8 > >> >>> > we'd still need to support iso.. for backwards compatibility. > >> >>> > > >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; > wrote: > >> >>> >> > >> >>> >> It makes sense, maybe file a Jira associated with: > >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? > >> >>> >> > >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: > >> >>> >> > Hello all, > >> >>> >> > > >> >>> >> > I am trying to translate all base theme messages to my country > >> >>> >> > language, Japanese. And I'd like to contribute them. Before > that > >> >>> >> > work, > >> >>> >> > I'd like to propose about the files encoding. > >> >>> >> > > >> >>> >> > Currently, the message files (*.properties) are loaded with > >> >>> >> > ISO-8859-1 > >> >>> >> > encoding. Therefore, it is necessary to convert the files by > >> >>> >> > 'native2ascii' command beforehand. However we can directly > read > >> >>> >> > the > >> >>> >> > property files with UTF-8 encoding in java 1.6 or later > because > >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was > introduced > >> >>> >> > as > >> >>> >> > below. > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > > http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... > >> >>> >> > > >> >>> >> > So, my proposal is supporting the message files with UTF-8 > >> >>> >> > encoding. > >> >>> >> > I > >> >>> >> > believe that it's very developers/customers friendly. In > >> >>> >> > addition, > >> >>> >> > we > >> >>> >> > can easily review the translated messages on the github pull > >> >>> >> > request > >> >>> >> > view and so on. What do you think? > >> >>> >> > > >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull > request. > >> >>> >> > > >> >>> >> > Regards, > >> >>> >> > > >> >>> >> > -- > >> >>> >> > Hiroyuki Wada, > >> >>> >> > Developer, > >> >>> >> > Nomura Research Institute, Ltd. > >> >>> >> > _______________________________________________ > >> >>> >> > keycloak-dev mailing list > >> >>> >> > keycloak-dev(a)lists.jboss.org > >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> >>> >> > >> >>> >> -- > >> >>> >> > >> >>> >> abstractj > >> >>> >> PGP: 0x84DC9914 > >> >>> >> _______________________________________________ > >> >>> >> keycloak-dev mailing list > >> >>> >> keycloak-dev(a)lists.jboss.org > >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> >>> > > >> >>> > > >> >> > >> >> > >> >> > >> >> _______________________________________________ > >> >> keycloak-dev mailing list > >> >> keycloak-dev(a)lists.jboss.org > >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > >> > > >> > > > > > > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev >

Ok. I created a JIRA issue. https://issues.jboss.org/browse/KEYCLOAK-3278 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger(a)redhat.com&gt;: > Let's go for it. Please create a JIRA and send PR. Please just use "# > encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter is just > weird if you ask me. > > Thomas: It's optional, defaults to iso-8859-1 and the header will clearly > state that a file uses a different encoding, so I disagree that there will > be any confusion. > > On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > >> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >> > From the javadoc of java.util.ResourceBundle the encoding for >> .properties is >> > still expected to be ISO-8859-1: >> > >> >> Constructing a PropertyResourceBundle instance from an InputStream >> >> requires that the input stream be encoded in ISO-8859-1. In that case, >> >> characters that cannot be represented in ISO-8859-1 encoding must be >> >> represented by Unicode Escapes as defined in section 3.3 of The Java™ >> >> Language Specification whereas the other constructor which takes a >> Reader >> >> does not have that limitation. >> >> It's a case of using constructor which takes an InputStream. >> It says we can use any encoding when we use the other constructor >> which takes a Reader. >> >> I understand your concern. So what do you think supporting utf-8 with >> a header like Stian suggested? >> I think it can avoid confusion because the encoding is noted in the >> file itself... >> >> Regards, >> >> > >> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... >> > >> > I understand that Spring allows one to read .properties using any >> encoding, >> > but you need to specify the encoding in Spring configuration external >> to the >> > .properties file. >> > >> > In my opinion it would cause unnecessary confusion amongst developers >> to use >> > any other encoding than the one defined by the official documentation >> for >> > Properties. >> > >> > Best regards, >> > Thomas >> > >> > >> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >> wrote: >> >> >> >> I think it was true before Java 1.6 because Java standard library >> >> (java.util.ResourceBundle or java.util.Properties) only supported >> >> ISO-8859-1 encoding file. >> >> But they support any encoding after Java 1.6. There are some >> >> frameworks which can read .properties with any encoding. For example, >> >> Spring Framework can read .properties with any encoding. I think it's >> >> very useful to read UTF-8 directly for multibyte language country. >> >> >> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme >> >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >> >> > Please correct me if I am wrong, but I have been under the >> impression >> >> > that >> >> > Java .properties files should always use encoding ISO-8859-1. >> Characters >> >> > not >> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make things >> >> > confusing to developers if another encoding is used here instead? >> >> > >> >> > >> >> > >> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >> >> > >> >> > >> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >> >> > >> >> > If alternate encodings are desired how about supporting the XML >> format >> >> > of >> >> > Properties? >> >> > >> >> > Best regards, >> >> > Thomas >> >> > >> >> > >> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < >> sthorger(a)redhat.com&gt; >> >> > wrote: >> >> >> >> >> >> Both iso-8859-1 and utf-8 message bundles should be able to >> co-exist. >> >> >> >> >> >> We can allow specifying the encoding in a comment on the first line >> >> >> like >> >> >> this: >> >> >> >> >> >> # encoding=utf-8 >> >> >> key=value >> >> >> >> >> >> # encoding=iso-8859-1 >> >> >> key=value >> >> >> >> >> >> If the first line in the file doesn't contain the comment with the >> >> >> encoding then we should default to iso-8859-1 for backwards >> >> >> compatibility >> >> >> >> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >> >> >>> >> >> >>> Thanks for your comment. >> >> >>> >> >> >>> > If we want to change to utf-8 we'd still need to support iso.. >> for >> >> >>> > backwards compatibility. >> >> >>> >> >> >>> If we change to UTF-8, we can still read unicode codepoint like >> >> >>> '\u00e8'. >> >> >>> There is an incompatibility when non-ascii characters are used in >> >> >>> message properties. >> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer >> codepage >> >> >>> layout: >> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >> >> >>> >> >> >>> I think the non-ascii characters might be used in French messages >> like >> >> >>> 'à' so I agree to support ISO-8859-1 for backwards compatibility. >> >> >>> To support this, I think we can add a property like >> "messageEncoding" >> >> >>> in keycloak-server.json as below. Is it a good idea? >> >> >>> >> >> >>> "theme": { >> >> >>> "staticMaxAge": 2592000, >> >> >>> "cacheTemplates": true, >> >> >>> "cacheThemes": true, >> >> >>> "messageEncoding": "ISO-8859-1", >> >> >>> "folder": { >> >> >>> "dir": "${jboss.home.dir}/themes" >> >> >>> } >> >> >>> }, >> >> >>> >> >> >>> Regards, >> >> >>> >> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen >> >> >>> <sthorger(a)redhat.com&gt; >> >> >>> wrote: >> >> >>> > We have in the past discussed this and decided to stick with >> >> >>> > ISO-8859-1. >> >> >>> > That was probably not the best idea though. If we want to >> change to >> >> >>> > utf-8 >> >> >>> > we'd still need to support iso.. for backwards compatibility. >> >> >>> > >> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; >> wrote: >> >> >>> >> >> >> >>> >> It makes sense, maybe file a Jira associated with: >> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >> >> >>> >> >> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: >> >> >>> >> > Hello all, >> >> >>> >> > >> >> >>> >> > I am trying to translate all base theme messages to my >> country >> >> >>> >> > language, Japanese. And I'd like to contribute them. Before >> that >> >> >>> >> > work, >> >> >>> >> > I'd like to propose about the files encoding. >> >> >>> >> > >> >> >>> >> > Currently, the message files (*.properties) are loaded with >> >> >>> >> > ISO-8859-1 >> >> >>> >> > encoding. Therefore, it is necessary to convert the files by >> >> >>> >> > 'native2ascii' command beforehand. However we can directly >> read >> >> >>> >> > the >> >> >>> >> > property files with UTF-8 encoding in java 1.6 or later >> because >> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was >> introduced >> >> >>> >> > as >> >> >>> >> > below. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >> >> >>> >> > >> >> >>> >> > So, my proposal is supporting the message files with UTF-8 >> >> >>> >> > encoding. >> >> >>> >> > I >> >> >>> >> > believe that it's very developers/customers friendly. In >> >> >>> >> > addition, >> >> >>> >> > we >> >> >>> >> > can easily review the translated messages on the github pull >> >> >>> >> > request >> >> >>> >> > view and so on. What do you think? >> >> >>> >> > >> >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull >> request. >> >> >>> >> > >> >> >>> >> > Regards, >> >> >>> >> > >> >> >>> >> > -- >> >> >>> >> > Hiroyuki Wada, >> >> >>> >> > Developer, >> >> >>> >> > Nomura Research Institute, Ltd. >> >> >>> >> > _______________________________________________ >> >> >>> >> > keycloak-dev mailing list >> >> >>> >> > keycloak-dev(a)lists.jboss.org >> >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >>> >> >> >> >>> >> -- >> >> >>> >> >> >> >>> >> abstractj >> >> >>> >> PGP: 0x84DC9914 >> >> >>> >> _______________________________________________ >> >> >>> >> keycloak-dev mailing list >> >> >>> >> keycloak-dev(a)lists.jboss.org >> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >>> > >> >> >>> > >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> keycloak-dev mailing list >> >> >> keycloak-dev(a)lists.jboss.org >> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > >> >> > >> >> > >> > >> > >> > >> > >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >

Yes. I will send a pull request later! 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger(a)redhat.com&gt;: > Are you doing a PR as well? > > On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > >> Ok. I created a JIRA issue. >> https://issues.jboss.org/browse/KEYCLOAK-3278 >> >> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger(a)redhat.com&gt;: >> >>> Let's go for it. Please create a JIRA and send PR. Please just use "# >>> encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter is just >>> weird if you ask me. >>> >>> Thomas: It's optional, defaults to iso-8859-1 and the header will >>> clearly state that a file uses a different encoding, so I disagree that >>> there will be any confusion. >>> >>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>> >>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme >>>> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>> > From the javadoc of java.util.ResourceBundle the encoding for >>>> .properties is >>>> > still expected to be ISO-8859-1: >>>> > >>>> >> Constructing a PropertyResourceBundle instance from an InputStream >>>> >> requires that the input stream be encoded in ISO-8859-1. In that >>>> case, >>>> >> characters that cannot be represented in ISO-8859-1 encoding must be >>>> >> represented by Unicode Escapes as defined in section 3.3 of The >>>> Java™ >>>> >> Language Specification whereas the other constructor which takes a >>>> Reader >>>> >> does not have that limitation. >>>> >>>> It's a case of using constructor which takes an InputStream. >>>> It says we can use any encoding when we use the other constructor >>>> which takes a Reader. >>>> >>>> I understand your concern. So what do you think supporting utf-8 with >>>> a header like Stian suggested? >>>> I think it can avoid confusion because the encoding is noted in the >>>> file itself... >>>> >>>> Regards, >>>> >>>> > >>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... >>>> > >>>> > I understand that Spring allows one to read .properties using any >>>> encoding, >>>> > but you need to specify the encoding in Spring configuration >>>> external to the >>>> > .properties file. >>>> > >>>> > In my opinion it would cause unnecessary confusion amongst >>>> developers to use >>>> > any other encoding than the one defined by the official >>>> documentation for >>>> > Properties. >>>> > >>>> > Best regards, >>>> > Thomas >>>> > >>>> > >>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>> wrote: >>>> >> >>>> >> I think it was true before Java 1.6 because Java standard library >>>> >> (java.util.ResourceBundle or java.util.Properties) only supported >>>> >> ISO-8859-1 encoding file. >>>> >> But they support any encoding after Java 1.6. There are some >>>> >> frameworks which can read .properties with any encoding. For >>>> example, >>>> >> Spring Framework can read .properties with any encoding. I think >>>> it's >>>> >> very useful to read UTF-8 directly for multibyte language country. >>>> >> >>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme >>>> >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>> >> > Please correct me if I am wrong, but I have been under the >>>> impression >>>> >> > that >>>> >> > Java .properties files should always use encoding ISO-8859-1. >>>> Characters >>>> >> > not >>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make >>>> things >>>> >> > confusing to developers if another encoding is used here instead? >>>> >> > >>>> >> > >>>> >> > >>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >>>> >> > >>>> >> > >>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >>>> >> > >>>> >> > If alternate encodings are desired how about supporting the XML >>>> format >>>> >> > of >>>> >> > Properties? >>>> >> > >>>> >> > Best regards, >>>> >> > Thomas >>>> >> > >>>> >> > >>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < >>>> sthorger(a)redhat.com&gt; >>>> >> > wrote: >>>> >> >> >>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able to >>>> co-exist. >>>> >> >> >>>> >> >> We can allow specifying the encoding in a comment on the first >>>> line >>>> >> >> like >>>> >> >> this: >>>> >> >> >>>> >> >> # encoding=utf-8 >>>> >> >> key=value >>>> >> >> >>>> >> >> # encoding=iso-8859-1 >>>> >> >> key=value >>>> >> >> >>>> >> >> If the first line in the file doesn't contain the comment with >>>> the >>>> >> >> encoding then we should default to iso-8859-1 for backwards >>>> >> >> compatibility >>>> >> >> >>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>> wrote: >>>> >> >>> >>>> >> >>> Thanks for your comment. >>>> >> >>> >>>> >> >>> > If we want to change to utf-8 we'd still need to support >>>> iso.. for >>>> >> >>> > backwards compatibility. >>>> >> >>> >>>> >> >>> If we change to UTF-8, we can still read unicode codepoint like >>>> >> >>> '\u00e8'. >>>> >> >>> There is an incompatibility when non-ascii characters are used >>>> in >>>> >> >>> message properties. >>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer >>>> codepage >>>> >> >>> layout: >>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >>>> >> >>> >>>> >> >>> I think the non-ascii characters might be used in French >>>> messages like >>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards >>>> compatibility. >>>> >> >>> To support this, I think we can add a property like >>>> "messageEncoding" >>>> >> >>> in keycloak-server.json as below. Is it a good idea? >>>> >> >>> >>>> >> >>> "theme": { >>>> >> >>> "staticMaxAge": 2592000, >>>> >> >>> "cacheTemplates": true, >>>> >> >>> "cacheThemes": true, >>>> >> >>> "messageEncoding": "ISO-8859-1", >>>> >> >>> "folder": { >>>> >> >>> "dir": "${jboss.home.dir}/themes" >>>> >> >>> } >>>> >> >>> }, >>>> >> >>> >>>> >> >>> Regards, >>>> >> >>> >>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen >>>> >> >>> <sthorger(a)redhat.com&gt; >>>> >> >>> wrote: >>>> >> >>> > We have in the past discussed this and decided to stick with >>>> >> >>> > ISO-8859-1. >>>> >> >>> > That was probably not the best idea though. If we want to >>>> change to >>>> >> >>> > utf-8 >>>> >> >>> > we'd still need to support iso.. for backwards compatibility. >>>> >> >>> > >>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; >>>> wrote: >>>> >> >>> >> >>>> >> >>> >> It makes sense, maybe file a Jira associated with: >>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >>>> >> >>> >> >>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: >>>> >> >>> >> > Hello all, >>>> >> >>> >> > >>>> >> >>> >> > I am trying to translate all base theme messages to my >>>> country >>>> >> >>> >> > language, Japanese. And I'd like to contribute them. >>>> Before that >>>> >> >>> >> > work, >>>> >> >>> >> > I'd like to propose about the files encoding. >>>> >> >>> >> > >>>> >> >>> >> > Currently, the message files (*.properties) are loaded with >>>> >> >>> >> > ISO-8859-1 >>>> >> >>> >> > encoding. Therefore, it is necessary to convert the files >>>> by >>>> >> >>> >> > 'native2ascii' command beforehand. However we can directly >>>> read >>>> >> >>> >> > the >>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or later >>>> because >>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was >>>> introduced >>>> >> >>> >> > as >>>> >> >>> >> > below. >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> > >>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >>>> >> >>> >> > >>>> >> >>> >> > So, my proposal is supporting the message files with UTF-8 >>>> >> >>> >> > encoding. >>>> >> >>> >> > I >>>> >> >>> >> > believe that it's very developers/customers friendly. In >>>> >> >>> >> > addition, >>>> >> >>> >> > we >>>> >> >>> >> > can easily review the translated messages on the github >>>> pull >>>> >> >>> >> > request >>>> >> >>> >> > view and so on. What do you think? >>>> >> >>> >> > >>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull >>>> request. >>>> >> >>> >> > >>>> >> >>> >> > Regards, >>>> >> >>> >> > >>>> >> >>> >> > -- >>>> >> >>> >> > Hiroyuki Wada, >>>> >> >>> >> > Developer, >>>> >> >>> >> > Nomura Research Institute, Ltd. >>>> >> >>> >> > _______________________________________________ >>>> >> >>> >> > keycloak-dev mailing list >>>> >> >>> >> > keycloak-dev(a)lists.jboss.org >>>> >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> >>> >> >>>> >> >>> >> -- >>>> >> >>> >> >>>> >> >>> >> abstractj >>>> >> >>> >> PGP: 0x84DC9914 >>>> >> >>> >> _______________________________________________ >>>> >> >>> >> keycloak-dev mailing list >>>> >> >>> >> keycloak-dev(a)lists.jboss.org >>>> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> >>> > >>>> >> >>> > >>>> >> >> >>>> >> >> >>>> >> >> >>>> >> >> _______________________________________________ >>>> >> >> keycloak-dev mailing list >>>> >> >> keycloak-dev(a)lists.jboss.org >>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> > >>>> >> > >>>> >> > >>>> > >>>> > >>>> > >>>> > >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> >>> >

No, we don't want the theme properties (and other properties files) to be utf-8. They should stick with the default encoding used by Java properties files. Messages are a special case, which is why we accepted it for them. On 14 July 2016 at 13:29, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > Thank you for merging the pull request. Could I send a pull request > converting to UTF-8 encoding against the theme properties? > > 2016年7月7日(木) 21:39 Hiroyuki Wada <wadahiro(a)gmail.com&gt;: > >> Yes. I will send a pull request later! >> >> 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger(a)redhat.com&gt;: >> >>> Are you doing a PR as well? >>> >>> On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>> >>>> Ok. I created a JIRA issue. >>>> https://issues.jboss.org/browse/KEYCLOAK-3278 >>>> >>>> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>> >>>>> Let's go for it. Please create a JIRA and send PR. Please just use "# >>>>> encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter is >>>>> just weird if you ask me. >>>>> >>>>> Thomas: It's optional, defaults to iso-8859-1 and the header will >>>>> clearly state that a file uses a different encoding, so I disagree that >>>>> there will be any confusion. >>>>> >>>>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>>> >>>>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme >>>>>> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>> > From the javadoc of java.util.ResourceBundle the encoding for >>>>>> .properties is >>>>>> > still expected to be ISO-8859-1: >>>>>> > >>>>>> >> Constructing a PropertyResourceBundle instance from an InputStream >>>>>> >> requires that the input stream be encoded in ISO-8859-1. In that >>>>>> case, >>>>>> >> characters that cannot be represented in ISO-8859-1 encoding must >>>>>> be >>>>>> >> represented by Unicode Escapes as defined in section 3.3 of The >>>>>> Java™ >>>>>> >> Language Specification whereas the other constructor which takes >>>>>> a Reader >>>>>> >> does not have that limitation. >>>>>> >>>>>> It's a case of using constructor which takes an InputStream. >>>>>> It says we can use any encoding when we use the other constructor >>>>>> which takes a Reader. >>>>>> >>>>>> I understand your concern. So what do you think supporting utf-8 with >>>>>> a header like Stian suggested? >>>>>> I think it can avoid confusion because the encoding is noted in the >>>>>> file itself... >>>>>> >>>>>> Regards, >>>>>> >>>>>> > >>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... >>>>>> > >>>>>> > I understand that Spring allows one to read .properties using any >>>>>> encoding, >>>>>> > but you need to specify the encoding in Spring configuration >>>>>> external to the >>>>>> > .properties file. >>>>>> > >>>>>> > In my opinion it would cause unnecessary confusion amongst >>>>>> developers to use >>>>>> > any other encoding than the one defined by the official >>>>>> documentation for >>>>>> > Properties. >>>>>> > >>>>>> > Best regards, >>>>>> > Thomas >>>>>> > >>>>>> > >>>>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>>>> wrote: >>>>>> >> >>>>>> >> I think it was true before Java 1.6 because Java standard library >>>>>> >> (java.util.ResourceBundle or java.util.Properties) only supported >>>>>> >> ISO-8859-1 encoding file. >>>>>> >> But they support any encoding after Java 1.6. There are some >>>>>> >> frameworks which can read .properties with any encoding. For >>>>>> example, >>>>>> >> Spring Framework can read .properties with any encoding. I think >>>>>> it's >>>>>> >> very useful to read UTF-8 directly for multibyte language country. >>>>>> >> >>>>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme >>>>>> >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>> >> > Please correct me if I am wrong, but I have been under the >>>>>> impression >>>>>> >> > that >>>>>> >> > Java .properties files should always use encoding ISO-8859-1. >>>>>> Characters >>>>>> >> > not >>>>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make >>>>>> things >>>>>> >> > confusing to developers if another encoding is used here >>>>>> instead? >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >>>>>> >> > >>>>>> >> > >>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >>>>>> >> > >>>>>> >> > If alternate encodings are desired how about supporting the XML >>>>>> format >>>>>> >> > of >>>>>> >> > Properties? >>>>>> >> > >>>>>> >> > Best regards, >>>>>> >> > Thomas >>>>>> >> > >>>>>> >> > >>>>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < >>>>>> sthorger(a)redhat.com&gt; >>>>>> >> > wrote: >>>>>> >> >> >>>>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able to >>>>>> co-exist. >>>>>> >> >> >>>>>> >> >> We can allow specifying the encoding in a comment on the first >>>>>> line >>>>>> >> >> like >>>>>> >> >> this: >>>>>> >> >> >>>>>> >> >> # encoding=utf-8 >>>>>> >> >> key=value >>>>>> >> >> >>>>>> >> >> # encoding=iso-8859-1 >>>>>> >> >> key=value >>>>>> >> >> >>>>>> >> >> If the first line in the file doesn't contain the comment with >>>>>> the >>>>>> >> >> encoding then we should default to iso-8859-1 for backwards >>>>>> >> >> compatibility >>>>>> >> >> >>>>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>>>> wrote: >>>>>> >> >>> >>>>>> >> >>> Thanks for your comment. >>>>>> >> >>> >>>>>> >> >>> > If we want to change to utf-8 we'd still need to support >>>>>> iso.. for >>>>>> >> >>> > backwards compatibility. >>>>>> >> >>> >>>>>> >> >>> If we change to UTF-8, we can still read unicode codepoint >>>>>> like >>>>>> >> >>> '\u00e8'. >>>>>> >> >>> There is an incompatibility when non-ascii characters are >>>>>> used in >>>>>> >> >>> message properties. >>>>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer >>>>>> codepage >>>>>> >> >>> layout: >>>>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >>>>>> >> >>> >>>>>> >> >>> I think the non-ascii characters might be used in French >>>>>> messages like >>>>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards >>>>>> compatibility. >>>>>> >> >>> To support this, I think we can add a property like >>>>>> "messageEncoding" >>>>>> >> >>> in keycloak-server.json as below. Is it a good idea? >>>>>> >> >>> >>>>>> >> >>> "theme": { >>>>>> >> >>> "staticMaxAge": 2592000, >>>>>> >> >>> "cacheTemplates": true, >>>>>> >> >>> "cacheThemes": true, >>>>>> >> >>> "messageEncoding": "ISO-8859-1", >>>>>> >> >>> "folder": { >>>>>> >> >>> "dir": "${jboss.home.dir}/themes" >>>>>> >> >>> } >>>>>> >> >>> }, >>>>>> >> >>> >>>>>> >> >>> Regards, >>>>>> >> >>> >>>>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen >>>>>> >> >>> <sthorger(a)redhat.com&gt; >>>>>> >> >>> wrote: >>>>>> >> >>> > We have in the past discussed this and decided to stick with >>>>>> >> >>> > ISO-8859-1. >>>>>> >> >>> > That was probably not the best idea though. If we want to >>>>>> change to >>>>>> >> >>> > utf-8 >>>>>> >> >>> > we'd still need to support iso.. for backwards >>>>>> compatibility. >>>>>> >> >>> > >>>>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira < >>>>>> bruno(a)abstractj.org&gt; wrote: >>>>>> >> >>> >> >>>>>> >> >>> >> It makes sense, maybe file a Jira associated with: >>>>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >>>>>> >> >>> >> >>>>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: >>>>>> >> >>> >> > Hello all, >>>>>> >> >>> >> > >>>>>> >> >>> >> > I am trying to translate all base theme messages to my >>>>>> country >>>>>> >> >>> >> > language, Japanese. And I'd like to contribute them. >>>>>> Before that >>>>>> >> >>> >> > work, >>>>>> >> >>> >> > I'd like to propose about the files encoding. >>>>>> >> >>> >> > >>>>>> >> >>> >> > Currently, the message files (*.properties) are loaded >>>>>> with >>>>>> >> >>> >> > ISO-8859-1 >>>>>> >> >>> >> > encoding. Therefore, it is necessary to convert the >>>>>> files by >>>>>> >> >>> >> > 'native2ascii' command beforehand. However we can >>>>>> directly read >>>>>> >> >>> >> > the >>>>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or later >>>>>> because >>>>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was >>>>>> introduced >>>>>> >> >>> >> > as >>>>>> >> >>> >> > below. >>>>>> >> >>> >> > >>>>>> >> >>> >> > >>>>>> >> >>> >> > >>>>>> >> >>> >> > >>>>>> >> >>> >> > >>>>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >>>>>> >> >>> >> > >>>>>> >> >>> >> > So, my proposal is supporting the message files with >>>>>> UTF-8 >>>>>> >> >>> >> > encoding. >>>>>> >> >>> >> > I >>>>>> >> >>> >> > believe that it's very developers/customers friendly. In >>>>>> >> >>> >> > addition, >>>>>> >> >>> >> > we >>>>>> >> >>> >> > can easily review the translated messages on the github >>>>>> pull >>>>>> >> >>> >> > request >>>>>> >> >>> >> > view and so on. What do you think? >>>>>> >> >>> >> > >>>>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull >>>>>> request. >>>>>> >> >>> >> > >>>>>> >> >>> >> > Regards, >>>>>> >> >>> >> > >>>>>> >> >>> >> > -- >>>>>> >> >>> >> > Hiroyuki Wada, >>>>>> >> >>> >> > Developer, >>>>>> >> >>> >> > Nomura Research Institute, Ltd. >>>>>> >> >>> >> > _______________________________________________ >>>>>> >> >>> >> > keycloak-dev mailing list >>>>>> >> >>> >> > keycloak-dev(a)lists.jboss.org >>>>>> >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >> >>> >> >>>>>> >> >>> >> -- >>>>>> >> >>> >> >>>>>> >> >>> >> abstractj >>>>>> >> >>> >> PGP: 0x84DC9914 >>>>>> >> >>> >> _______________________________________________ >>>>>> >> >>> >> keycloak-dev mailing list >>>>>> >> >>> >> keycloak-dev(a)lists.jboss.org >>>>>> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >> >>> > >>>>>> >> >>> > >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> _______________________________________________ >>>>>> >> >> keycloak-dev mailing list >>>>>> >> >> keycloak-dev(a)lists.jboss.org >>>>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >>>>> >>>>> >>>

I'd like the current messages to be converted by the people that have contributed them. They will be able to determine what encoding they prefer to use and also confirm that the conversion has been successful. On 15 July 2016 at 07:59, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > I mean I'd like to change the message files encoding only. > For example, admin-messages for Russian language is currently written in > ISO-8859-1 encoding. > > > https://github.com/keycloak/keycloak/blob/master/themes/src/main/resource... > > Can I convert the language messages except for English? > > > 2016年7月15日(金) 14:03 Stian Thorgersen <sthorger(a)redhat.com&gt;: > >> No, we don't want the theme properties (and other properties files) to >> be utf-8. They should stick with the default encoding used by Java >> properties files. >> >> Messages are a special case, which is why we accepted it for them. >> >> On 14 July 2016 at 13:29, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >> >>> Thank you for merging the pull request. Could I send a pull request >>> converting to UTF-8 encoding against the theme properties? >>> >>> 2016年7月7日(木) 21:39 Hiroyuki Wada <wadahiro(a)gmail.com&gt;: >>> >>>> Yes. I will send a pull request later! >>>> >>>> 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>> >>>>> Are you doing a PR as well? >>>>> >>>>> On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>>> >>>>>> Ok. I created a JIRA issue. >>>>>> https://issues.jboss.org/browse/KEYCLOAK-3278 >>>>>> >>>>>> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>>>> >>>>>>> Let's go for it. Please create a JIRA and send PR. Please just use >>>>>>> "# encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter >>>>>>> is just weird if you ask me. >>>>>>> >>>>>>> Thomas: It's optional, defaults to iso-8859-1 and the header will >>>>>>> clearly state that a file uses a different encoding, so I disagree that >>>>>>> there will be any confusion. >>>>>>> >>>>>>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>>>>> >>>>>>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme >>>>>>>> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>>>> > From the javadoc of java.util.ResourceBundle the encoding for >>>>>>>> .properties is >>>>>>>> > still expected to be ISO-8859-1: >>>>>>>> > >>>>>>>> >> Constructing a PropertyResourceBundle instance from an >>>>>>>> InputStream >>>>>>>> >> requires that the input stream be encoded in ISO-8859-1. In >>>>>>>> that case, >>>>>>>> >> characters that cannot be represented in ISO-8859-1 encoding >>>>>>>> must be >>>>>>>> >> represented by Unicode Escapes as defined in section 3.3 of The >>>>>>>> Java™ >>>>>>>> >> Language Specification whereas the other constructor which >>>>>>>> takes a Reader >>>>>>>> >> does not have that limitation. >>>>>>>> >>>>>>>> It's a case of using constructor which takes an InputStream. >>>>>>>> It says we can use any encoding when we use the other constructor >>>>>>>> which takes a Reader. >>>>>>>> >>>>>>>> I understand your concern. So what do you think supporting utf-8 >>>>>>>> with >>>>>>>> a header like Stian suggested? >>>>>>>> I think it can avoid confusion because the encoding is noted in the >>>>>>>> file itself... >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> > >>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... >>>>>>>> > >>>>>>>> > I understand that Spring allows one to read .properties using >>>>>>>> any encoding, >>>>>>>> > but you need to specify the encoding in Spring configuration >>>>>>>> external to the >>>>>>>> > .properties file. >>>>>>>> > >>>>>>>> > In my opinion it would cause unnecessary confusion amongst >>>>>>>> developers to use >>>>>>>> > any other encoding than the one defined by the official >>>>>>>> documentation for >>>>>>>> > Properties. >>>>>>>> > >>>>>>>> > Best regards, >>>>>>>> > Thomas >>>>>>>> > >>>>>>>> > >>>>>>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada < >>>>>>>> wadahiro(a)gmail.com&gt; wrote: >>>>>>>> >> >>>>>>>> >> I think it was true before Java 1.6 because Java standard >>>>>>>> library >>>>>>>> >> (java.util.ResourceBundle or java.util.Properties) only >>>>>>>> supported >>>>>>>> >> ISO-8859-1 encoding file. >>>>>>>> >> But they support any encoding after Java 1.6. There are some >>>>>>>> >> frameworks which can read .properties with any encoding. For >>>>>>>> example, >>>>>>>> >> Spring Framework can read .properties with any encoding. I >>>>>>>> think it's >>>>>>>> >> very useful to read UTF-8 directly for multibyte language >>>>>>>> country. >>>>>>>> >> >>>>>>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme >>>>>>>> >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>>>> >> > Please correct me if I am wrong, but I have been under the >>>>>>>> impression >>>>>>>> >> > that >>>>>>>> >> > Java .properties files should always use encoding ISO-8859-1. >>>>>>>> Characters >>>>>>>> >> > not >>>>>>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make >>>>>>>> things >>>>>>>> >> > confusing to developers if another encoding is used here >>>>>>>> instead? >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >>>>>>>> >> > >>>>>>>> >> > If alternate encodings are desired how about supporting the >>>>>>>> XML format >>>>>>>> >> > of >>>>>>>> >> > Properties? >>>>>>>> >> > >>>>>>>> >> > Best regards, >>>>>>>> >> > Thomas >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < >>>>>>>> sthorger(a)redhat.com&gt; >>>>>>>> >> > wrote: >>>>>>>> >> >> >>>>>>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able to >>>>>>>> co-exist. >>>>>>>> >> >> >>>>>>>> >> >> We can allow specifying the encoding in a comment on the >>>>>>>> first line >>>>>>>> >> >> like >>>>>>>> >> >> this: >>>>>>>> >> >> >>>>>>>> >> >> # encoding=utf-8 >>>>>>>> >> >> key=value >>>>>>>> >> >> >>>>>>>> >> >> # encoding=iso-8859-1 >>>>>>>> >> >> key=value >>>>>>>> >> >> >>>>>>>> >> >> If the first line in the file doesn't contain the comment >>>>>>>> with the >>>>>>>> >> >> encoding then we should default to iso-8859-1 for backwards >>>>>>>> >> >> compatibility >>>>>>>> >> >> >>>>>>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>>>>>> wrote: >>>>>>>> >> >>> >>>>>>>> >> >>> Thanks for your comment. >>>>>>>> >> >>> >>>>>>>> >> >>> > If we want to change to utf-8 we'd still need to support >>>>>>>> iso.. for >>>>>>>> >> >>> > backwards compatibility. >>>>>>>> >> >>> >>>>>>>> >> >>> If we change to UTF-8, we can still read unicode codepoint >>>>>>>> like >>>>>>>> >> >>> '\u00e8'. >>>>>>>> >> >>> There is an incompatibility when non-ascii characters are >>>>>>>> used in >>>>>>>> >> >>> message properties. >>>>>>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please >>>>>>>> refer codepage >>>>>>>> >> >>> layout: >>>>>>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >>>>>>>> >> >>> >>>>>>>> >> >>> I think the non-ascii characters might be used in French >>>>>>>> messages like >>>>>>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards >>>>>>>> compatibility. >>>>>>>> >> >>> To support this, I think we can add a property like >>>>>>>> "messageEncoding" >>>>>>>> >> >>> in keycloak-server.json as below. Is it a good idea? >>>>>>>> >> >>> >>>>>>>> >> >>> "theme": { >>>>>>>> >> >>> "staticMaxAge": 2592000, >>>>>>>> >> >>> "cacheTemplates": true, >>>>>>>> >> >>> "cacheThemes": true, >>>>>>>> >> >>> "messageEncoding": "ISO-8859-1", >>>>>>>> >> >>> "folder": { >>>>>>>> >> >>> "dir": "${jboss.home.dir}/themes" >>>>>>>> >> >>> } >>>>>>>> >> >>> }, >>>>>>>> >> >>> >>>>>>>> >> >>> Regards, >>>>>>>> >> >>> >>>>>>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen >>>>>>>> >> >>> <sthorger(a)redhat.com&gt; >>>>>>>> >> >>> wrote: >>>>>>>> >> >>> > We have in the past discussed this and decided to stick >>>>>>>> with >>>>>>>> >> >>> > ISO-8859-1. >>>>>>>> >> >>> > That was probably not the best idea though. If we want to >>>>>>>> change to >>>>>>>> >> >>> > utf-8 >>>>>>>> >> >>> > we'd still need to support iso.. for backwards >>>>>>>> compatibility. >>>>>>>> >> >>> > >>>>>>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira < >>>>>>>> bruno(a)abstractj.org&gt; wrote: >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> It makes sense, maybe file a Jira associated with: >>>>>>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: >>>>>>>> >> >>> >> > Hello all, >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > I am trying to translate all base theme messages to my >>>>>>>> country >>>>>>>> >> >>> >> > language, Japanese. And I'd like to contribute them. >>>>>>>> Before that >>>>>>>> >> >>> >> > work, >>>>>>>> >> >>> >> > I'd like to propose about the files encoding. >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > Currently, the message files (*.properties) are loaded >>>>>>>> with >>>>>>>> >> >>> >> > ISO-8859-1 >>>>>>>> >> >>> >> > encoding. Therefore, it is necessary to convert the >>>>>>>> files by >>>>>>>> >> >>> >> > 'native2ascii' command beforehand. However we can >>>>>>>> directly read >>>>>>>> >> >>> >> > the >>>>>>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or >>>>>>>> later because >>>>>>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was >>>>>>>> introduced >>>>>>>> >> >>> >> > as >>>>>>>> >> >>> >> > below. >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > >>>>>>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > So, my proposal is supporting the message files with >>>>>>>> UTF-8 >>>>>>>> >> >>> >> > encoding. >>>>>>>> >> >>> >> > I >>>>>>>> >> >>> >> > believe that it's very developers/customers friendly. >>>>>>>> In >>>>>>>> >> >>> >> > addition, >>>>>>>> >> >>> >> > we >>>>>>>> >> >>> >> > can easily review the translated messages on the >>>>>>>> github pull >>>>>>>> >> >>> >> > request >>>>>>>> >> >>> >> > view and so on. What do you think? >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull >>>>>>>> request. >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > Regards, >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > -- >>>>>>>> >> >>> >> > Hiroyuki Wada, >>>>>>>> >> >>> >> > Developer, >>>>>>>> >> >>> >> > Nomura Research Institute, Ltd. >>>>>>>> >> >>> >> > _______________________________________________ >>>>>>>> >> >>> >> > keycloak-dev mailing list >>>>>>>> >> >>> >> > keycloak-dev(a)lists.jboss.org >>>>>>>> >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> -- >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> abstractj >>>>>>>> >> >>> >> PGP: 0x84DC9914 >>>>>>>> >> >>> >> _______________________________________________ >>>>>>>> >> >>> >> keycloak-dev mailing list >>>>>>>> >> >>> >> keycloak-dev(a)lists.jboss.org >>>>>>>> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>> >> >>> > >>>>>>>> >> >>> > >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> _______________________________________________ >>>>>>>> >> >> keycloak-dev mailing list >>>>>>>> >> >> keycloak-dev(a)lists.jboss.org >>>>>>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> keycloak-dev mailing list >>>>>>>> keycloak-dev(a)lists.jboss.org >>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>> >>>>>>> >>>>>>> >>>>> >>

BTW do you have a reliable way to convert the files? I've tried various > tools, but none worked for me. > Yes, we can use native2ascii command. Example: native2ascii -encoding UTF-8 -reverse <input> <output> > On 15 July 2016 at 08:14, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: > >> I'd like the current messages to be converted by the people that have >> contributed them. They will be able to determine what encoding they prefer >> to use and also confirm that the conversion has been successful. >> >> On 15 July 2016 at 07:59, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >> >>> I mean I'd like to change the message files encoding only. >>> For example, admin-messages for Russian language is currently written >>> in ISO-8859-1 encoding. >>> >>> >>> https://github.com/keycloak/keycloak/blob/master/themes/src/main/resource... >>> >>> Can I convert the language messages except for English? >>> >>> >>> 2016年7月15日(金) 14:03 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>> >>>> No, we don't want the theme properties (and other properties files) to >>>> be utf-8. They should stick with the default encoding used by Java >>>> properties files. >>>> >>>> Messages are a special case, which is why we accepted it for them. >>>> >>>> On 14 July 2016 at 13:29, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>> >>>>> Thank you for merging the pull request. Could I send a pull request >>>>> converting to UTF-8 encoding against the theme properties? >>>>> >>>>> 2016年7月7日(木) 21:39 Hiroyuki Wada <wadahiro(a)gmail.com&gt;: >>>>> >>>>>> Yes. I will send a pull request later! >>>>>> >>>>>> 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>>>> >>>>>>> Are you doing a PR as well? >>>>>>> >>>>>>> On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>>>>> >>>>>>>> Ok. I created a JIRA issue. >>>>>>>> https://issues.jboss.org/browse/KEYCLOAK-3278 >>>>>>>> >>>>>>>> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>>>>>> >>>>>>>>> Let's go for it. Please create a JIRA and send PR. Please just >>>>>>>>> use "# encoding: utf-8" and not "# -*- coding: utf-8 -*-". The >>>>>>>>> latter is just weird if you ask me. >>>>>>>>> >>>>>>>>> Thomas: It's optional, defaults to iso-8859-1 and the header will >>>>>>>>> clearly state that a file uses a different encoding, so I disagree that >>>>>>>>> there will be any confusion. >>>>>>>>> >>>>>>>>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme >>>>>>>>>> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>>>>>> > From the javadoc of java.util.ResourceBundle the encoding for >>>>>>>>>> .properties is >>>>>>>>>> > still expected to be ISO-8859-1: >>>>>>>>>> > >>>>>>>>>> >> Constructing a PropertyResourceBundle instance from an >>>>>>>>>> InputStream >>>>>>>>>> >> requires that the input stream be encoded in ISO-8859-1. In >>>>>>>>>> that case, >>>>>>>>>> >> characters that cannot be represented in ISO-8859-1 encoding >>>>>>>>>> must be >>>>>>>>>> >> represented by Unicode Escapes as defined in section 3.3 of >>>>>>>>>> The Java™ >>>>>>>>>> >> Language Specification whereas the other constructor which >>>>>>>>>> takes a Reader >>>>>>>>>> >> does not have that limitation. >>>>>>>>>> >>>>>>>>>> It's a case of using constructor which takes an InputStream. >>>>>>>>>> It says we can use any encoding when we use the other constructor >>>>>>>>>> which takes a Reader. >>>>>>>>>> >>>>>>>>>> I understand your concern. So what do you think supporting utf-8 >>>>>>>>>> with >>>>>>>>>> a header like Stian suggested? >>>>>>>>>> I think it can avoid confusion because the encoding is noted in >>>>>>>>>> the >>>>>>>>>> file itself... >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> > >>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... >>>>>>>>>> > >>>>>>>>>> > I understand that Spring allows one to read .properties using >>>>>>>>>> any encoding, >>>>>>>>>> > but you need to specify the encoding in Spring configuration >>>>>>>>>> external to the >>>>>>>>>> > .properties file. >>>>>>>>>> > >>>>>>>>>> > In my opinion it would cause unnecessary confusion amongst >>>>>>>>>> developers to use >>>>>>>>>> > any other encoding than the one defined by the official >>>>>>>>>> documentation for >>>>>>>>>> > Properties. >>>>>>>>>> > >>>>>>>>>> > Best regards, >>>>>>>>>> > Thomas >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada < >>>>>>>>>> wadahiro(a)gmail.com&gt; wrote: >>>>>>>>>> >> >>>>>>>>>> >> I think it was true before Java 1.6 because Java standard >>>>>>>>>> library >>>>>>>>>> >> (java.util.ResourceBundle or java.util.Properties) only >>>>>>>>>> supported >>>>>>>>>> >> ISO-8859-1 encoding file. >>>>>>>>>> >> But they support any encoding after Java 1.6. There are some >>>>>>>>>> >> frameworks which can read .properties with any encoding. For >>>>>>>>>> example, >>>>>>>>>> >> Spring Framework can read .properties with any encoding. I >>>>>>>>>> think it's >>>>>>>>>> >> very useful to read UTF-8 directly for multibyte language >>>>>>>>>> country. >>>>>>>>>> >> >>>>>>>>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme >>>>>>>>>> >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>>>>>> >> > Please correct me if I am wrong, but I have been under the >>>>>>>>>> impression >>>>>>>>>> >> > that >>>>>>>>>> >> > Java .properties files should always use encoding >>>>>>>>>> ISO-8859-1. Characters >>>>>>>>>> >> > not >>>>>>>>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it >>>>>>>>>> make things >>>>>>>>>> >> > confusing to developers if another encoding is used here >>>>>>>>>> instead? >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >>>>>>>>>> >> > >>>>>>>>>> >> > If alternate encodings are desired how about supporting the >>>>>>>>>> XML format >>>>>>>>>> >> > of >>>>>>>>>> >> > Properties? >>>>>>>>>> >> > >>>>>>>>>> >> > Best regards, >>>>>>>>>> >> > Thomas >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < >>>>>>>>>> sthorger(a)redhat.com&gt; >>>>>>>>>> >> > wrote: >>>>>>>>>> >> >> >>>>>>>>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able >>>>>>>>>> to co-exist. >>>>>>>>>> >> >> >>>>>>>>>> >> >> We can allow specifying the encoding in a comment on the >>>>>>>>>> first line >>>>>>>>>> >> >> like >>>>>>>>>> >> >> this: >>>>>>>>>> >> >> >>>>>>>>>> >> >> # encoding=utf-8 >>>>>>>>>> >> >> key=value >>>>>>>>>> >> >> >>>>>>>>>> >> >> # encoding=iso-8859-1 >>>>>>>>>> >> >> key=value >>>>>>>>>> >> >> >>>>>>>>>> >> >> If the first line in the file doesn't contain the comment >>>>>>>>>> with the >>>>>>>>>> >> >> encoding then we should default to iso-8859-1 for backwards >>>>>>>>>> >> >> compatibility >>>>>>>>>> >> >> >>>>>>>>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>>>>>>>> wrote: >>>>>>>>>> >> >>> >>>>>>>>>> >> >>> Thanks for your comment. >>>>>>>>>> >> >>> >>>>>>>>>> >> >>> > If we want to change to utf-8 we'd still need to >>>>>>>>>> support iso.. for >>>>>>>>>> >> >>> > backwards compatibility. >>>>>>>>>> >> >>> >>>>>>>>>> >> >>> If we change to UTF-8, we can still read unicode >>>>>>>>>> codepoint like >>>>>>>>>> >> >>> '\u00e8'. >>>>>>>>>> >> >>> There is an incompatibility when non-ascii characters are >>>>>>>>>> used in >>>>>>>>>> >> >>> message properties. >>>>>>>>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please >>>>>>>>>> refer codepage >>>>>>>>>> >> >>> layout: >>>>>>>>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >>>>>>>>>> >> >>> >>>>>>>>>> >> >>> I think the non-ascii characters might be used in French >>>>>>>>>> messages like >>>>>>>>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards >>>>>>>>>> compatibility. >>>>>>>>>> >> >>> To support this, I think we can add a property like >>>>>>>>>> "messageEncoding" >>>>>>>>>> >> >>> in keycloak-server.json as below. Is it a good idea? >>>>>>>>>> >> >>> >>>>>>>>>> >> >>> "theme": { >>>>>>>>>> >> >>> "staticMaxAge": 2592000, >>>>>>>>>> >> >>> "cacheTemplates": true, >>>>>>>>>> >> >>> "cacheThemes": true, >>>>>>>>>> >> >>> "messageEncoding": "ISO-8859-1", >>>>>>>>>> >> >>> "folder": { >>>>>>>>>> >> >>> "dir": "${jboss.home.dir}/themes" >>>>>>>>>> >> >>> } >>>>>>>>>> >> >>> }, >>>>>>>>>> >> >>> >>>>>>>>>> >> >>> Regards, >>>>>>>>>> >> >>> >>>>>>>>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen >>>>>>>>>> >> >>> <sthorger(a)redhat.com&gt; >>>>>>>>>> >> >>> wrote: >>>>>>>>>> >> >>> > We have in the past discussed this and decided to stick >>>>>>>>>> with >>>>>>>>>> >> >>> > ISO-8859-1. >>>>>>>>>> >> >>> > That was probably not the best idea though. If we want >>>>>>>>>> to change to >>>>>>>>>> >> >>> > utf-8 >>>>>>>>>> >> >>> > we'd still need to support iso.. for backwards >>>>>>>>>> compatibility. >>>>>>>>>> >> >>> > >>>>>>>>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira < >>>>>>>>>> bruno(a)abstractj.org&gt; wrote: >>>>>>>>>> >> >>> >> >>>>>>>>>> >> >>> >> It makes sense, maybe file a Jira associated with: >>>>>>>>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >>>>>>>>>> >> >>> >> >>>>>>>>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: >>>>>>>>>> >> >>> >> > Hello all, >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > I am trying to translate all base theme messages to >>>>>>>>>> my country >>>>>>>>>> >> >>> >> > language, Japanese. And I'd like to contribute them. >>>>>>>>>> Before that >>>>>>>>>> >> >>> >> > work, >>>>>>>>>> >> >>> >> > I'd like to propose about the files encoding. >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > Currently, the message files (*.properties) are >>>>>>>>>> loaded with >>>>>>>>>> >> >>> >> > ISO-8859-1 >>>>>>>>>> >> >>> >> > encoding. Therefore, it is necessary to convert the >>>>>>>>>> files by >>>>>>>>>> >> >>> >> > 'native2ascii' command beforehand. However we can >>>>>>>>>> directly read >>>>>>>>>> >> >>> >> > the >>>>>>>>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or >>>>>>>>>> later because >>>>>>>>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method >>>>>>>>>> was introduced >>>>>>>>>> >> >>> >> > as >>>>>>>>>> >> >>> >> > below. >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > >>>>>>>>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > So, my proposal is supporting the message files with >>>>>>>>>> UTF-8 >>>>>>>>>> >> >>> >> > encoding. >>>>>>>>>> >> >>> >> > I >>>>>>>>>> >> >>> >> > believe that it's very developers/customers >>>>>>>>>> friendly. In >>>>>>>>>> >> >>> >> > addition, >>>>>>>>>> >> >>> >> > we >>>>>>>>>> >> >>> >> > can easily review the translated messages on the >>>>>>>>>> github pull >>>>>>>>>> >> >>> >> > request >>>>>>>>>> >> >>> >> > view and so on. What do you think? >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a >>>>>>>>>> pull request. >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > Regards, >>>>>>>>>> >> >>> >> > >>>>>>>>>> >> >>> >> > -- >>>>>>>>>> >> >>> >> > Hiroyuki Wada, >>>>>>>>>> >> >>> >> > Developer, >>>>>>>>>> >> >>> >> > Nomura Research Institute, Ltd. >>>>>>>>>> >> >>> >> > _______________________________________________ >>>>>>>>>> >> >>> >> > keycloak-dev mailing list >>>>>>>>>> >> >>> >> > keycloak-dev(a)lists.jboss.org >>>>>>>>>> >> >>> >> > >>>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>>>> >> >>> >> >>>>>>>>>> >> >>> >> -- >>>>>>>>>> >> >>> >> >>>>>>>>>> >> >>> >> abstractj >>>>>>>>>> >> >>> >> PGP: 0x84DC9914 >>>>>>>>>> >> >>> >> _______________________________________________ >>>>>>>>>> >> >>> >> keycloak-dev mailing list >>>>>>>>>> >> >>> >> keycloak-dev(a)lists.jboss.org >>>>>>>>>> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>>>> >> >>> > >>>>>>>>>> >> >>> > >>>>>>>>>> >> >> >>>>>>>>>> >> >> >>>>>>>>>> >> >> >>>>>>>>>> >> >> _______________________________________________ >>>>>>>>>> >> >> keycloak-dev mailing list >>>>>>>>>> >> >> keycloak-dev(a)lists.jboss.org >>>>>>>>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> keycloak-dev mailing list >>>>>>>>>> keycloak-dev(a)lists.jboss.org >>>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>> >> > _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev

I tried iconv, but it doesn't do anything. Just outputs the iso-8859-1 as before: iconv -f iso-8859-1 -t utf-8 ./themes/src/main/resources/theme/base/login/messages/messages_ru.properties On 15 July 2016 at 09:04, Thomas Raehalme < thomas.raehalme(a)aitiofinland.com&gt; wrote: > iconv has been a good tool to convert files from one encoding to another. > > Best regards, > Thomas > > On Jul 15, 2016 09:57, "Hiroyuki Wada" <wadahiro(a)gmail.com&gt; wrote: > >> >> BTW do you have a reliable way to convert the files? I've tried various >>> tools, but none worked for me. >>> >> >> Yes, we can use native2ascii command. >> >> Example: >> native2ascii -encoding UTF-8 -reverse <input> <output> >> >> >> >> >>> On 15 July 2016 at 08:14, Stian Thorgersen <sthorger(a)redhat.com&gt; wrote: >>> >>>> I'd like the current messages to be converted by the people that have >>>> contributed them. They will be able to determine what encoding they prefer >>>> to use and also confirm that the conversion has been successful. >>>> >>>> On 15 July 2016 at 07:59, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>> >>>>> I mean I'd like to change the message files encoding only. >>>>> For example, admin-messages for Russian language is currently written >>>>> in ISO-8859-1 encoding. >>>>> >>>>> >>>>> https://github.com/keycloak/keycloak/blob/master/themes/src/main/resource... >>>>> >>>>> Can I convert the language messages except for English? >>>>> >>>>> >>>>> 2016年7月15日(金) 14:03 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>>> >>>>>> No, we don't want the theme properties (and other properties files) >>>>>> to be utf-8. They should stick with the default encoding used by Java >>>>>> properties files. >>>>>> >>>>>> Messages are a special case, which is why we accepted it for them. >>>>>> >>>>>> On 14 July 2016 at 13:29, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>>>> >>>>>>> Thank you for merging the pull request. Could I send a pull request >>>>>>> converting to UTF-8 encoding against the theme properties? >>>>>>> >>>>>>> 2016年7月7日(木) 21:39 Hiroyuki Wada <wadahiro(a)gmail.com&gt;: >>>>>>> >>>>>>>> Yes. I will send a pull request later! >>>>>>>> >>>>>>>> 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>>>>>> >>>>>>>>> Are you doing a PR as well? >>>>>>>>> >>>>>>>>> On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Ok. I created a JIRA issue. >>>>>>>>>> https://issues.jboss.org/browse/KEYCLOAK-3278 >>>>>>>>>> >>>>>>>>>> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>>>>>>>> >>>>>>>>>>> Let's go for it. Please create a JIRA and send PR. Please just >>>>>>>>>>> use "# encoding: utf-8" and not "# -*- coding: utf-8 -*-". The >>>>>>>>>>> latter is just weird if you ask me. >>>>>>>>>>> >>>>>>>>>>> Thomas: It's optional, defaults to iso-8859-1 and the header >>>>>>>>>>> will clearly state that a file uses a different encoding, so I disagree >>>>>>>>>>> that there will be any confusion. >>>>>>>>>>> >>>>>>>>>>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme >>>>>>>>>>>> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>>>>>>>> > From the javadoc of java.util.ResourceBundle the encoding >>>>>>>>>>>> for .properties is >>>>>>>>>>>> > still expected to be ISO-8859-1: >>>>>>>>>>>> > >>>>>>>>>>>> >> Constructing a PropertyResourceBundle instance from an >>>>>>>>>>>> InputStream >>>>>>>>>>>> >> requires that the input stream be encoded in ISO-8859-1. In >>>>>>>>>>>> that case, >>>>>>>>>>>> >> characters that cannot be represented in ISO-8859-1 >>>>>>>>>>>> encoding must be >>>>>>>>>>>> >> represented by Unicode Escapes as defined in section 3.3 of >>>>>>>>>>>> The Java™ >>>>>>>>>>>> >> Language Specification whereas the other constructor which >>>>>>>>>>>> takes a Reader >>>>>>>>>>>> >> does not have that limitation. >>>>>>>>>>>> >>>>>>>>>>>> It's a case of using constructor which takes an InputStream. >>>>>>>>>>>> It says we can use any encoding when we use the other >>>>>>>>>>>> constructor >>>>>>>>>>>> which takes a Reader. >>>>>>>>>>>> >>>>>>>>>>>> I understand your concern. So what do you think supporting >>>>>>>>>>>> utf-8 with >>>>>>>>>>>> a header like Stian suggested? >>>>>>>>>>>> I think it can avoid confusion because the encoding is noted >>>>>>>>>>>> in the >>>>>>>>>>>> file itself... >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... >>>>>>>>>>>> > >>>>>>>>>>>> > I understand that Spring allows one to read .properties >>>>>>>>>>>> using any encoding, >>>>>>>>>>>> > but you need to specify the encoding in Spring configuration >>>>>>>>>>>> external to the >>>>>>>>>>>> > .properties file. >>>>>>>>>>>> > >>>>>>>>>>>> > In my opinion it would cause unnecessary confusion amongst >>>>>>>>>>>> developers to use >>>>>>>>>>>> > any other encoding than the one defined by the official >>>>>>>>>>>> documentation for >>>>>>>>>>>> > Properties. >>>>>>>>>>>> > >>>>>>>>>>>> > Best regards, >>>>>>>>>>>> > Thomas >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada < >>>>>>>>>>>> wadahiro(a)gmail.com&gt; wrote: >>>>>>>>>>>> >> >>>>>>>>>>>> >> I think it was true before Java 1.6 because Java standard >>>>>>>>>>>> library >>>>>>>>>>>> >> (java.util.ResourceBundle or java.util.Properties) only >>>>>>>>>>>> supported >>>>>>>>>>>> >> ISO-8859-1 encoding file. >>>>>>>>>>>> >> But they support any encoding after Java 1.6. There are some >>>>>>>>>>>> >> frameworks which can read .properties with any encoding. >>>>>>>>>>>> For example, >>>>>>>>>>>> >> Spring Framework can read .properties with any encoding. I >>>>>>>>>>>> think it's >>>>>>>>>>>> >> very useful to read UTF-8 directly for multibyte language >>>>>>>>>>>> country. >>>>>>>>>>>> >> >>>>>>>>>>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme >>>>>>>>>>>> >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>>>>>>>> >> > Please correct me if I am wrong, but I have been under >>>>>>>>>>>> the impression >>>>>>>>>>>> >> > that >>>>>>>>>>>> >> > Java .properties files should always use encoding >>>>>>>>>>>> ISO-8859-1. Characters >>>>>>>>>>>> >> > not >>>>>>>>>>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it >>>>>>>>>>>> make things >>>>>>>>>>>> >> > confusing to developers if another encoding is used here >>>>>>>>>>>> instead? >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > If alternate encodings are desired how about supporting >>>>>>>>>>>> the XML format >>>>>>>>>>>> >> > of >>>>>>>>>>>> >> > Properties? >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > Best regards, >>>>>>>>>>>> >> > Thomas >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < >>>>>>>>>>>> sthorger(a)redhat.com&gt; >>>>>>>>>>>> >> > wrote: >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able >>>>>>>>>>>> to co-exist. >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> We can allow specifying the encoding in a comment on the >>>>>>>>>>>> first line >>>>>>>>>>>> >> >> like >>>>>>>>>>>> >> >> this: >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> # encoding=utf-8 >>>>>>>>>>>> >> >> key=value >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> # encoding=iso-8859-1 >>>>>>>>>>>> >> >> key=value >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> If the first line in the file doesn't contain the >>>>>>>>>>>> comment with the >>>>>>>>>>>> >> >> encoding then we should default to iso-8859-1 for >>>>>>>>>>>> backwards >>>>>>>>>>>> >> >> compatibility >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada < >>>>>>>>>>>> wadahiro(a)gmail.com&gt; wrote: >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> Thanks for your comment. >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> > If we want to change to utf-8 we'd still need to >>>>>>>>>>>> support iso.. for >>>>>>>>>>>> >> >>> > backwards compatibility. >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> If we change to UTF-8, we can still read unicode >>>>>>>>>>>> codepoint like >>>>>>>>>>>> >> >>> '\u00e8'. >>>>>>>>>>>> >> >>> There is an incompatibility when non-ascii characters >>>>>>>>>>>> are used in >>>>>>>>>>>> >> >>> message properties. >>>>>>>>>>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please >>>>>>>>>>>> refer codepage >>>>>>>>>>>> >> >>> layout: >>>>>>>>>>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> I think the non-ascii characters might be used in >>>>>>>>>>>> French messages like >>>>>>>>>>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards >>>>>>>>>>>> compatibility. >>>>>>>>>>>> >> >>> To support this, I think we can add a property like >>>>>>>>>>>> "messageEncoding" >>>>>>>>>>>> >> >>> in keycloak-server.json as below. Is it a good idea? >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> "theme": { >>>>>>>>>>>> >> >>> "staticMaxAge": 2592000, >>>>>>>>>>>> >> >>> "cacheTemplates": true, >>>>>>>>>>>> >> >>> "cacheThemes": true, >>>>>>>>>>>> >> >>> "messageEncoding": "ISO-8859-1", >>>>>>>>>>>> >> >>> "folder": { >>>>>>>>>>>> >> >>> "dir": "${jboss.home.dir}/themes" >>>>>>>>>>>> >> >>> } >>>>>>>>>>>> >> >>> }, >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> Regards, >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen >>>>>>>>>>>> >> >>> <sthorger(a)redhat.com&gt; >>>>>>>>>>>> >> >>> wrote: >>>>>>>>>>>> >> >>> > We have in the past discussed this and decided to >>>>>>>>>>>> stick with >>>>>>>>>>>> >> >>> > ISO-8859-1. >>>>>>>>>>>> >> >>> > That was probably not the best idea though. If we >>>>>>>>>>>> want to change to >>>>>>>>>>>> >> >>> > utf-8 >>>>>>>>>>>> >> >>> > we'd still need to support iso.. for backwards >>>>>>>>>>>> compatibility. >>>>>>>>>>>> >> >>> > >>>>>>>>>>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira < >>>>>>>>>>>> bruno(a)abstractj.org&gt; wrote: >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> >>> >> It makes sense, maybe file a Jira associated with: >>>>>>>>>>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: >>>>>>>>>>>> >> >>> >> > Hello all, >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > I am trying to translate all base theme messages >>>>>>>>>>>> to my country >>>>>>>>>>>> >> >>> >> > language, Japanese. And I'd like to contribute >>>>>>>>>>>> them. Before that >>>>>>>>>>>> >> >>> >> > work, >>>>>>>>>>>> >> >>> >> > I'd like to propose about the files encoding. >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > Currently, the message files (*.properties) are >>>>>>>>>>>> loaded with >>>>>>>>>>>> >> >>> >> > ISO-8859-1 >>>>>>>>>>>> >> >>> >> > encoding. Therefore, it is necessary to convert >>>>>>>>>>>> the files by >>>>>>>>>>>> >> >>> >> > 'native2ascii' command beforehand. However we can >>>>>>>>>>>> directly read >>>>>>>>>>>> >> >>> >> > the >>>>>>>>>>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or >>>>>>>>>>>> later because >>>>>>>>>>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method >>>>>>>>>>>> was introduced >>>>>>>>>>>> >> >>> >> > as >>>>>>>>>>>> >> >>> >> > below. >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > So, my proposal is supporting the message files >>>>>>>>>>>> with UTF-8 >>>>>>>>>>>> >> >>> >> > encoding. >>>>>>>>>>>> >> >>> >> > I >>>>>>>>>>>> >> >>> >> > believe that it's very developers/customers >>>>>>>>>>>> friendly. In >>>>>>>>>>>> >> >>> >> > addition, >>>>>>>>>>>> >> >>> >> > we >>>>>>>>>>>> >> >>> >> > can easily review the translated messages on the >>>>>>>>>>>> github pull >>>>>>>>>>>> >> >>> >> > request >>>>>>>>>>>> >> >>> >> > view and so on. What do you think? >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a >>>>>>>>>>>> pull request. >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > Regards, >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> >> >>> >> > -- >>>>>>>>>>>> >> >>> >> > Hiroyuki Wada, >>>>>>>>>>>> >> >>> >> > Developer, >>>>>>>>>>>> >> >>> >> > Nomura Research Institute, Ltd. >>>>>>>>>>>> >> >>> >> > _______________________________________________ >>>>>>>>>>>> >> >>> >> > keycloak-dev mailing list >>>>>>>>>>>> >> >>> >> > keycloak-dev(a)lists.jboss.org >>>>>>>>>>>> >> >>> >> > >>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> >>> >> -- >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> >> >>> >> abstractj >>>>>>>>>>>> >> >>> >> PGP: 0x84DC9914 >>>>>>>>>>>> >> >>> >> _______________________________________________ >>>>>>>>>>>> >> >>> >> keycloak-dev mailing list >>>>>>>>>>>> >> >>> >> keycloak-dev(a)lists.jboss.org >>>>>>>>>>>> >> >>> >> >>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>>>>>> >> >>> > >>>>>>>>>>>> >> >>> > >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> _______________________________________________ >>>>>>>>>>>> >> >> keycloak-dev mailing list >>>>>>>>>>>> >> >> keycloak-dev(a)lists.jboss.org >>>>>>>>>>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> keycloak-dev mailing list >>>>>>>>>>>> keycloak-dev(a)lists.jboss.org >>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>> >>>> >>> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >