Re: [keycloak-dev] Support UTF-8 encoding theme message properites

Let's go for it. Please create a JIRA and send PR. Please just use "# encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter is just weird if you ask me. Thomas: It's optional, defaults to iso-8859-1 and the header will clearly state that a file uses a different encoding, so I disagree that there will be any confusion. On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme > <thomas.raehalme(a)aitiofinland.com&gt; wrote: > > From the javadoc of java.util.ResourceBundle the encoding for > .properties is > > still expected to be ISO-8859-1: > > > >> Constructing a PropertyResourceBundle instance from an InputStream > >> requires that the input stream be encoded in ISO-8859-1. In that case, > >> characters that cannot be represented in ISO-8859-1 encoding must be > >> represented by Unicode Escapes as defined in section 3.3 of The Java™ > >> Language Specification whereas the other constructor which takes a > Reader > >> does not have that limitation. > > It's a case of using constructor which takes an InputStream. > It says we can use any encoding when we use the other constructor > which takes a Reader. > > I understand your concern. So what do you think supporting utf-8 with > a header like Stian suggested? > I think it can avoid confusion because the encoding is noted in the > file itself... > > Regards, > > > > https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... > > > > I understand that Spring allows one to read .properties using any > encoding, > > but you need to specify the encoding in Spring configuration external > to the > > .properties file. > > > > In my opinion it would cause unnecessary confusion amongst developers > to use > > any other encoding than the one defined by the official documentation > for > > Properties. > > > > Best regards, > > Thomas > > > > > > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <wadahiro(a)gmail.com&gt; > wrote: > >> > >> I think it was true before Java 1.6 because Java standard library > >> (java.util.ResourceBundle or java.util.Properties) only supported > >> ISO-8859-1 encoding file. > >> But they support any encoding after Java 1.6. There are some > >> frameworks which can read .properties with any encoding. For example, > >> Spring Framework can read .properties with any encoding. I think it's > >> very useful to read UTF-8 directly for multibyte language country. > >> > >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme > >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: > >> > Please correct me if I am wrong, but I have been under the impression > >> > that > >> > Java .properties files should always use encoding ISO-8859-1. > Characters > >> > not > >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make things > >> > confusing to developers if another encoding is used here instead? > >> > > >> > > >> > > https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... > >> > > >> > > https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... > >> > > >> > If alternate encodings are desired how about supporting the XML > format > >> > of > >> > Properties? > >> > > >> > Best regards, > >> > Thomas > >> > > >> > > >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < > sthorger(a)redhat.com&gt; > >> > wrote: > >> >> > >> >> Both iso-8859-1 and utf-8 message bundles should be able to > co-exist. > >> >> > >> >> We can allow specifying the encoding in a comment on the first line > >> >> like > >> >> this: > >> >> > >> >> # encoding=utf-8 > >> >> key=value > >> >> > >> >> # encoding=iso-8859-1 > >> >> key=value > >> >> > >> >> If the first line in the file doesn't contain the comment with the > >> >> encoding then we should default to iso-8859-1 for backwards > >> >> compatibility > >> >> > >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > >> >>> > >> >>> Thanks for your comment. > >> >>> > >> >>> > If we want to change to utf-8 we'd still need to support iso.. > for > >> >>> > backwards compatibility. > >> >>> > >> >>> If we change to UTF-8, we can still read unicode codepoint like > >> >>> '\u00e8'. > >> >>> There is an incompatibility when non-ascii characters are used in > >> >>> message properties. > >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer > codepage > >> >>> layout: > https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) > >> >>> > >> >>> I think the non-ascii characters might be used in French messages > like > >> >>> 'à' so I agree to support ISO-8859-1 for backwards compatibility. > >> >>> To support this, I think we can add a property like > "messageEncoding" > >> >>> in keycloak-server.json as below. Is it a good idea? > >> >>> > >> >>> "theme": { > >> >>> "staticMaxAge": 2592000, > >> >>> "cacheTemplates": true, > >> >>> "cacheThemes": true, > >> >>> "messageEncoding": "ISO-8859-1", > >> >>> "folder": { > >> >>> "dir": "${jboss.home.dir}/themes" > >> >>> } > >> >>> }, > >> >>> > >> >>> Regards, > >> >>> > >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen > >> >>> <sthorger(a)redhat.com&gt; > >> >>> wrote: > >> >>> > We have in the past discussed this and decided to stick with > >> >>> > ISO-8859-1. > >> >>> > That was probably not the best idea though. If we want to change > to > >> >>> > utf-8 > >> >>> > we'd still need to support iso.. for backwards compatibility. > >> >>> > > >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; > wrote: > >> >>> >> > >> >>> >> It makes sense, maybe file a Jira associated with: > >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? > >> >>> >> > >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: > >> >>> >> > Hello all, > >> >>> >> > > >> >>> >> > I am trying to translate all base theme messages to my country > >> >>> >> > language, Japanese. And I'd like to contribute them. Before > that > >> >>> >> > work, > >> >>> >> > I'd like to propose about the files encoding. > >> >>> >> > > >> >>> >> > Currently, the message files (*.properties) are loaded with > >> >>> >> > ISO-8859-1 > >> >>> >> > encoding. Therefore, it is necessary to convert the files by > >> >>> >> > 'native2ascii' command beforehand. However we can directly > read > >> >>> >> > the > >> >>> >> > property files with UTF-8 encoding in java 1.6 or later > because > >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was > introduced > >> >>> >> > as > >> >>> >> > below. > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > > http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... > >> >>> >> > > >> >>> >> > So, my proposal is supporting the message files with UTF-8 > >> >>> >> > encoding. > >> >>> >> > I > >> >>> >> > believe that it's very developers/customers friendly. In > >> >>> >> > addition, > >> >>> >> > we > >> >>> >> > can easily review the translated messages on the github pull > >> >>> >> > request > >> >>> >> > view and so on. What do you think? > >> >>> >> > > >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull > request. > >> >>> >> > > >> >>> >> > Regards, > >> >>> >> > > >> >>> >> > -- > >> >>> >> > Hiroyuki Wada, > >> >>> >> > Developer, > >> >>> >> > Nomura Research Institute, Ltd. > >> >>> >> > _______________________________________________ > >> >>> >> > keycloak-dev mailing list > >> >>> >> > keycloak-dev(a)lists.jboss.org > >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> >>> >> > >> >>> >> -- > >> >>> >> > >> >>> >> abstractj > >> >>> >> PGP: 0x84DC9914 > >> >>> >> _______________________________________________ > >> >>> >> keycloak-dev mailing list > >> >>> >> keycloak-dev(a)lists.jboss.org > >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> >>> > > >> >>> > > >> >> > >> >> > >> >> > >> >> _______________________________________________ > >> >> keycloak-dev mailing list > >> >> keycloak-dev(a)lists.jboss.org > >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > >> > > >> > > > > > > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev >