Re: [keycloak-dev] Support UTF-8 encoding theme message properites

Yes. I will send a pull request later! 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger(a)redhat.com&gt;: > Are you doing a PR as well? > > On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > >> Ok. I created a JIRA issue. >> https://issues.jboss.org/browse/KEYCLOAK-3278 >> >> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger(a)redhat.com&gt;: >> >>> Let's go for it. Please create a JIRA and send PR. Please just use "# >>> encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter is just >>> weird if you ask me. >>> >>> Thomas: It's optional, defaults to iso-8859-1 and the header will >>> clearly state that a file uses a different encoding, so I disagree that >>> there will be any confusion. >>> >>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>> >>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme >>>> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>> > From the javadoc of java.util.ResourceBundle the encoding for >>>> .properties is >>>> > still expected to be ISO-8859-1: >>>> > >>>> >> Constructing a PropertyResourceBundle instance from an InputStream >>>> >> requires that the input stream be encoded in ISO-8859-1. In that >>>> case, >>>> >> characters that cannot be represented in ISO-8859-1 encoding must be >>>> >> represented by Unicode Escapes as defined in section 3.3 of The >>>> Java™ >>>> >> Language Specification whereas the other constructor which takes a >>>> Reader >>>> >> does not have that limitation. >>>> >>>> It's a case of using constructor which takes an InputStream. >>>> It says we can use any encoding when we use the other constructor >>>> which takes a Reader. >>>> >>>> I understand your concern. So what do you think supporting utf-8 with >>>> a header like Stian suggested? >>>> I think it can avoid confusion because the encoding is noted in the >>>> file itself... >>>> >>>> Regards, >>>> >>>> > >>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... >>>> > >>>> > I understand that Spring allows one to read .properties using any >>>> encoding, >>>> > but you need to specify the encoding in Spring configuration >>>> external to the >>>> > .properties file. >>>> > >>>> > In my opinion it would cause unnecessary confusion amongst >>>> developers to use >>>> > any other encoding than the one defined by the official >>>> documentation for >>>> > Properties. >>>> > >>>> > Best regards, >>>> > Thomas >>>> > >>>> > >>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>> wrote: >>>> >> >>>> >> I think it was true before Java 1.6 because Java standard library >>>> >> (java.util.ResourceBundle or java.util.Properties) only supported >>>> >> ISO-8859-1 encoding file. >>>> >> But they support any encoding after Java 1.6. There are some >>>> >> frameworks which can read .properties with any encoding. For >>>> example, >>>> >> Spring Framework can read .properties with any encoding. I think >>>> it's >>>> >> very useful to read UTF-8 directly for multibyte language country. >>>> >> >>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme >>>> >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>> >> > Please correct me if I am wrong, but I have been under the >>>> impression >>>> >> > that >>>> >> > Java .properties files should always use encoding ISO-8859-1. >>>> Characters >>>> >> > not >>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make >>>> things >>>> >> > confusing to developers if another encoding is used here instead? >>>> >> > >>>> >> > >>>> >> > >>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >>>> >> > >>>> >> > >>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >>>> >> > >>>> >> > If alternate encodings are desired how about supporting the XML >>>> format >>>> >> > of >>>> >> > Properties? >>>> >> > >>>> >> > Best regards, >>>> >> > Thomas >>>> >> > >>>> >> > >>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < >>>> sthorger(a)redhat.com&gt; >>>> >> > wrote: >>>> >> >> >>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able to >>>> co-exist. >>>> >> >> >>>> >> >> We can allow specifying the encoding in a comment on the first >>>> line >>>> >> >> like >>>> >> >> this: >>>> >> >> >>>> >> >> # encoding=utf-8 >>>> >> >> key=value >>>> >> >> >>>> >> >> # encoding=iso-8859-1 >>>> >> >> key=value >>>> >> >> >>>> >> >> If the first line in the file doesn't contain the comment with >>>> the >>>> >> >> encoding then we should default to iso-8859-1 for backwards >>>> >> >> compatibility >>>> >> >> >>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>> wrote: >>>> >> >>> >>>> >> >>> Thanks for your comment. >>>> >> >>> >>>> >> >>> > If we want to change to utf-8 we'd still need to support >>>> iso.. for >>>> >> >>> > backwards compatibility. >>>> >> >>> >>>> >> >>> If we change to UTF-8, we can still read unicode codepoint like >>>> >> >>> '\u00e8'. >>>> >> >>> There is an incompatibility when non-ascii characters are used >>>> in >>>> >> >>> message properties. >>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer >>>> codepage >>>> >> >>> layout: >>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >>>> >> >>> >>>> >> >>> I think the non-ascii characters might be used in French >>>> messages like >>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards >>>> compatibility. >>>> >> >>> To support this, I think we can add a property like >>>> "messageEncoding" >>>> >> >>> in keycloak-server.json as below. Is it a good idea? >>>> >> >>> >>>> >> >>> "theme": { >>>> >> >>> "staticMaxAge": 2592000, >>>> >> >>> "cacheTemplates": true, >>>> >> >>> "cacheThemes": true, >>>> >> >>> "messageEncoding": "ISO-8859-1", >>>> >> >>> "folder": { >>>> >> >>> "dir": "${jboss.home.dir}/themes" >>>> >> >>> } >>>> >> >>> }, >>>> >> >>> >>>> >> >>> Regards, >>>> >> >>> >>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen >>>> >> >>> <sthorger(a)redhat.com&gt; >>>> >> >>> wrote: >>>> >> >>> > We have in the past discussed this and decided to stick with >>>> >> >>> > ISO-8859-1. >>>> >> >>> > That was probably not the best idea though. If we want to >>>> change to >>>> >> >>> > utf-8 >>>> >> >>> > we'd still need to support iso.. for backwards compatibility. >>>> >> >>> > >>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno(a)abstractj.org&gt; >>>> wrote: >>>> >> >>> >> >>>> >> >>> >> It makes sense, maybe file a Jira associated with: >>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >>>> >> >>> >> >>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: >>>> >> >>> >> > Hello all, >>>> >> >>> >> > >>>> >> >>> >> > I am trying to translate all base theme messages to my >>>> country >>>> >> >>> >> > language, Japanese. And I'd like to contribute them. >>>> Before that >>>> >> >>> >> > work, >>>> >> >>> >> > I'd like to propose about the files encoding. >>>> >> >>> >> > >>>> >> >>> >> > Currently, the message files (*.properties) are loaded with >>>> >> >>> >> > ISO-8859-1 >>>> >> >>> >> > encoding. Therefore, it is necessary to convert the files >>>> by >>>> >> >>> >> > 'native2ascii' command beforehand. However we can directly >>>> read >>>> >> >>> >> > the >>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or later >>>> because >>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was >>>> introduced >>>> >> >>> >> > as >>>> >> >>> >> > below. >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> > >>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >>>> >> >>> >> > >>>> >> >>> >> > So, my proposal is supporting the message files with UTF-8 >>>> >> >>> >> > encoding. >>>> >> >>> >> > I >>>> >> >>> >> > believe that it's very developers/customers friendly. In >>>> >> >>> >> > addition, >>>> >> >>> >> > we >>>> >> >>> >> > can easily review the translated messages on the github >>>> pull >>>> >> >>> >> > request >>>> >> >>> >> > view and so on. What do you think? >>>> >> >>> >> > >>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull >>>> request. >>>> >> >>> >> > >>>> >> >>> >> > Regards, >>>> >> >>> >> > >>>> >> >>> >> > -- >>>> >> >>> >> > Hiroyuki Wada, >>>> >> >>> >> > Developer, >>>> >> >>> >> > Nomura Research Institute, Ltd. >>>> >> >>> >> > _______________________________________________ >>>> >> >>> >> > keycloak-dev mailing list >>>> >> >>> >> > keycloak-dev(a)lists.jboss.org >>>> >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> >>> >> >>>> >> >>> >> -- >>>> >> >>> >> >>>> >> >>> >> abstractj >>>> >> >>> >> PGP: 0x84DC9914 >>>> >> >>> >> _______________________________________________ >>>> >> >>> >> keycloak-dev mailing list >>>> >> >>> >> keycloak-dev(a)lists.jboss.org >>>> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> >>> > >>>> >> >>> > >>>> >> >> >>>> >> >> >>>> >> >> >>>> >> >> _______________________________________________ >>>> >> >> keycloak-dev mailing list >>>> >> >> keycloak-dev(a)lists.jboss.org >>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> > >>>> >> > >>>> >> > >>>> > >>>> > >>>> > >>>> > >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> >>> >