Re: [keycloak-dev] Support UTF-8 encoding theme message properites

I mean I'd like to change the message files encoding only. For example, admin-messages for Russian language is currently written in ISO-8859-1 encoding. https://github.com/keycloak/keycloak/blob/master/themes/src/main/resource... Can I convert the language messages except for English? 2016年7月15日(金) 14:03 Stian Thorgersen <sthorger(a)redhat.com&gt;: > No, we don't want the theme properties (and other properties files) to be > utf-8. They should stick with the default encoding used by Java properties > files. > > Messages are a special case, which is why we accepted it for them. > > On 14 July 2016 at 13:29, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: > >> Thank you for merging the pull request. Could I send a pull request >> converting to UTF-8 encoding against the theme properties? >> >> 2016年7月7日(木) 21:39 Hiroyuki Wada <wadahiro(a)gmail.com&gt;: >> >>> Yes. I will send a pull request later! >>> >>> 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>> >>>> Are you doing a PR as well? >>>> >>>> On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>> >>>>> Ok. I created a JIRA issue. >>>>> https://issues.jboss.org/browse/KEYCLOAK-3278 >>>>> >>>>> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger(a)redhat.com&gt;: >>>>> >>>>>> Let's go for it. Please create a JIRA and send PR. Please just use >>>>>> "# encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter >>>>>> is just weird if you ask me. >>>>>> >>>>>> Thomas: It's optional, defaults to iso-8859-1 and the header will >>>>>> clearly state that a file uses a different encoding, so I disagree that >>>>>> there will be any confusion. >>>>>> >>>>>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro(a)gmail.com&gt; wrote: >>>>>> >>>>>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme >>>>>>> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>>> > From the javadoc of java.util.ResourceBundle the encoding for >>>>>>> .properties is >>>>>>> > still expected to be ISO-8859-1: >>>>>>> > >>>>>>> >> Constructing a PropertyResourceBundle instance from an >>>>>>> InputStream >>>>>>> >> requires that the input stream be encoded in ISO-8859-1. In that >>>>>>> case, >>>>>>> >> characters that cannot be represented in ISO-8859-1 encoding >>>>>>> must be >>>>>>> >> represented by Unicode Escapes as defined in section 3.3 of The >>>>>>> Java™ >>>>>>> >> Language Specification whereas the other constructor which takes >>>>>>> a Reader >>>>>>> >> does not have that limitation. >>>>>>> >>>>>>> It's a case of using constructor which takes an InputStream. >>>>>>> It says we can use any encoding when we use the other constructor >>>>>>> which takes a Reader. >>>>>>> >>>>>>> I understand your concern. So what do you think supporting utf-8 >>>>>>> with >>>>>>> a header like Stian suggested? >>>>>>> I think it can avoid confusion because the encoding is noted in the >>>>>>> file itself... >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> > >>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundl... >>>>>>> > >>>>>>> > I understand that Spring allows one to read .properties using any >>>>>>> encoding, >>>>>>> > but you need to specify the encoding in Spring configuration >>>>>>> external to the >>>>>>> > .properties file. >>>>>>> > >>>>>>> > In my opinion it would cause unnecessary confusion amongst >>>>>>> developers to use >>>>>>> > any other encoding than the one defined by the official >>>>>>> documentation for >>>>>>> > Properties. >>>>>>> > >>>>>>> > Best regards, >>>>>>> > Thomas >>>>>>> > >>>>>>> > >>>>>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada < >>>>>>> wadahiro(a)gmail.com&gt; wrote: >>>>>>> >> >>>>>>> >> I think it was true before Java 1.6 because Java standard library >>>>>>> >> (java.util.ResourceBundle or java.util.Properties) only supported >>>>>>> >> ISO-8859-1 encoding file. >>>>>>> >> But they support any encoding after Java 1.6. There are some >>>>>>> >> frameworks which can read .properties with any encoding. For >>>>>>> example, >>>>>>> >> Spring Framework can read .properties with any encoding. I think >>>>>>> it's >>>>>>> >> very useful to read UTF-8 directly for multibyte language >>>>>>> country. >>>>>>> >> >>>>>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme >>>>>>> >> <thomas.raehalme(a)aitiofinland.com&gt; wrote: >>>>>>> >> > Please correct me if I am wrong, but I have been under the >>>>>>> impression >>>>>>> >> > that >>>>>>> >> > Java .properties files should always use encoding ISO-8859-1. >>>>>>> Characters >>>>>>> >> > not >>>>>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make >>>>>>> things >>>>>>> >> > confusing to developers if another encoding is used here >>>>>>> instead? >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-... >>>>>>> >> > >>>>>>> >> > >>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store... >>>>>>> >> > >>>>>>> >> > If alternate encodings are desired how about supporting the >>>>>>> XML format >>>>>>> >> > of >>>>>>> >> > Properties? >>>>>>> >> > >>>>>>> >> > Best regards, >>>>>>> >> > Thomas >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen < >>>>>>> sthorger(a)redhat.com&gt; >>>>>>> >> > wrote: >>>>>>> >> >> >>>>>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able to >>>>>>> co-exist. >>>>>>> >> >> >>>>>>> >> >> We can allow specifying the encoding in a comment on the >>>>>>> first line >>>>>>> >> >> like >>>>>>> >> >> this: >>>>>>> >> >> >>>>>>> >> >> # encoding=utf-8 >>>>>>> >> >> key=value >>>>>>> >> >> >>>>>>> >> >> # encoding=iso-8859-1 >>>>>>> >> >> key=value >>>>>>> >> >> >>>>>>> >> >> If the first line in the file doesn't contain the comment >>>>>>> with the >>>>>>> >> >> encoding then we should default to iso-8859-1 for backwards >>>>>>> >> >> compatibility >>>>>>> >> >> >>>>>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro(a)gmail.com&gt; >>>>>>> wrote: >>>>>>> >> >>> >>>>>>> >> >>> Thanks for your comment. >>>>>>> >> >>> >>>>>>> >> >>> > If we want to change to utf-8 we'd still need to support >>>>>>> iso.. for >>>>>>> >> >>> > backwards compatibility. >>>>>>> >> >>> >>>>>>> >> >>> If we change to UTF-8, we can still read unicode codepoint >>>>>>> like >>>>>>> >> >>> '\u00e8'. >>>>>>> >> >>> There is an incompatibility when non-ascii characters are >>>>>>> used in >>>>>>> >> >>> message properties. >>>>>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer >>>>>>> codepage >>>>>>> >> >>> layout: >>>>>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout ) >>>>>>> >> >>> >>>>>>> >> >>> I think the non-ascii characters might be used in French >>>>>>> messages like >>>>>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards >>>>>>> compatibility. >>>>>>> >> >>> To support this, I think we can add a property like >>>>>>> "messageEncoding" >>>>>>> >> >>> in keycloak-server.json as below. Is it a good idea? >>>>>>> >> >>> >>>>>>> >> >>> "theme": { >>>>>>> >> >>> "staticMaxAge": 2592000, >>>>>>> >> >>> "cacheTemplates": true, >>>>>>> >> >>> "cacheThemes": true, >>>>>>> >> >>> "messageEncoding": "ISO-8859-1", >>>>>>> >> >>> "folder": { >>>>>>> >> >>> "dir": "${jboss.home.dir}/themes" >>>>>>> >> >>> } >>>>>>> >> >>> }, >>>>>>> >> >>> >>>>>>> >> >>> Regards, >>>>>>> >> >>> >>>>>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen >>>>>>> >> >>> <sthorger(a)redhat.com&gt; >>>>>>> >> >>> wrote: >>>>>>> >> >>> > We have in the past discussed this and decided to stick >>>>>>> with >>>>>>> >> >>> > ISO-8859-1. >>>>>>> >> >>> > That was probably not the best idea though. If we want to >>>>>>> change to >>>>>>> >> >>> > utf-8 >>>>>>> >> >>> > we'd still need to support iso.. for backwards >>>>>>> compatibility. >>>>>>> >> >>> > >>>>>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira < >>>>>>> bruno(a)abstractj.org&gt; wrote: >>>>>>> >> >>> >> >>>>>>> >> >>> >> It makes sense, maybe file a Jira associated with: >>>>>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ? >>>>>>> >> >>> >> >>>>>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote: >>>>>>> >> >>> >> > Hello all, >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > I am trying to translate all base theme messages to my >>>>>>> country >>>>>>> >> >>> >> > language, Japanese. And I'd like to contribute them. >>>>>>> Before that >>>>>>> >> >>> >> > work, >>>>>>> >> >>> >> > I'd like to propose about the files encoding. >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > Currently, the message files (*.properties) are loaded >>>>>>> with >>>>>>> >> >>> >> > ISO-8859-1 >>>>>>> >> >>> >> > encoding. Therefore, it is necessary to convert the >>>>>>> files by >>>>>>> >> >>> >> > 'native2ascii' command beforehand. However we can >>>>>>> directly read >>>>>>> >> >>> >> > the >>>>>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or later >>>>>>> because >>>>>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was >>>>>>> introduced >>>>>>> >> >>> >> > as >>>>>>> >> >>> >> > below. >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > >>>>>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(j... >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > So, my proposal is supporting the message files with >>>>>>> UTF-8 >>>>>>> >> >>> >> > encoding. >>>>>>> >> >>> >> > I >>>>>>> >> >>> >> > believe that it's very developers/customers friendly. In >>>>>>> >> >>> >> > addition, >>>>>>> >> >>> >> > we >>>>>>> >> >>> >> > can easily review the translated messages on the github >>>>>>> pull >>>>>>> >> >>> >> > request >>>>>>> >> >>> >> > view and so on. What do you think? >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull >>>>>>> request. >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > Regards, >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > -- >>>>>>> >> >>> >> > Hiroyuki Wada, >>>>>>> >> >>> >> > Developer, >>>>>>> >> >>> >> > Nomura Research Institute, Ltd. >>>>>>> >> >>> >> > _______________________________________________ >>>>>>> >> >>> >> > keycloak-dev mailing list >>>>>>> >> >>> >> > keycloak-dev(a)lists.jboss.org >>>>>>> >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> >> >>> >> >>>>>>> >> >>> >> -- >>>>>>> >> >>> >> >>>>>>> >> >>> >> abstractj >>>>>>> >> >>> >> PGP: 0x84DC9914 >>>>>>> >> >>> >> _______________________________________________ >>>>>>> >> >>> >> keycloak-dev mailing list >>>>>>> >> >>> >> keycloak-dev(a)lists.jboss.org >>>>>>> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> >> >>> > >>>>>>> >> >>> > >>>>>>> >> >> >>>>>>> >> >> >>>>>>> >> >> >>>>>>> >> >> _______________________________________________ >>>>>>> >> >> keycloak-dev mailing list >>>>>>> >> >> keycloak-dev(a)lists.jboss.org >>>>>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> >>>>>>> _______________________________________________ >>>>>>> keycloak-dev mailing list >>>>>>> keycloak-dev(a)lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> >>>>>> >>>>>> >>>> >