[keycloak-dev] Support UTF-8 encoding theme message properites

Fri Jul 15 01:03:18 EDT 2016

No, we don't want the theme properties (and other properties files) to be
utf-8. They should stick with the default encoding used by Java properties
files.

Messages are a special case, which is why we accepted it for them.

On 14 July 2016 at 13:29, Hiroyuki Wada <wadahiro at gmail.com> wrote:

> Thank you for merging the pull request. Could I send a pull request
> converting to UTF-8 encoding against the theme properties?
>
> 2016年7月7日(木) 21:39 Hiroyuki Wada <wadahiro at gmail.com>:
>
>> Yes. I will send a pull request later!
>>
>> 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger at redhat.com>:
>>
>>> Are you doing a PR as well?
>>>
>>> On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro at gmail.com> wrote:
>>>
>>>> Ok. I created a JIRA issue.
>>>> https://issues.jboss.org/browse/KEYCLOAK-3278
>>>>
>>>> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger at redhat.com>:
>>>>
>>>>> Let's go for it. Please create a JIRA and send PR. Please just use "#
>>>>> encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter is
>>>>> just weird if you ask me.
>>>>>
>>>>> Thomas: It's optional, defaults to iso-8859-1 and the header will
>>>>> clearly state that a file uses a different encoding, so I disagree that
>>>>> there will be any confusion.
>>>>>
>>>>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro at gmail.com> wrote:
>>>>>
>>>>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme
>>>>>> <thomas.raehalme at aitiofinland.com> wrote:
>>>>>> > From the javadoc of java.util.ResourceBundle the encoding for
>>>>>> .properties is
>>>>>> > still expected to be ISO-8859-1:
>>>>>> >
>>>>>> >> Constructing a PropertyResourceBundle instance from an InputStream
>>>>>> >> requires that the input stream be encoded in ISO-8859-1. In that
>>>>>> case,
>>>>>> >> characters that cannot be represented in ISO-8859-1 encoding must
>>>>>> be
>>>>>> >> represented by Unicode Escapes as defined in section 3.3 of The
>>>>>> Java™
>>>>>> >> Language Specification whereas the other constructor which takes a
>>>>>> Reader
>>>>>> >> does not have that limitation.
>>>>>>
>>>>>> It's a case of using constructor which takes an InputStream.
>>>>>> It says we can use any encoding when we use the other constructor
>>>>>> which takes a Reader.
>>>>>>
>>>>>> I understand your concern. So what do you think supporting utf-8 with
>>>>>> a header like Stian suggested?
>>>>>> I think it can avoid confusion because the encoding is noted in the
>>>>>> file itself...
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> >
>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundle.html
>>>>>> >
>>>>>> > I understand that Spring allows one to read .properties using any
>>>>>> encoding,
>>>>>> > but you need to specify the encoding in Spring configuration
>>>>>> external to the
>>>>>> > .properties file.
>>>>>> >
>>>>>> > In my opinion it would cause unnecessary confusion amongst
>>>>>> developers to use
>>>>>> > any other encoding than the one defined by the official
>>>>>> documentation for
>>>>>> > Properties.
>>>>>> >
>>>>>> > Best regards,
>>>>>> > Thomas
>>>>>> >
>>>>>> >
>>>>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <wadahiro at gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> I think it was true before Java 1.6 because Java standard library
>>>>>> >> (java.util.ResourceBundle or java.util.Properties) only supported
>>>>>> >> ISO-8859-1 encoding file.
>>>>>> >> But they support any encoding after Java 1.6. There are some
>>>>>> >> frameworks which can read .properties with any encoding. For
>>>>>> example,
>>>>>> >> Spring Framework can read .properties with any encoding. I think
>>>>>> it's
>>>>>> >> very useful to read UTF-8 directly for multibyte language country.
>>>>>> >>
>>>>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme
>>>>>> >> <thomas.raehalme at aitiofinland.com> wrote:
>>>>>> >> > Please correct me if I am wrong, but I have been under the
>>>>>> impression
>>>>>> >> > that
>>>>>> >> > Java .properties files should always use encoding ISO-8859-1.
>>>>>> Characters
>>>>>> >> > not
>>>>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make
>>>>>> things
>>>>>> >> > confusing to developers if another encoding is used here instead?
>>>>>> >> >
>>>>>> >> >
>>>>>> >> >
>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-java.io.InputStream-
>>>>>> >> >
>>>>>> >> >
>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store-java.io.OutputStream-java.lang.String-
>>>>>> >> >
>>>>>> >> > If alternate encodings are desired how about supporting the XML
>>>>>> format
>>>>>> >> > of
>>>>>> >> > Properties?
>>>>>> >> >
>>>>>> >> > Best regards,
>>>>>> >> > Thomas
>>>>>> >> >
>>>>>> >> >
>>>>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen <
>>>>>> sthorger at redhat.com>
>>>>>> >> > wrote:
>>>>>> >> >>
>>>>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able to
>>>>>> co-exist.
>>>>>> >> >>
>>>>>> >> >> We can allow specifying the encoding in a comment on the first
>>>>>> line
>>>>>> >> >> like
>>>>>> >> >> this:
>>>>>> >> >>
>>>>>> >> >> # encoding=utf-8
>>>>>> >> >> key=value
>>>>>> >> >>
>>>>>> >> >> # encoding=iso-8859-1
>>>>>> >> >> key=value
>>>>>> >> >>
>>>>>> >> >> If the first line in the file doesn't contain the comment with
>>>>>> the
>>>>>> >> >> encoding then we should default to iso-8859-1 for backwards
>>>>>> >> >> compatibility
>>>>>> >> >>
>>>>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro at gmail.com>
>>>>>> wrote:
>>>>>> >> >>>
>>>>>> >> >>> Thanks for your comment.
>>>>>> >> >>>
>>>>>> >> >>> > If we want to change to utf-8 we'd still need to support
>>>>>> iso.. for
>>>>>> >> >>> > backwards compatibility.
>>>>>> >> >>>
>>>>>> >> >>> If we change to UTF-8, we can still read unicode codepoint like
>>>>>> >> >>> '\u00e8'.
>>>>>> >> >>> There is an incompatibility when non-ascii characters are used
>>>>>> in
>>>>>> >> >>> message properties.
>>>>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please refer
>>>>>> codepage
>>>>>> >> >>> layout:
>>>>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout )
>>>>>> >> >>>
>>>>>> >> >>> I think the non-ascii characters might be used in French
>>>>>> messages like
>>>>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards
>>>>>> compatibility.
>>>>>> >> >>> To support this, I think we can add a property like
>>>>>> "messageEncoding"
>>>>>> >> >>> in keycloak-server.json as below. Is it a good idea?
>>>>>> >> >>>
>>>>>> >> >>>     "theme": {
>>>>>> >> >>>         "staticMaxAge": 2592000,
>>>>>> >> >>>         "cacheTemplates": true,
>>>>>> >> >>>         "cacheThemes": true,
>>>>>> >> >>>         "messageEncoding": "ISO-8859-1",
>>>>>> >> >>>         "folder": {
>>>>>> >> >>>           "dir": "${jboss.home.dir}/themes"
>>>>>> >> >>>         }
>>>>>> >> >>>     },
>>>>>> >> >>>
>>>>>> >> >>> Regards,
>>>>>> >> >>>
>>>>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen
>>>>>> >> >>> <sthorger at redhat.com>
>>>>>> >> >>> wrote:
>>>>>> >> >>> > We have in the past discussed this and decided to stick with
>>>>>> >> >>> > ISO-8859-1.
>>>>>> >> >>> > That was probably not the best idea though. If we want to
>>>>>> change to
>>>>>> >> >>> > utf-8
>>>>>> >> >>> > we'd still need to support iso.. for backwards compatibility.
>>>>>> >> >>> >
>>>>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira <bruno at abstractj.org>
>>>>>> wrote:
>>>>>> >> >>> >>
>>>>>> >> >>> >> It makes sense, maybe file a Jira associated with:
>>>>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ?
>>>>>> >> >>> >>
>>>>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote:
>>>>>> >> >>> >> > Hello all,
>>>>>> >> >>> >> >
>>>>>> >> >>> >> > I am trying to translate all base theme messages to my
>>>>>> country
>>>>>> >> >>> >> > language, Japanese. And I'd like to contribute them.
>>>>>> Before that
>>>>>> >> >>> >> > work,
>>>>>> >> >>> >> > I'd like to propose about the files encoding.
>>>>>> >> >>> >> >
>>>>>> >> >>> >> > Currently, the message files (*.properties) are loaded
>>>>>> with
>>>>>> >> >>> >> > ISO-8859-1
>>>>>> >> >>> >> > encoding. Therefore, it is necessary to convert the files
>>>>>> by
>>>>>> >> >>> >> > 'native2ascii' command beforehand. However we can
>>>>>> directly read
>>>>>> >> >>> >> > the
>>>>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or later
>>>>>> because
>>>>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was
>>>>>> introduced
>>>>>> >> >>> >> > as
>>>>>> >> >>> >> > below.
>>>>>> >> >>> >> >
>>>>>> >> >>> >> >
>>>>>> >> >>> >> >
>>>>>> >> >>> >> >
>>>>>> >> >>> >> >
>>>>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(java.io.Reader)
>>>>>> >> >>> >> >
>>>>>> >> >>> >> > So, my proposal is supporting the message files with UTF-8
>>>>>> >> >>> >> > encoding.
>>>>>> >> >>> >> > I
>>>>>> >> >>> >> > believe that it's very developers/customers friendly. In
>>>>>> >> >>> >> > addition,
>>>>>> >> >>> >> > we
>>>>>> >> >>> >> > can easily review the translated messages on the github
>>>>>> pull
>>>>>> >> >>> >> > request
>>>>>> >> >>> >> > view and so on. What do you think?
>>>>>> >> >>> >> >
>>>>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull
>>>>>> request.
>>>>>> >> >>> >> >
>>>>>> >> >>> >> > Regards,
>>>>>> >> >>> >> >
>>>>>> >> >>> >> > --
>>>>>> >> >>> >> > Hiroyuki Wada,
>>>>>> >> >>> >> > Developer,
>>>>>> >> >>> >> > Nomura Research Institute, Ltd.
>>>>>> >> >>> >> > _______________________________________________
>>>>>> >> >>> >> > keycloak-dev mailing list
>>>>>> >> >>> >> > keycloak-dev at lists.jboss.org
>>>>>> >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>> >> >>> >>
>>>>>> >> >>> >> --
>>>>>> >> >>> >>
>>>>>> >> >>> >> abstractj
>>>>>> >> >>> >> PGP: 0x84DC9914
>>>>>> >> >>> >> _______________________________________________
>>>>>> >> >>> >> keycloak-dev mailing list
>>>>>> >> >>> >> keycloak-dev at lists.jboss.org
>>>>>> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>> >> >>> >
>>>>>> >> >>> >
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >>
>>>>>> >> >> _______________________________________________
>>>>>> >> >> keycloak-dev mailing list
>>>>>> >> >> keycloak-dev at lists.jboss.org
>>>>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>> >> >
>>>>>> >> >
>>>>>> >> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>>
>>>>>> _______________________________________________
>>>>>> keycloak-dev mailing list
>>>>>> keycloak-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>
>>>>>
>>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160715/fa4323b8/attachment-0001.html 