[keycloak-dev] Support UTF-8 encoding theme message properites

Fri Jul 15 02:14:55 EDT 2016

BTW do you have a reliable way to convert the files? I've tried various
tools, but none worked for me.

On 15 July 2016 at 08:14, Stian Thorgersen <sthorger at redhat.com> wrote:

> I'd like the current messages to be converted by the people that have
> contributed them. They will be able to determine what encoding they prefer
> to use and also confirm that the conversion has been successful.
>
> On 15 July 2016 at 07:59, Hiroyuki Wada <wadahiro at gmail.com> wrote:
>
>> I mean I'd like to change the message files encoding only.
>> For example, admin-messages for Russian language is currently written in
>> ISO-8859-1 encoding.
>>
>>
>> https://github.com/keycloak/keycloak/blob/master/themes/src/main/resources/theme/base/admin/messages/admin-messages_ru.properties
>>
>> Can I convert the language messages except for English?
>>
>>
>> 2016年7月15日(金) 14:03 Stian Thorgersen <sthorger at redhat.com>:
>>
>>> No, we don't want the theme properties (and other properties files) to
>>> be utf-8. They should stick with the default encoding used by Java
>>> properties files.
>>>
>>> Messages are a special case, which is why we accepted it for them.
>>>
>>> On 14 July 2016 at 13:29, Hiroyuki Wada <wadahiro at gmail.com> wrote:
>>>
>>>> Thank you for merging the pull request. Could I send a pull request
>>>> converting to UTF-8 encoding against the theme properties?
>>>>
>>>> 2016年7月7日(木) 21:39 Hiroyuki Wada <wadahiro at gmail.com>:
>>>>
>>>>> Yes. I will send a pull request later!
>>>>>
>>>>> 2016年7月7日(木) 21:20 Stian Thorgersen <sthorger at redhat.com>:
>>>>>
>>>>>> Are you doing a PR as well?
>>>>>>
>>>>>> On 7 July 2016 at 12:33, Hiroyuki Wada <wadahiro at gmail.com> wrote:
>>>>>>
>>>>>>> Ok. I created a JIRA issue.
>>>>>>> https://issues.jboss.org/browse/KEYCLOAK-3278
>>>>>>>
>>>>>>> 2016年7月6日(水) 19:36 Stian Thorgersen <sthorger at redhat.com>:
>>>>>>>
>>>>>>>> Let's go for it. Please create a JIRA and send PR. Please just use
>>>>>>>> "# encoding: utf-8" and not "# -*- coding: utf-8 -*-". The latter
>>>>>>>> is just weird if you ask me.
>>>>>>>>
>>>>>>>> Thomas: It's optional, defaults to iso-8859-1 and the header will
>>>>>>>> clearly state that a file uses a different encoding, so I disagree that
>>>>>>>> there will be any confusion.
>>>>>>>>
>>>>>>>> On 6 July 2016 at 12:30, Hiroyuki Wada <wadahiro at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> On Wed, Jul 6, 2016 at 6:53 PM, Thomas Raehalme
>>>>>>>>> <thomas.raehalme at aitiofinland.com> wrote:
>>>>>>>>> > From the javadoc of java.util.ResourceBundle the encoding for
>>>>>>>>> .properties is
>>>>>>>>> > still expected to be ISO-8859-1:
>>>>>>>>> >
>>>>>>>>> >> Constructing a PropertyResourceBundle instance from an
>>>>>>>>> InputStream
>>>>>>>>> >> requires that the input stream be encoded in ISO-8859-1. In
>>>>>>>>> that case,
>>>>>>>>> >> characters that cannot be represented in ISO-8859-1 encoding
>>>>>>>>> must be
>>>>>>>>> >> represented by Unicode Escapes as defined in section 3.3 of The
>>>>>>>>> Java™
>>>>>>>>> >> Language Specification whereas the other constructor which
>>>>>>>>> takes a Reader
>>>>>>>>> >> does not have that limitation.
>>>>>>>>>
>>>>>>>>> It's a case of using constructor which takes an InputStream.
>>>>>>>>> It says we can use any encoding when we use the other constructor
>>>>>>>>> which takes a Reader.
>>>>>>>>>
>>>>>>>>> I understand your concern. So what do you think supporting utf-8
>>>>>>>>> with
>>>>>>>>> a header like Stian suggested?
>>>>>>>>> I think it can avoid confusion because the encoding is noted in the
>>>>>>>>> file itself...
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> >
>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/PropertyResourceBundle.html
>>>>>>>>> >
>>>>>>>>> > I understand that Spring allows one to read .properties using
>>>>>>>>> any encoding,
>>>>>>>>> > but you need to specify the encoding in Spring configuration
>>>>>>>>> external to the
>>>>>>>>> > .properties file.
>>>>>>>>> >
>>>>>>>>> > In my opinion it would cause unnecessary confusion amongst
>>>>>>>>> developers to use
>>>>>>>>> > any other encoding than the one defined by the official
>>>>>>>>> documentation for
>>>>>>>>> > Properties.
>>>>>>>>> >
>>>>>>>>> > Best regards,
>>>>>>>>> > Thomas
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Wed, Jul 6, 2016 at 12:40 PM, Hiroyuki Wada <
>>>>>>>>> wadahiro at gmail.com> wrote:
>>>>>>>>> >>
>>>>>>>>> >> I think it was true before Java 1.6 because Java standard
>>>>>>>>> library
>>>>>>>>> >> (java.util.ResourceBundle or java.util.Properties) only
>>>>>>>>> supported
>>>>>>>>> >> ISO-8859-1 encoding file.
>>>>>>>>> >> But they support any encoding after Java 1.6. There are some
>>>>>>>>> >> frameworks which can read .properties with any encoding. For
>>>>>>>>> example,
>>>>>>>>> >> Spring Framework can read .properties with any encoding. I
>>>>>>>>> think it's
>>>>>>>>> >> very useful to read UTF-8 directly for multibyte language
>>>>>>>>> country.
>>>>>>>>> >>
>>>>>>>>> >> On Wed, Jul 6, 2016 at 3:32 PM, Thomas Raehalme
>>>>>>>>> >> <thomas.raehalme at aitiofinland.com> wrote:
>>>>>>>>> >> > Please correct me if I am wrong, but I have been under the
>>>>>>>>> impression
>>>>>>>>> >> > that
>>>>>>>>> >> > Java .properties files should always use encoding ISO-8859-1.
>>>>>>>>> Characters
>>>>>>>>> >> > not
>>>>>>>>> >> > present in ISO-8859-1 can be written in \uxxxx. Won't it make
>>>>>>>>> things
>>>>>>>>> >> > confusing to developers if another encoding is used here
>>>>>>>>> instead?
>>>>>>>>> >> >
>>>>>>>>> >> >
>>>>>>>>> >> >
>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#load-java.io.InputStream-
>>>>>>>>> >> >
>>>>>>>>> >> >
>>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/util/Properties.html#store-java.io.OutputStream-java.lang.String-
>>>>>>>>> >> >
>>>>>>>>> >> > If alternate encodings are desired how about supporting the
>>>>>>>>> XML format
>>>>>>>>> >> > of
>>>>>>>>> >> > Properties?
>>>>>>>>> >> >
>>>>>>>>> >> > Best regards,
>>>>>>>>> >> > Thomas
>>>>>>>>> >> >
>>>>>>>>> >> >
>>>>>>>>> >> > On Wed, Jul 6, 2016 at 9:04 AM, Stian Thorgersen <
>>>>>>>>> sthorger at redhat.com>
>>>>>>>>> >> > wrote:
>>>>>>>>> >> >>
>>>>>>>>> >> >> Both iso-8859-1 and utf-8 message bundles should be able to
>>>>>>>>> co-exist.
>>>>>>>>> >> >>
>>>>>>>>> >> >> We can allow specifying the encoding in a comment on the
>>>>>>>>> first line
>>>>>>>>> >> >> like
>>>>>>>>> >> >> this:
>>>>>>>>> >> >>
>>>>>>>>> >> >> # encoding=utf-8
>>>>>>>>> >> >> key=value
>>>>>>>>> >> >>
>>>>>>>>> >> >> # encoding=iso-8859-1
>>>>>>>>> >> >> key=value
>>>>>>>>> >> >>
>>>>>>>>> >> >> If the first line in the file doesn't contain the comment
>>>>>>>>> with the
>>>>>>>>> >> >> encoding then we should default to iso-8859-1 for backwards
>>>>>>>>> >> >> compatibility
>>>>>>>>> >> >>
>>>>>>>>> >> >> On 5 July 2016 at 09:49, Hiroyuki Wada <wadahiro at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> >> >>>
>>>>>>>>> >> >>> Thanks for your comment.
>>>>>>>>> >> >>>
>>>>>>>>> >> >>> > If we want to change to utf-8 we'd still need to support
>>>>>>>>> iso.. for
>>>>>>>>> >> >>> > backwards compatibility.
>>>>>>>>> >> >>>
>>>>>>>>> >> >>> If we change to UTF-8, we can still read unicode codepoint
>>>>>>>>> like
>>>>>>>>> >> >>> '\u00e8'.
>>>>>>>>> >> >>> There is an incompatibility when non-ascii characters are
>>>>>>>>> used in
>>>>>>>>> >> >>> message properties.
>>>>>>>>> >> >>> The non-ascii characters are 0xA0 - 0xFF codes (please
>>>>>>>>> refer codepage
>>>>>>>>> >> >>> layout:
>>>>>>>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Codepage_layout )
>>>>>>>>> >> >>>
>>>>>>>>> >> >>> I think the non-ascii characters might be used in French
>>>>>>>>> messages like
>>>>>>>>> >> >>> 'à' so I agree to support ISO-8859-1 for backwards
>>>>>>>>> compatibility.
>>>>>>>>> >> >>> To support this, I think we can add a property like
>>>>>>>>> "messageEncoding"
>>>>>>>>> >> >>> in keycloak-server.json as below. Is it a good idea?
>>>>>>>>> >> >>>
>>>>>>>>> >> >>>     "theme": {
>>>>>>>>> >> >>>         "staticMaxAge": 2592000,
>>>>>>>>> >> >>>         "cacheTemplates": true,
>>>>>>>>> >> >>>         "cacheThemes": true,
>>>>>>>>> >> >>>         "messageEncoding": "ISO-8859-1",
>>>>>>>>> >> >>>         "folder": {
>>>>>>>>> >> >>>           "dir": "${jboss.home.dir}/themes"
>>>>>>>>> >> >>>         }
>>>>>>>>> >> >>>     },
>>>>>>>>> >> >>>
>>>>>>>>> >> >>> Regards,
>>>>>>>>> >> >>>
>>>>>>>>> >> >>> On Mon, Jul 4, 2016 at 10:33 PM, Stian Thorgersen
>>>>>>>>> >> >>> <sthorger at redhat.com>
>>>>>>>>> >> >>> wrote:
>>>>>>>>> >> >>> > We have in the past discussed this and decided to stick
>>>>>>>>> with
>>>>>>>>> >> >>> > ISO-8859-1.
>>>>>>>>> >> >>> > That was probably not the best idea though. If we want to
>>>>>>>>> change to
>>>>>>>>> >> >>> > utf-8
>>>>>>>>> >> >>> > we'd still need to support iso.. for backwards
>>>>>>>>> compatibility.
>>>>>>>>> >> >>> >
>>>>>>>>> >> >>> > On 4 July 2016 at 14:37, Bruno Oliveira <
>>>>>>>>> bruno at abstractj.org> wrote:
>>>>>>>>> >> >>> >>
>>>>>>>>> >> >>> >> It makes sense, maybe file a Jira associated with:
>>>>>>>>> >> >>> >> https://issues.jboss.org/browse/KEYCLOAK-3259 ?
>>>>>>>>> >> >>> >>
>>>>>>>>> >> >>> >> On 2016-07-04, Hiroyuki Wada wrote:
>>>>>>>>> >> >>> >> > Hello all,
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> > I am trying to translate all base theme messages to my
>>>>>>>>> country
>>>>>>>>> >> >>> >> > language, Japanese. And I'd like to contribute them.
>>>>>>>>> Before that
>>>>>>>>> >> >>> >> > work,
>>>>>>>>> >> >>> >> > I'd like to propose about the files encoding.
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> > Currently, the message files (*.properties) are loaded
>>>>>>>>> with
>>>>>>>>> >> >>> >> > ISO-8859-1
>>>>>>>>> >> >>> >> > encoding. Therefore, it is necessary to convert the
>>>>>>>>> files by
>>>>>>>>> >> >>> >> > 'native2ascii' command beforehand. However we can
>>>>>>>>> directly read
>>>>>>>>> >> >>> >> > the
>>>>>>>>> >> >>> >> > property files with UTF-8 encoding in java 1.6 or
>>>>>>>>> later because
>>>>>>>>> >> >>> >> > 'java.util.Properties#load(java.io.Reader)' method was
>>>>>>>>> introduced
>>>>>>>>> >> >>> >> > as
>>>>>>>>> >> >>> >> > below.
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> >
>>>>>>>>> http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html#load(java.io.Reader)
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> > So, my proposal is supporting the message files with
>>>>>>>>> UTF-8
>>>>>>>>> >> >>> >> > encoding.
>>>>>>>>> >> >>> >> > I
>>>>>>>>> >> >>> >> > believe that it's very developers/customers friendly.
>>>>>>>>> In
>>>>>>>>> >> >>> >> > addition,
>>>>>>>>> >> >>> >> > we
>>>>>>>>> >> >>> >> > can easily review the translated messages on the
>>>>>>>>> github pull
>>>>>>>>> >> >>> >> > request
>>>>>>>>> >> >>> >> > view and so on. What do you think?
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> > If it's ok, I'll create a JIRA issue and create a pull
>>>>>>>>> request.
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> > Regards,
>>>>>>>>> >> >>> >> >
>>>>>>>>> >> >>> >> > --
>>>>>>>>> >> >>> >> > Hiroyuki Wada,
>>>>>>>>> >> >>> >> > Developer,
>>>>>>>>> >> >>> >> > Nomura Research Institute, Ltd.
>>>>>>>>> >> >>> >> > _______________________________________________
>>>>>>>>> >> >>> >> > keycloak-dev mailing list
>>>>>>>>> >> >>> >> > keycloak-dev at lists.jboss.org
>>>>>>>>> >> >>> >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>>> >> >>> >>
>>>>>>>>> >> >>> >> --
>>>>>>>>> >> >>> >>
>>>>>>>>> >> >>> >> abstractj
>>>>>>>>> >> >>> >> PGP: 0x84DC9914
>>>>>>>>> >> >>> >> _______________________________________________
>>>>>>>>> >> >>> >> keycloak-dev mailing list
>>>>>>>>> >> >>> >> keycloak-dev at lists.jboss.org
>>>>>>>>> >> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>>> >> >>> >
>>>>>>>>> >> >>> >
>>>>>>>>> >> >>
>>>>>>>>> >> >>
>>>>>>>>> >> >>
>>>>>>>>> >> >> _______________________________________________
>>>>>>>>> >> >> keycloak-dev mailing list
>>>>>>>>> >> >> keycloak-dev at lists.jboss.org
>>>>>>>>> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>>> >> >
>>>>>>>>> >> >
>>>>>>>>> >> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> keycloak-dev mailing list
>>>>>>>>> keycloak-dev at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160715/98e7fa42/attachment-0001.html 