[gatein-dev] Setting URIEncoding to UTF-8 in server.xml

[Julien Viet]

ideally we should have one entry point for any access to any  
parameter, which will probably be the WebRequest from WCI in the future.

On Mar 18, 2010, at 5:03 PM, Matthew Wringe wrote:

> On Thu, 2010-03-18 at 16:53 +0100, Julien Viet wrote:
>> I'm assuming that UIPortlet do the same and use the query string
>> parser when it constructs the parameter passed to the portlet  
>> invoker.
>>
>>
>> Is that what you mean ?
>
> yes
>
>> On Mar 18, 2010, at 3:19 PM, Matthew Wringe wrote:
>>
>>> On Thu, 2010-03-18 at 15:04 +0100, Julien Viet wrote:
>>>> are we talking here about Portlet parameters or something else ?
>>>
>>> We are talking about request parameters. But if we are getting the
>>> wrong
>>> values from the request parameters, and passing on those values to
>>> the
>>> pc, then we get back wrongly encoded portlet parameters.
>>>
>>>> On Mar 18, 2010, at 2:20 PM, Matthew Wringe wrote:
>>>>
>>>>> On Wed, 2010-03-17 at 23:39 +0100, Julien Viet wrote:
>>>>>> portlet request context are I believe different as they are
>>>>>> never
>>>>>> encoded to anything except they are passed in clear as a kind
>>>>>> of
>>>>>> Map<String,String[]>, so where would be the encoding problem ?
>>>>>
>>>>> there isn't (unless you are passing the wrongly encoded
>>>>> strings).
>>>>> I have a patch ready, I just need to wait until its safe to
>>>>> start
>>>>> committing again.
>>>>>
>>>>>> On Mar 17, 2010, at 11:08 PM, Matthew Wringe wrote:
>>>>>>
>>>>>>> On Wed, 2010-03-17 at 21:38 +0100, Julien Viet wrote:
>>>>>>>> On Mar 17, 2010, at 8:54 PM, Julien Viet wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mar 17, 2010, at 8:48 PM, Matthew Wringe wrote:
>>>>>>>>>
>>>>>>>>>> On Wed, 2010-03-17 at 20:20 +0100, Julien Viet wrote:
>>>>>>>>>>> Hi Matt
>>>>>>>>>>>
>>>>>>>>>>> in WCI there is code to decode query parameters
>>>>>>>>>>> using UTF-8
>>>>>>>>>>> charset:
>>>>>>>>>>>
>>>>>>>>>>> http://anonsvn.jboss.org/repos/gatein/components/wci/trunk/wci/src/main/java/org/gatein/wci/util/RequestDecoder.java
>>>>>>>>>>>
>>>>>>>>>>> that delegates most of the job to :
>>>>>>>>>>> org.gatein.common.http.QueryStringParser
>>>>>>>>>>
>>>>>>>>>> hmm, ok, so it handles the querystring parameters
>>>>>>>>>> directly from
>>>>>>>>>> the
>>>>>>>>>> url
>>>>>>>>>> and not from the request properties. Which means we
>>>>>>>>>> need to
>>>>>>>>>> bypass
>>>>>>>>>> the
>>>>>>>>>> request parameters itself to add this feature. We can
>>>>>>>>>> do this in
>>>>>>>>>> the
>>>>>>>>>> PortalRequestContext [or maybe we should wait to add
>>>>>>>>>> this feature
>>>>>>>>>> until
>>>>>>>>>> a later date? with wci]
>>>>>>>>>
>>>>>>>>> I think you can handle the request query string at any
>>>>>>>>> time, what
>>>>>>>>> needs to be changed is how the portal consumer the query
>>>>>>>>> string
>>>>>>>>> as a
>>>>>>>>> parameter map.
>>>>>>>>>
>>>>>>>>
>>>>>>>> i.e, the request parameter in webui should parse the query
>>>>>>>> string
>>>>>>>> in
>>>>>>>> UTF8 and make them probably available via the class
>>>>>>>> RequestContext:
>>>>>>>>
>>>>>>>> abstract public String getRequestParameter(String name);
>>>>>>>>
>>>>>>>> abstract public String[] getRequestParameterValues(String
>>>>>>>> name);
>>>>>>>>
>>>>>>>> and any call to obtain a request parameter should use
>>>>>>>> this.
>>>>>>>
>>>>>>> that will work for the PortalRequestContext, but not the
>>>>>>> PortletRequestContext (since that uses the portlet responses
>>>>>>> instead
>>>>>>> of
>>>>>>> the http ones).
>>>>>>>
>>>>>>> I have a patch for it that will make the
>>>>>>> PortalRequestContext work
>>>>>>> this
>>>>>>> way, and then makes the UIPortlet use the parameter map from
>>>>>>> the
>>>>>>> PortalRequestContext which is used by the pc to set the
>>>>>>> PortletRequestContext.
>>>>>>>
>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> (somehow the code of WCI: WebRequest and WebResponse
>>>>>>>>>>> should be
>>>>>>>>>>> used in
>>>>>>>>>>> GateIn in a later feature release, as it provides
>>>>>>>>>>> value add
>>>>>>>>>>> services
>>>>>>>>>>> on top of HTTP request/response)
>>>>>>>>>>
>>>>>>>>>> yeah, it probably should, that code isn't being used
>>>>>>>>>> right now
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thomas and myself will probably discuss about that in a
>>>>>>>>> near
>>>>>>>>> future.
>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mar 17, 2010, at 7:36 PM, Matthew Wringe wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On Wed, 2010-03-17 at 18:59 +0100, Julien Viet
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> actually it is servlet container specific so we
>>>>>>>>>>>>> want to avoid
>>>>>>>>>>>>> it.
>>>>>>>>>>>>
>>>>>>>>>>>> ok, so that means we want to handle it in the
>>>>>>>>>>>> portal code
>>>>>>>>>>>> completely
>>>>>>>>>>>> then?
>>>>>>>>>>>>
>>>>>>>>>>>>> where does the encoding issue occur ?
>>>>>>>>>>>>
>>>>>>>>>>>> It occurs when the url is encoded to one value,
>>>>>>>>>>>> but we are
>>>>>>>>>>>> expecting
>>>>>>>>>>>> another. So when the servlet container handles
>>>>>>>>>>>> decoding
>>>>>>>>>>>> something
>>>>>>>>>>>> from
>>>>>>>>>>>> the url (pathInfo, properties from query strings,
>>>>>>>>>>>> etc...) it
>>>>>>>>>>>> will
>>>>>>>>>>>> decode
>>>>>>>>>>>> it using whatever encoding it has been configured
>>>>>>>>>>>> to use
>>>>>>>>>>>> (iso-8859-1
>>>>>>>>>>>> by
>>>>>>>>>>>> default, or another value configured in the
>>>>>>>>>>>> server.xml).
>>>>>>>>>>>>
>>>>>>>>>>>> So we have the situation where something we set
>>>>>>>>>>>> something like
>>>>>>>>>>>> 'thúy' in
>>>>>>>>>>>> the url, the servlet container will take those
>>>>>>>>>>>> bytes and
>>>>>>>>>>>> convert
>>>>>>>>>>>> them
>>>>>>>>>>>> into the iso-8859-1 standard (where the ú gets
>>>>>>>>>>>> converted into
>>>>>>>>>>>> two
>>>>>>>>>>>> bytes), and then the string value we get back is
>>>>>>>>>>>> 'thúy' (it
>>>>>>>>>>>> converts
>>>>>>>>>>>> the two bytes back into utf-8 characters).
>>>>>>>>>>>>
>>>>>>>>>>>> If we set the url encoding to be UTF-8 in the
>>>>>>>>>>>> server.xml,
>>>>>>>>>>>> then we
>>>>>>>>>>>> don't
>>>>>>>>>>>> get these encoding issues since everything is
>>>>>>>>>>>> handled in the
>>>>>>>>>>>> same
>>>>>>>>>>>> encoding.
>>>>>>>>>>>>
>>>>>>>>>>>> For the portal naming issue, I found a way to get
>>>>>>>>>>>> around that
>>>>>>>>>>>> by
>>>>>>>>>>>> using
>>>>>>>>>>>> values not decoded by the servlet container. For
>>>>>>>>>>>> the dashboard
>>>>>>>>>>>> naming
>>>>>>>>>>>> issue, it might be a bit more complex to handle.
>>>>>>>>>>>>
>>>>>>>>>>>> I really wish the javax.servlet classes/spec was
>>>>>>>>>>>> written better
>>>>>>>>>>>> to
>>>>>>>>>>>> handle encoding issue more nicely. Some methods
>>>>>>>>>>>> allow you to
>>>>>>>>>>>> get
>>>>>>>>>>>> the
>>>>>>>>>>>> non-decoded values, other don't. Character
>>>>>>>>>>>> encoding only
>>>>>>>>>>>> works on
>>>>>>>>>>>> non-querystring properties, etc...
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On Mar 17, 2010, at 4:46 PM, Matthew Wringe
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> There have been a few issues which have been
>>>>>>>>>>>>>> brought up in
>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> portal and tab names not working properly if
>>>>>>>>>>>>>> they contain non
>>>>>>>>>>>>>> ascii
>>>>>>>>>>>>>> characters.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://jira.jboss.org/jira/browse/GTNPORTAL-337
>>>>>>>>>>>>>> https://jira.jboss.org/jira/browse/GTNPORTAL-596
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> These are due to the portal using utf-8, while
>>>>>>>>>>>>>> tomcat/jboss
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>> configured to use iso-8859-1 encoding for the
>>>>>>>>>>>>>> url.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In both cases if the URIEncoding is set to
>>>>>>>>>>>>>> UTF-8 in the
>>>>>>>>>>>>>> server.xml
>>>>>>>>>>>>>> (or
>>>>>>>>>>>>>> if useBodyEncodingForURI is set to true) then
>>>>>>>>>>>>>> it works.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> For the portal naming issue it was possible to
>>>>>>>>>>>>>> get around
>>>>>>>>>>>>>> it in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> code, for the tab issue its going to be a bit
>>>>>>>>>>>>>> more complex.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can we just change the URIEncoding in the
>>>>>>>>>>>>>> server.xml for the
>>>>>>>>>>>>>> bundles
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>> create? or would it be better to find
>>>>>>>>>>>>>> workarounds for this
>>>>>>>>>>>>>> issue in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> portal code itself?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> gatein-dev mailing list
>>>>>>>>>>>>>> gatein-dev at lists.jboss.org
>>>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/gatein-dev
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
>