[jbossws-issues] [JBoss JIRA] Created: (JBWS-1859) Concurrency issue in building the request handler chain when more than one handler is specified

[Bruce (JIRA)]

Concurrency issue in building the request handler chain when more than one handler is specified
-----------------------------------------------------------------------------------------------

                 Key: JBWS-1859
                 URL: http://jira.jboss.com/jira/browse/JBWS-1859
             Project: JBoss Web Services
          Issue Type: Bug
      Security Level: Public (Everyone can see)
          Components: jbossws-jaxws
    Affects Versions: jbossws-2.0.1, jbossws-1.2.1
            Reporter: Bruce


>From what I can tell jbossws has concurrency issues in building the request handler chain when more than one handler is specified, and multiple requests are made to the server while it is starting.  The problem is that too many copies of the handlers are inserted into the handler chain because more than one thread is building it.  This only becomes apparent when one of the handlers is for WS Security.  During request handling, the first WS Security handler removes the wsse header, so any subsequent handler throws an exception.  The client receives the fault string "This service requires <wsse:Security>, which is missing".  The server stack trace:

javax.xml.ws.WebServiceException: org.jboss.ws.core.CommonSOAPFaultException: This service requires <wsse:Security>, which is missing.
        at org.jboss.ws.core.jaxws.handler.HandlerChainExecutor.processHandlerFailure(HandlerChainExecutor.java:276)
        at org.jboss.ws.core.jaxws.handler.HandlerChainExecutor.handleMessage(HandlerChainExecutor.java:155)
        at org.jboss.ws.core.jaxws.handler.HandlerDelegateJAXWS.callRequestHandlerChain(HandlerDelegateJAXWS.java:87)
        at org.jboss.ws.core.server.ServiceEndpointInvoker.callRequestHandlerChain(ServiceEndpointInvoker.java:126)
        at org.jboss.ws.core.server.ServiceEndpointInvoker.invoke(ServiceEndpointInvoker.java:170)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.processRequest(RequestHandlerImpl.java:408)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleRequest(RequestHandlerImpl.java:272)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.doPost(RequestHandlerImpl.java:189)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleHttpRequest(RequestHandlerImpl.java:122)
        at org.jboss.wsf.stack.jbws.EndpointServlet.service(EndpointServlet.java:84)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:175)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:74)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
        at org.jboss.web.tomcat.tc5.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:156)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
        at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
        at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
        at org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
        at java.lang.Thread.run(Thread.java:595)

Steps to duplicate
- Create web service that exposes a ping() method, and requires both WS Security and WS Addressing.  A customized standard-jaxws-endpoint-config.xml file is required.
- Create a client application that calls ping() every second in a loop
- Start the server
- Once the server initializes, start 2 instances of the client
- Restart the server
- Most of the time the error will start to occur for every request.  You may have to restart the server a few time to get the error condition.  Sometimes the server is able to initialize successfully.  If it is successful, it will stay that way.  Even having many clients pounding on the server for hours will not cause the error.
- Once the error condition begins, the server is unable to properly process any more ping requests, even if both clients are stopped, and only one is restarted. 

standard-jaxws-endpoint-config.xml addition:
   <endpoint-config>
      <config-name>WSAddressing + WSSecurity Endpoint</config-name>
      <post-handler-chains>
         <javaee:handler-chain>
            <javaee:protocol-bindings>##SOAP11_HTTP</javaee:protocol-bindings>
            <javaee:handler>
               <javaee:handler-name>WSAddressing Handler</javaee:handler-name>
               <javaee:handler-class>org.jboss.ws.extensions.addressing.jaxws.WSAddressingServerHandler</javaee:handler-class>
            </javaee:handler>
            <javaee:handler>
               <javaee:handler-name>WSSecurity Handler</javaee:handler-name>
               <javaee:handler-class>org.jboss.ws.extensions.security.jaxws.WSSecurityHandlerServer</javaee:handler-class>
            </javaee:handler>
         </javaee:handler-chain>
      </post-handler-chains>
   </endpoint-config>

I have sample server and client code that you can use, but could not see how to attach it to the bug report.

I was able to prevent the error from occurring with a slight modification to callRequestHandlerChain() in org.jboss.ws.core.jaxws.handler.HandlerDelegateJAXWS.  I changed:

if (isInitialized() == false)
{
   resolver.initHandlerChain(sepMetaData, HandlerType.PRE, true);
   resolver.initHandlerChain(sepMetaData, HandlerType.ENDPOINT, true);
   resolver.initHandlerChain(sepMetaData, HandlerType.POST, true);
   setInitialized(true);
}

to 

if (isInitialized() == false)                                                  
{                                                                              
    synchronized(this)                                                         
    {                                                                          
        if (isInitialized() == false)                                          
        {                                                                      
            resolver.initHandlerChain(sepMetaData, HandlerType.PRE, true);     
            resolver.initHandlerChain(sepMetaData, HandlerType.ENDPOINT, true);
            resolver.initHandlerChain(sepMetaData, HandlerType.POST, true);    
            setInitialized(true);                                              
        }                                                                      
    }                                                                          
}  


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira