[jboss-jira] [JBoss JIRA] Commented: (JBREM-821) JBoss Remoting fails under load

[Ron Sigal (JIRA)]

    [ http://jira.jboss.com/jira/browse/JBREM-821?page=comments#action_12389509 ] 
            
Ron Sigal commented on JBREM-821:
---------------------------------

Various changes have been made to org.jboss.remoting.transport.socket.SocketServerInvoker and org.jboss.remoting.transport.socket.ServerThread in response to issue 2) above:

1. ServerThread.processInvocation() has been partiioned into processInvocation() and completeProcessInvocation().  The former reads a version number from the wire, and the latter, which is synchronized, reads and processes the invocation.  

2. ServerThread.shutdown(), which is also synchronized now, closes the socket used by the ServerThread.  Since completeProcessInvocation() is synchronized, shutdown can not close the socket in the middle of an invocation, and it will not return until any invocation currently being processed is complete.  

Since SocketServerInvoker.stop() calls ServerThread.shutdown() on all worker threads, it will not return until

   (a) all invocations are complete, and
   (b) all worker threads are guaranteed to be unable to accept any more invocations.

Testing:

The *ShutdownTestCase unit tests orginally created for JBREM-674 "add test case for client exiting correctly" also serve as unit tests for the current JIRA issue.  The tests are:

   org.jboss.test.remoting.transport.bisocket.BisocketShutdownTestCase
   org.jboss.test.remoting.transport.bisocket.ssl.SSLBisocketShutdownTestCase

   org.jboss.test.remoting.transport.http.HTTPShutdownTestCase
   org.jboss.test.remoting.transport.http.ssl.HTTPSShutdownTestCase

   org.jboss.test.remoting.transport.rmi.RMIShutdownTestCase
   org.jboss.test.remoting.transport.rmi.ssl.SSLRMIShutdownTestCase

   org.jboss.test.remoting.transport.socket.SocketShutdownTestCase
   org.jboss.test.remoting.transport.socket.ssl.SSLSocketShutdownTestCase

A new test method, testOpenClient(), has been added to the root class, org.jboss.test.remoting.shutdown.ShutdownTestParent, which verifies that the server can shut down even if the client does not.

A new test method, testWithServerThreadsInAcknowledge(), has been added to SocketShutdownTestCase, which is the root of the socket and bisocket tests.  testWithServerThreadsInAcknowledge() turns on the connection checking facility, which implies that ServerThreads waiting for an invocation are waiting in ServerThread.acknowledge() instead of ServerThread.readVersion().

> JBoss Remoting fails under load
> -------------------------------
>
>                 Key: JBREM-821
>                 URL: http://jira.jboss.com/jira/browse/JBREM-821
>             Project: JBoss Remoting
>          Issue Type: Bug
>      Security Level: Public(Everyone can see) 
>    Affects Versions: 2.2.2.SP1
>            Reporter: Ovidiu Feodorov
>         Assigned To: Ron Sigal
>             Fix For: 2.4.0.Beta1 (Pinto)
>
>
> This is a duplicate of  JBMESSAGING-1114 present here to remind Remoting people that Messaging has this urgent need.
> Phrasing belongs to Tim:
> JBoss Remoting fails with various different errors when under extreme load.
> To replicate this, set up two clustered server nodes, using a MySQL database.
> These can both be on the same machine, using ServiceBindingManager.
> On a second machine run Ovidiu's messkit toolki, first to send some messages:
> mess -stat send -size 10240 50000
> And then to receive them back using 50 concurrent consumers:
> mess -stat -sessions 50 receive all
> You will notice that JBoss Remoting fails with errors:
> I believe this is due to remoting incorrectly thinking a connection has failed and shutting down the connection. Perhaps due to the load, the ping does not get through in time to refresh the lease?
> I would like a remoting solution that *does not ping* from server to client - for us this is unnecessary.
> It also seems remoting is continually timing out and recreating connections - this could also be a source of error.
> How do we configure remoting so it does not do this?

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira