[Remoting] - Re: java.net.SocketException: Connection reset
by mrbalde
Hi Ron,
thanks for your quick answer.
"ron.sigal(a)jboss.com" wrote :
| The question is, then, what is going on at the server? One possibility, if the server is very busy, is that all of the worker threads are in use and the server tries to interrupt one to reuse it. If that's the case, then increasing the size of the worker thread pool might alleviate the problem. In fact, I see you have already increased it from the default size (300) to 400. Did that reduce the frequency of the exception?
|
No, it did not reduced the frequency of the exception.
"ron.sigal(a)jboss.com" wrote :
| It would be very useful to see what's going on at the server by setting the log level for Remoting to TRACE.
|
We have already enabled tracing. But we were not able to find the cause for this problem by analyzing the log file. Maybe you will see something. Here is an excerpt from the log file (I'm sorry for the long post. I was not able to figure out how to attach a file to the message.):
| 2008-10-29 16:40:45,556 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#45[10.10.80.51:1040]:) ***acknowledge read byteWorkerThread#45[10.10.80.51:1040]
| 2008-10-29 16:40:45,556 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,556 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) blocking to read version from input stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) read version 22 from input stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) read InvocationRequest[1e0742, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#31[10.10.80.92:2304]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#31[10.10.80.92:2304]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[1e0742, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa282k-90o3zf-fmw2ck22-1-fmw4wy0v-1jj to subsystem 'AOP'
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#31[10.10.80.92:2304]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@4466b2 from subsystem 'AOP' to client aa282k-90o3zf-fmw2ck22-1-fmw4wy0v-1jj
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@4466b2
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) creating response instance
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) wrote response to the output stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) checking connection
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) read version 22 from input stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) read InvocationRequest[1705018, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#13[10.10.80.56:3710]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#13[10.10.80.56:3710]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[1705018, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa281k-fs0a1l-fmvm5k1x-1-fmw4uslz-2us to subsystem 'AOP'
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#17[10.10.80.56:3709]:) ***acknowledge read byteWorkerThread#17[10.10.80.56:3709]
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) blocking to read version from input stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#34[10.10.80.92:2307]:) ***acknowledge read byteWorkerThread#34[10.10.80.92:2307]
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) blocking to read version from input stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#13[10.10.80.56:3710]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@1cc987c from subsystem 'AOP' to client aa281k-fs0a1l-fmvm5k1x-1-fmw4uslz-2us
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@1cc987c
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) creating response instance
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) wrote response to the output stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) checking connection
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) read version 22 from input stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) read InvocationRequest[fc6855, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#1[10.10.80.128:2350]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#1[10.10.80.128:2350]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[fc6855, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa283k-z1yiwd-fmvlnyoa-1-fmw4wy1s-4c3 to subsystem 'AOP'
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#23[10.10.80.33:3295]:) ***acknowledge read byteWorkerThread#23[10.10.80.33:3295]
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) blocking to read version from input stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#15[10.10.80.128:2351]:) ***acknowledge read byteWorkerThread#15[10.10.80.128:2351]
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) blocking to read version from input stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#1[10.10.80.128:2350]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@146924b from subsystem 'AOP' to client aa283k-z1yiwd-fmvlnyoa-1-fmw4wy1s-4c3
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@146924b
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) creating response instance
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) wrote response to the output stream
| 2008-10-29 16:40:45,612 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) checking connection
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#44[10.10.80.20:4113]:) read version 22 from input stream
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#44[10.10.80.20:4113]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#44[10.10.80.20:4113]:) read InvocationRequest[22f3f1, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@e92749f5], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@7e46abb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#44[10.10.80.20:4113]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#44[10.10.80.20:4113]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@e92749f5], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@7e46abb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#44[10.10.80.20:4113]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[22f3f1, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@e92749f5], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@7e46abb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa28k-nivyng-fmvo6k8i-1-fmw5b7iz-3ql to subsystem 'AOP'
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#44[10.10.80.20:4113]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@186edb7 from subsystem 'AOP' to client aa28k-nivyng-fmvo6k8i-1-fmw5b7iz-3ql
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#44[10.10.80.20:4113]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@186edb7
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#44[10.10.80.20:4113]:) creating response instance
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#44[10.10.80.20:4113]:) wrote response to the output stream
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#44[10.10.80.20:4113]:) checking connection
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#5[10.10.80.63:3768]:) ***acknowledge read byteWorkerThread#5[10.10.80.63:3768]
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,668 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) blocking to read version from input stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) read version 22 from input stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) read InvocationRequest[1a9d137, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@dd0b9400], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5aaec57]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#45[10.10.80.51:1040]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@dd0b9400], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5aaec57]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#45[10.10.80.51:1040]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[1a9d137, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@dd0b9400], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5aaec57]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa281f-haukiu-fmw071xe-1-fmw4wczb-nc to subsystem 'AOP'
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) read version 22 from input stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#8[10.10.80.56:3711]:) ***acknowledge read byteWorkerThread#8[10.10.80.56:3711]
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) blocking to read version from input stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) read InvocationRequest[1108e34, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#17[10.10.80.56:3709]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#17[10.10.80.56:3709]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[1108e34, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa281k-fs0a1l-fmvm5k1x-1-fmw4uslz-2uu to subsystem 'AOP'
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) read version 22 from input stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#45[10.10.80.51:1040]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@13037e4 from subsystem 'AOP' to client aa281f-haukiu-fmw071xe-1-fmw4wczb-nc
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@13037e4
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) creating response instance
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) wrote response to the output stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#45[10.10.80.51:1040]:) checking connection
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#17[10.10.80.56:3709]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@b4c055 from subsystem 'AOP' to client aa281k-fs0a1l-fmvm5k1x-1-fmw4uslz-2uu
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@b4c055
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) creating response instance
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) wrote response to the output stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#17[10.10.80.56:3709]:) checking connection
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#36[10.10.80.92:2305]:) ***acknowledge read byteWorkerThread#36[10.10.80.92:2305]
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) read InvocationRequest[4453c6, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) blocking to read version from input stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#34[10.10.80.92:2307]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#34[10.10.80.92:2307]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[4453c6, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa282k-90o3zf-fmw2ck22-1-fmw4wy4s-1jk to subsystem 'AOP'
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#34[10.10.80.92:2307]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@117d030 from subsystem 'AOP' to client aa282k-90o3zf-fmw2ck22-1-fmw4wy4s-1jk
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@117d030
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) creating response instance
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) wrote response to the output stream
| 2008-10-29 16:40:45,724 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#34[10.10.80.92:2307]:) checking connection
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) read version 22 from input stream
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) read InvocationRequest[31dc14, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#15[10.10.80.128:2351]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#15[10.10.80.128:2351]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[31dc14, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa283k-z1yiwd-fmvlnyoa-1-fmw4wy5p-4c4 to subsystem 'AOP'
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#15[10.10.80.128:2351]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@c792ea from subsystem 'AOP' to client aa283k-z1yiwd-fmvlnyoa-1-fmw4wy5p-4c4
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@c792ea
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) creating response instance
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) wrote response to the output stream
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#15[10.10.80.128:2351]:) checking connection
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) read version 22 from input stream
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) read InvocationRequest[f75d82, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@42807a6d], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@62781f99]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#23[10.10.80.33:3295]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@42807a6d], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@62781f99]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#23[10.10.80.33:3295]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[f75d82, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@42807a6d], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@62781f99]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa28x-u75xh0-fmvmqmqv-1-fmw4r4ql-50t to subsystem 'AOP'
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#1[10.10.80.128:2350]:) ***acknowledge read byteWorkerThread#1[10.10.80.128:2350]
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) blocking to read version from input stream
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#23[10.10.80.33:3295]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@18dc529 from subsystem 'AOP' to client aa28x-u75xh0-fmvmqmqv-1-fmw4r4ql-50t
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@18dc529
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) creating response instance
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) wrote response to the output stream
| 2008-10-29 16:40:45,780 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) checking connection
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) read version 22 from input stream
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) read InvocationRequest[158a06c, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@6d2dbe96], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c5e21fbd]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#5[10.10.80.63:3768]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@6d2dbe96], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c5e21fbd]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#5[10.10.80.63:3768]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[158a06c, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@6d2dbe96], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c5e21fbd]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa281r-izte3s-fmvmrbt7-1-fmw4wzdx-4pz to subsystem 'AOP'
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#5[10.10.80.63:3768]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@d3fdab from subsystem 'AOP' to client aa281r-izte3s-fmvmrbt7-1-fmw4wzdx-4pz
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@d3fdab
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) creating response instance
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) wrote response to the output stream
| 2008-10-29 16:40:45,836 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#5[10.10.80.63:3768]:) checking connection
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) read version 22 from input stream
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#13[10.10.80.56:3710]:) ***acknowledge read byteWorkerThread#13[10.10.80.56:3710]
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) preparing to process next invocation invocation
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#13[10.10.80.56:3710]:) blocking to read version from input stream
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) read InvocationRequest[1d152b8, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#8[10.10.80.56:3711]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#8[10.10.80.56:3711]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[1d152b8, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5ae76643], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@5bfcdfea]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa281k-fs0a1l-fmvm5k1x-1-fmw4uslz-2ut to subsystem 'AOP'
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#8[10.10.80.56:3711]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@1426fdb from subsystem 'AOP' to client aa281k-fs0a1l-fmvm5k1x-1-fmw4uslz-2ut
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@1426fdb
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) creating response instance
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) wrote response to the output stream
| 2008-10-29 16:40:45,892 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#8[10.10.80.56:3711]:) checking connection
| 2008-10-29 16:40:45,949 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) read version 22 from input stream
| 2008-10-29 16:40:45,949 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:45,949 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) read InvocationRequest[1c61422, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:45,949 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:45,949 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#36[10.10.80.92:2305]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:45,949 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#36[10.10.80.92:2305]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[1c61422, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@c8f3f687], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@829d6beb]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa282k-90o3zf-fmw2ck22-1-fmw4wy89-1jl to subsystem 'AOP'
| 2008-10-29 16:40:46,005 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#36[10.10.80.92:2305]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@1ada3d7 from subsystem 'AOP' to client aa282k-90o3zf-fmw2ck22-1-fmw4wy89-1jl
| 2008-10-29 16:40:46,005 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@1ada3d7
| 2008-10-29 16:40:46,005 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) creating response instance
| 2008-10-29 16:40:46,005 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) wrote response to the output stream
| 2008-10-29 16:40:46,005 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#36[10.10.80.92:2305]:) checking connection
| 2008-10-29 16:40:46,005 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#31[10.10.80.92:2304]:) ***acknowledge read byteWorkerThread#31[10.10.80.92:2304]
| 2008-10-29 16:40:46,005 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) preparing to process next invocation invocation
| 2008-10-29 16:40:46,005 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#31[10.10.80.92:2304]:) blocking to read version from input stream
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) read version 22 from input stream
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) blocking to read invocation from unmarshaller
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) read InvocationRequest[a42ad0, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from unmarshaller
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) about to call SocketServerInvoker[10.10.81.1:3873].invoke()
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#1[10.10.80.128:2350]:) SocketServerInvoker[10.10.81.1:3873] received [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#1[10.10.80.128:2350]:) SocketServerInvoker[10.10.81.1:3873] dispatching InvocationRequest[a42ad0, AOP, [advisedMethod=null, unadvisedMethod=null, metadata=[metaData={DISPATCHER={OID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3]}, REMOTING={SUBSYSTEM=[type=AS_ISvalue=AOP], INVOKER_LOCATOR=[type=AS_ISvalue=InvokerLocator [socket://10.10.81.1:3873/?clientMaxPoolSize=300&socket.check_connection=true&timeout=600000]]}, security={principal=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@562e31d6], credential=[type=MARSHALLEDvalue=org.jboss.aop.util.MarshalledValue@f5421e66]}, IS_LOCAL={GUID=[type=AS_ISvalue=jboss.j2ee:ear=zyx.ear,jar=com.xyz.zyx_1.6.15.2.jar,name=ServerTunnelService,service=EJB3,VMID=aea9360e98ebb809:-b827537:11d46c25921:-7ffd]}}], targetObject=null, arguments=null]] from client aa283k-z1yiwd-fmvlnyoa-1-fmw4wy9l-4c5 to subsystem 'AOP'
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#23[10.10.80.33:3295]:) ***acknowledge read byteWorkerThread#23[10.10.80.33:3295]
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) preparing to process next invocation invocation
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#23[10.10.80.33:3295]:) blocking to read version from input stream
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.ServerInvoker] (WorkerThread#1[10.10.80.128:2350]:) SocketServerInvoker[10.10.81.1:3873] successfully dispatched invocation, returning org.jboss.aop.joinpoint.InvocationResponse@12dd397 from subsystem 'AOP' to client aa283k-z1yiwd-fmvlnyoa-1-fmw4wy9l-4c5
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) SocketServerInvoker[10.10.81.1:3873].invoke() returned org.jboss.aop.joinpoint.InvocationResponse@12dd397
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) creating response instance
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) wrote response to the output stream
| 2008-10-29 16:40:46,060 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#1[10.10.80.128:2350]:) checking connection
| 2008-10-29 16:40:46,173 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#41[10.10.80.51:1041]:) ***acknowledge read byteWorkerThread#41[10.10.80.51:1041]
| 2008-10-29 16:40:46,173 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#41[10.10.80.51:1041]:) preparing to process next invocation invocation
| 2008-10-29 16:40:46,173 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#41[10.10.80.51:1041]:) blocking to read version from input stream
| 2008-10-29 16:40:46,173 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#41[10.10.80.51:1041]:) read version -1 from input stream
| 2008-10-29 16:40:46,173 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#41[10.10.80.51:1041]:) EOFException received. This is likely due to client finishing communication.
| java.io.EOFException
| at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:565)
| at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:420)
| at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:173)
| 2008-10-29 16:40:46,173 DEBUG [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#41[10.10.80.51:1041]:) WorkerThread#41[10.10.80.51:1041] closing socketWrapper: ServerSocketWrapper[Socket[addr=/10.10.80.51,port=1041,localport=3873].1866a6f]
| 2008-10-29 16:40:46,173 DEBUG [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#41[10.10.80.51:1041]:) wrote CLOSING
| 2008-10-29 16:40:46,173 DEBUG [org.jboss.remoting.transport.socket.SocketWrapper] (WorkerThread#41[10.10.80.51:1041]:) ServerSocketWrapper[Socket[addr=/10.10.80.51,port=1041,localport=3873].1866a6f] closing
| 2008-10-29 16:40:46,173 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#41[10.10.80.51:1041]:) WorkerThread#41[10.10.80.51:1041] removing itself from clientpool and going to threadpool
| 2008-10-29 16:40:46,173 TRACE [org.jboss.remoting.transport.socket.ServerThread] (WorkerThread#41[10.10.80.51:1041]:) WorkerThread#41[10.10.80.51:1041] begins to wait
| 2008-10-29 16:40:46,173 TRACE [org.jboss.remoting.transport.socket.ServerSocketWrapper] (WorkerThread#5[10.10.80.63:3768]:) ***acknowledge read byteWorkerThread#5[10.10.80.63:3768]
| 2008-10-29 16:40:46,
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4185621#4185621
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4185621
17 years, 6 months
[Performance Tuning] - Re: [JBoss 5.0-CR2] OutOfMemory JVM Crash during JIT
by ceciiil
Hi Peter,
Thanks for that.
To be honest, I strongly doubt there is any problem on the physical server. I have been using it for a couple of months doing intensive testing on my app running on JBoss 4.2.3 with both JRE5 and JRE6 without encountering any JVM problem. Since this JVM crash on JBoss 5 I have switched back to JBoss 423 and again : no JVM problem.
As far as JVM settings are concerned, I am not saying I can explain, I have just observed and reproduced all the items below :
- Xms1024 Xmx1024 PermGen512 = JVM crash at JBoss startup time
- Xms1024 Xmx1024 PermGen256 = JVM crashes with 5 users after 10 mins
- Xms768 Xmx768 PermGen512 = System up to 20 users and then crashes
I have raised the issue to Sun already, waiting for their feedback. If I have anything meaningful back from them, I'll make sure I post it here.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4185618#4185618
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4185618
17 years, 6 months
[Installation, Configuration & DEPLOYMENT] - Re: ObjectName: jboss.remoting:service=Connector, transport=s
by rks_86
File name is:
jboss_EAP4.3_linux.tar.gz
Actually this file is shipped to us by our clients for developmwnt proposes and its a full version with out any limitations.
jboss_EAP4.3_linux.tar size is: 350Mb
Am using Red Hat Enterprise Linux Server release 5.1 (Tikanga) 64 bit
with java version "1.5.0_15"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_15-b04)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_15-b04, mixed mode)
Hey do i need to use any higher version of java..or something like that.
and as you told you downloaded the EAP 4.3 and got folder structure like default,minimal,all and production.
but here i dont have any of these except production which is default that is if you will not mention anything during startup default will go to production as its hard coded in run.sh file.
I need to check these three folders jboss1 jboss2 jboss3 in my soft.
If you get any further info then please do post here.
Thanks.
-Ricky
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4185591#4185591
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4185591
17 years, 6 months