[jboss-remoting-issues] [JBoss JIRA] Updated: (JBREM-1227) Fix deadlock between MicroRemoteClientInvoker and RemotingClassLoader

Fix deadlock between MicroRemoteClientInvoker and RemotingClassLoader --------------------------------------------------------------------- Key: JBREM-1227 URL: https://jira.jboss.org/browse/JBREM-1227 Project: JBoss Remoting Issue Type: Bug Security Level: Public(Everyone can see) Affects Versions: 2.5.2.SP3 (Flounder) Reporter: Ron Sigal Assignee: Ron Sigal Fix For: 2.5.3.SP2 Attachments: deadlock The following deadlock can occur: 1. Thread T1 reads a response with an object whose class is not available locally, so it uses its org.jboss.remoting.loading.RemotingClassLoader to try to download the class a. java.lang.ClassLoader.loadClassInternal() acquires a lock on itself (an instance of RemotingClassLoader) b. RemotingClassLoader uses its copy of an org.jboss.remoting.loading.ClassByteClassLoader to try to download the class c. ClassByteClassLoader calls org.jboss.remoting.MicroRemoteClientInvoker.invoke(), which tries to synchronize on MicroRemoteClientInvoker.class as it sets up an UnMarshaller with a ClassLoader 2. Thread T2 attempts to make an invocation a. MicroRemoteClientInvoker.invoke() acquires a lock on MicroRemoteClientInvoker.class b. MicroRemoteClientInvoker.invoke() calls the synchronized method RemotingClassLoader.setUserClassLoader(), which tries to acquire a lock on the RemotingClassLoader Deadlock. The only two synchronized methods in RemotingClassLoader are setUserClassLoader() and unsetUserClassLoader(), which are both called from MicroRemoteClientInvoker.invoke(). The call to setUserClassLoader() is already done while the MicroRemoteClientInvoker.class lock is held. If the call to unsetUserClassLoader() is made with the same lock, then setUserClassLoader() and unsetClassLoader() don't have to be synchronized. That will eliminate the deadlock because step 2b is removed.