[ https://jira.jboss.org/jira/browse/JBAS-5943?page=com.atlassian.jira.plug... ] Scott Marlow updated JBAS-5943: ------------------------------- Workaround Description: Stop the master/singleton node to correct the 02 nodes familyClusterInfo (list of valid servers in the cluster). You can restart the 01 node after the cluster membership has been updated. Description: In a five node cluster {01, 02, 03, 04, 05}, the 01 node was running as the master (all singleton services are running here). The 02 node was making a stateless session bean invocation to the 01 (singleton) node via the PooledInvokerHA transport. Some type of transient failure occurred on the pooled socket connection. The java.io.EOFException occurred while waiting to read the response from the singleton node. The PooledInvokerProxy.invoke wraps all Exceptions as java.rmi.ConnectException which is caught by JRMPInvokerProxyHA. JRMPInvokerProxyHA currently will treat all java.rmi.ConnectException as a reason to fail-over to a different node, however, that is wrong for the case of EOFException (only the socket connection is broken, not the 01 node). As part of the failover handling, the 01 node is removed from the 02 nodes familyClusterInfo which means that the singleton node will no longer be accessible to this singleton session bean on the 02 node. I was able to recreate this with an application test running on my laptop (with two instances of as running) but haven't reproduced the failure with a unit test yet. Exception stacktrace: java.rmi.ConnectException: Failed to communicate; nested exception is: java.io.EOFException at org.jboss.invocation.pooled.interfaces.PooledInvokerProxy.invoke(PooledInvokerProxy.java:628) at org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA.invoke(JRMPInvokerProxyHA.java:200) at org.jboss.invocation.InvokerInterceptor.invokeInvoker(InvokerInterceptor.java:331) at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:194) at org.jboss.proxy.ejb.RetryInterceptor.invoke(RetryInterceptor.java:176) ... [more] Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2498) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) at org.jboss.invocation.pooled.interfaces.PooledInvokerProxy.invoke(PooledInvokerProxy.java:611) ... 102 more was: In a five node cluster {01, 02, 03, 04, 05}, the 01 node was running as the master (all singleton services are running here). The 02 node was making a stateless session bean invocation to the 01 (singleton) node via the PooledInvokerHA transport. Some type of transient failure occurred on the pooled socket connection. The java.io.EOFException occurred while waiting to read the response from the singleton node. The PooledInvokerProxy.invoke wraps all Exceptions as java.rmi.ConnectException which is caught by JRMPInvokerProxyHA. JRMPInvokerProxyHA currently will treat all java.rmi.ConnectException as a reason to fail-over to a different node, however, that is wrong for the case of EOFException (only the socket connection is broken, not the 01 node). I was able to recreate this with an application test running on my laptop (with two instances of as running) but haven't reproduced the failure with a unit test yet. Exception stacktrace: java.rmi.ConnectException: Failed to communicate; nested exception is: java.io.EOFException at org.jboss.invocation.pooled.interfaces.PooledInvokerProxy.invoke(PooledInvokerProxy.java:628) at org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA.invoke(JRMPInvokerProxyHA.java:200) at org.jboss.invocation.InvokerInterceptor.invokeInvoker(InvokerInterceptor.java:331) at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:194) at org.jboss.proxy.ejb.RetryInterceptor.invoke(RetryInterceptor.java:176) ... [more] Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2498) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) at org.jboss.invocation.pooled.interfaces.PooledInvokerProxy.invoke(PooledInvokerProxy.java:611) ... 102 more Workaround: [Workaround Exists] -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JBAS-5943?page=com.atlassian.jira.plug... ] Scott Marlow updated JBAS-5943: ------------------------------- Updated JRMPInvokerProxyHA.java checked into 4_2. Will update trunk in a few days (after freeze for AS5 CR2 ). -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JBAS-5943?page=com.atlassian.jira.plug... ] Scott Marlow closed JBAS-5943. ------------------------------ Resolution: Done -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JBAS-5943?page=com.atlassian.jira.plug... ] Scott Marlow closed JBAS-5943. ------------------------------ Fix Version/s: JBossAS-5.0.0.GA JBossAS-4.2.4.GA Resolution: Done -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira