[jboss-jira] [JBoss JIRA] Commented: (JBMESSAGING-1657) IllegalStateException upon cluster fail-over

[Howard Gao (JIRA)]

    [ https://jira.jboss.org/jira/browse/JBMESSAGING-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12472489#action_12472489 ] 

Howard Gao commented on JBMESSAGING-1657:
-----------------------------------------

The test does not always fail, sometimes it is ok.
My observation:

1. client connects to node1, whenever a consumer is created, node1 will tell node2 to do something (in case of topic, node2 will add a binding accordingly). whenever a consumer is closed, node1 will tell node2 to remove the binding. This is achieved through jgroups.

2. when node1 is killed, client failover to node 2 and continue the operation. node2 will get notification of node1's death and do server side failover.
if client requests come in after node1's death but before node2 get notified, node2 still keeps the node1 in its cluster view, so it will try to tell node1 and expect response. But in fact node1 is already dead, so it will throw the exception.

3. actually if we kill node2, there is still a chance to get the same error. 

I'll try to figure out how to fix it and update soon.


> IllegalStateException upon cluster fail-over
> --------------------------------------------
>
>                 Key: JBMESSAGING-1657
>                 URL: https://jira.jboss.org/jira/browse/JBMESSAGING-1657
>             Project: JBoss Messaging
>          Issue Type: Bug
>    Affects Versions: 1.4.0.SP3.CP05
>            Reporter: Justin Bertram
>            Assignee: Howard Gao
>         Attachments: 127.0.0.1.log, 127.0.0.2.log, client.log, JmsTest.java
>
>
> I set up two cluster nodes like so:
> 1) unzipped fresh JBoss EAP 4.3 CP04
> 2) renamed <JBOSS_HOME>/server/default/deploy/jboss-messaging.sar/hsqldb-persistence-service.xml to hsqldb-persistence-service.xml.rej
> 3) copied <JBOSS_HOME>/docs/examples/jms/mysql-persistence-service.xml to <JBOSS_HOME>/server/default/deploy/jboss-messaging.sar
> 4) Modified the "Clustered" attribute to be "true" in <JBOSS_HOME>/server/default/deploy/jboss-messaging.sar/mysql-persistence-service.xml
> 5) copied <JBOSS_HOME>/docs/examples/jca/mysql-ds.xml to <JBOSS_HOME>/server/default/deploy and changed the jndi-name to "DefaultDS" and pointed it at my local MySQL
> 6) copied MySQL driver to <JBOSS_HOME>/server/default/lib
> 7) copied jgroups.jar from <JBOSS_HOME>/server/all/lib to <JBOSS_HOME>/server/default/lib
> 8) copied <JBOSS_HOME> to <JBOSS_HOME2>
> 9) Start first instance with "./run.sh -c default -b 127.0.0.1 -Djboss.messaging.ServerPeerID=1"
> 10) Start second instance with "./run.sh -c default -b 127.0.0.2 -Djboss.messaging.ServerPeerID=2"
> Then I ran the test (which is attached as JmsTest.java)
>  -Start test client connecting to 127.0.0.1
>  -Observe client log filling up with "."
>  -Drop JBoss running on 127.0.0.1 (using kill -9 <pid>)
>  -Client fails with org.jboss.jms.exception.MessagingJMSException (see log attached as client.log)
>  -Observe java.lang.IllegalStateException on the 127.0.0.2 (attached as 127.0.0.2.log)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira