We have a cluster environment with two nodes (Node1 and Node2) of JBoss 5.1. We have two queues deployed.
When we bring up the Node1 and view consumer count of queue in admin-console it shows consumer count as 1 for each queue. Till now everything fine and work as expected.
When we bring up the Node2 of cluster and view consumer count of queues in admin-console of both the node, we observe a random behavior sometime consumer count is incremented to 2 on Node1 whereas consumer count stays 0 on Node 2 and some time it is increase to 1 on Node2.
When we have consumers for queues on both the node i.e.
Node1 Queue1 consumer count =1
Node1 Queue2 consumer count =1
Node2 Queue1 consumer count=1
Node2 Queue2 consumer count=1
Everything work as expected
BUT
When we have a situation like
Node1 Queue1 consumer count =2
Node1 Queue2 consumer count =1
Node2 Queue1 consumer count=0
Node2 Queue2 consumer count=1
Then all the messages which reach Node2 Queue1 are not processed.
When one of the nodes goes down these failover happens and these messages get processed
This situation is happening very often.
We tried to upgrade to JBoss Messaging 1.4.5 and JBossRemoting 2.5.2.SP3 but it didn’t help.
messaging-service.xml
<?xml version="1.0" encoding="UTF-8"?>
<depends optional-attribute-name="PersistenceManager">jboss.messaging:service=PersistenceManager</depends>
<depends optional-attribute-name="JMSUserManager">jboss.messaging:service=JMSUserManager</depends>
<depends>jboss.messaging:service=Connector,transport=bisocket</depends>
<depends optional-attribute-name="SecurityStore"
proxy-type="org.jboss.jms.server.SecurityStore">jboss.messaging:service=SecurityStore</depends>
</mbean>
</server>