[jboss-jira] [JBoss JIRA] (WFLY-10075) [Artemis 2.x upgrade] Stuck messages in artemis.internal.sf.my-cluster... queue after restarting nodes in cluster

[Artemis 2.x upgrade] Stuck messages in artemis.internal.sf.my-cluster... queue after restarting nodes in cluster ----------------------------------------------------------------------------------------------------------------- Key: WFLY-10075 URL: https://issues.jboss.org/browse/WFLY-10075 Project: WildFly Issue Type: Bug Components: JMS Reporter: Miroslav Novak Assignee: Jeff Mesnil Priority: Blocker Labels: activemq, feature-branch-blocker Attachments: journal-node-2.txt There are lost messages in scenario where nodes in cluster are cleanly stopped and started again. This issue was hit with Artemis 2.5.0.Final and WF Jeff's integration branch WFLY-9407_upgrade_artemis_2.4.0_with_prefix. Test Scenario: * start two servers in cluster (JGroups used for discovery) * send messages to testQueue0 on node-1 and node-2 * wait until consumers on both nodes receive 300 messages * cleanly shut down 1st and then 2nd server * leave servers shut down for one minute * start both servers * wait until both consumers receive 500 messages * stop sending messages and receive all remaining messages Pass Criteria: All send messages are received by consumer Actual Result: There are lost messages. Investigation: There are lost messages which were sent to 2nd node. However they got stuck in queue {{.artemis.internal.sf.my-cluster.8a7e9e98-2c36-11e8-9737-fa163ea20b26}} during load balancing to 1st server. I'm attaching trace logs from client and servers and content of journal from 2nd server. This is regression against Artemis 1.5.5 thus setting blocker priority.