[jboss-jira] [JBoss JIRA] Commented: (JBAS-5670) Using MessageConsumers with Message Selector make messages stuck in queue

[Adrian Brock (JIRA)]

    [ http://jira.jboss.com/jira/browse/JBAS-5670?page=comments#action_12418730 ] 
            
Adrian Brock commented on JBAS-5670:
------------------------------------

This is a race condition between the MessageConsumer.close() and
the asynchronous delivery to the message listener thread.

Selectors have nothing to do with the issue.

The race goes something like the following:

1) MessageListener: asks server for a message which responds with none so it waits
2) Server: message arrives so it pushes it to the client
3) Main: MessageConsumer.close()

At this point three threads are contending for control of the consumer.

In the bad case (other orders are ok), the race goes something like this:

4) Server: add message to delivery queue (MessageListener is still listening)
5) Main: Closes the message listener
6) MessageListener: spots it is now closed and ends

But there is still a message in its delivery queue from step (4)

Since the message is now stuck in an orphaned queue of the closed MessageListener, only
a Connection.close() would NACK the message and make it available for redelivery,
which is not correct behaviour.

You can see this on the server when it reaches a "steady state", 
with the number of subscriptions equal to the number receivers + the number of "in process" messages. 

The extra subscriptions are in a "delayed removal" state. 
Although they are closed, the server is waiting for the client to NACK 
(or possibly in an XA environment ACK)  the unacknowledged messages.

The fix is to make MessageConsumer.close() NACK any messages in the delivery queue
(those that will never actually get delivered) when it wins the race between steps (5) and (6)

Index: SpyMessageConsumer.java
===================================================================
--- SpyMessageConsumer.java     (revision 74899)
+++ SpyMessageConsumer.java     (working copy)
@@ -518,20 +518,35 @@
 
    public void close() throws JMSException
    {
       synchronized (messages)
       {
          if (closed.set(true))
             return;
 
          if (trace)      
             log.trace("Message consumer closing. " + this);
+         
+         while (messages.isEmpty() == false)
+         {
+            SpyMessage mes = (SpyMessage) messages.removeFirst();
+            if (trace)
+               log.trace("close() nacking undelivered message mes=" + mes.getJMSMessageID() + " " + this); 
+            try
+            {
+               session.connection.send(mes.getAcknowledgementRequest(false));
+            }
+            catch (Exception e)
+            {
+               log.debug("Error nacking message: " + mes.getJMSMessageID(), e);
+            }
+         }
          messages.notifyAll();
       }
       
       // Notification to break out of delivery lock loop
       session.interruptDeliveryLockWaiters();
 
       if (listenerThread != null && !Thread.currentThread().equals(listenerThread))
       {
          try
          {


> Using MessageConsumers with Message Selector make messages stuck in queue
> -------------------------------------------------------------------------
>
>                 Key: JBAS-5670
>                 URL: http://jira.jboss.com/jira/browse/JBAS-5670
>             Project: JBoss Application Server
>          Issue Type: Bug
>      Security Level: Public(Everyone can see) 
>          Components: JMS (JBossMQ)
>    Affects Versions: JBossAS-4.2.2.GA
>         Environment: Java 5.0, WinXP
>            Reporter: Kevin Lohmann
>         Assigned To: Adrian Brock
>         Attachments: Situation.zip
>
>
> Hi *,
> here's the situation:
> We have JBossAS 4.2.2 with no changes. We're using the example queue 'A' (test with a new queue was not successfull) and transacted sessions.
> Every MessageConsumer and MessageProducer share the same Connection-object (changing this didn't help) and have their own Session-object.
> We use MessageConsumers with a MessageSelector and have to change the selector after a while. So that means closing the old consumer and create an new consumer with a new selector.
> First we sent 10.000 messages through a MessageProducer and _after_ the messages have arrived we started two consumers (and changed the selector again and again). Everything works fine: The programm ends, all messages sent are recieved.
> But then the MessageProducer runs parallel as a seperate thread. 
> What happens?
> The programm doesn't quite (because not all messages are recieved) and the jmx-console shows that a few messages are stuck in the "InProcess"-list. When you now close the programm the messages return to the queue.
> We tried that with not using selectors and everything worked fine. But we need to use selectors.
> I attached the test-programm.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira