[esb-issues] [JBoss JIRA] Updated: (JBESB-3473) Message Loss or premature execution termination in an action during shutdown

Message Loss or premature execution termination in an action during shutdown ---------------------------------------------------------------------------- Key: JBESB-3473 URL: https://jira.jboss.org/browse/JBESB-3473 Project: JBoss ESB Issue Type: Bug Security Level: Public(Everyone can see) Affects Versions: 4.9 Environment: RHEL 64bit Reporter: Dave Siracusa Assignee: Kevin Conner Fix For: 4.9 CP1 I'm able to reproduce JMS message loss or premature action termination during shutdown. I enqueued several thousand mesages into an ESB application. While the messages were being processed I toggle (stop/start) lifecycle state for the acting jms listeners via JMX in order to simulate shutdown. I noticed periodic warnings in the serverlog (listing 1). The final tally indicated message loss. Via code inspection the bug appears to be self-evident. Basically when the doStop method (JMXConsole) is called it sets state to STOPPING and immediately terminates the executor thread (MessageAwareListener.java). If the thread was currently processing bad things happen. I'm using transcated queues and it appears the JMS message transaction evaporates and the message is not returned to the queue. I added some code to AbstractThreadedManagedLifecycle.java to remedy the issue. It uses the default terminationPeriod (60 seconds), unless it is specified in the jboss-esb.xml file. AbstractThreadedManagedLifecycle.java --------------------------------------- protected void doStop() throws ManagedLifecycleException { runningLock.lock() ; try { if (isRunning()) { setRunning(ManagedLifecycleThreadStatae.STOPPING) ; } // Dave Siracusa -start if (!waitUntilStopped()) { throw new ManagedLifecycleException("Thread still active!") ; } // Dave Siracusa - end } finally { runningLock.unlock() ; } } MessageAwareListener.java --------------------------- protected void doStop() throws ManagedLifecycleException { super.doStop(); _execService.shutdown() ; } Listing 1 ---------- 2010-09-01 15:04:40,924 WARN [org.jboss.soa.esb.listeners.lifecycle.AbstractThreadedManagedLifecycle] (Thread-200) Unexpected error from doRun() java.util.concurrent.RejectedExecutionException at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecutionThreadPoolExecutor.java:1768) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658) at org.jboss.soa.esb.listeners.message.MessageAwareListener.waitForEventAndProcess(MessageAwareListener.java:359) at org.jboss.soa.esb.listeners.message.MessageAwareListener.doRun(MessageAwareListener.java:253) at org.jboss.soa.esb.listeners.lifecycle.AbstractThreadedManagedLifecycle.run(AbstractThreadedManagedLifecycle.java:115) at java.lang.Thread.run(Thread.java:619)