[esb-issues] [JBoss JIRA] Commented: (JBESB-3473) Message Loss or premature execution termination in an action during shutdown

[Kevin Conner (JIRA)]

    [ https://jira.jboss.org/browse/JBESB-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12550147#action_12550147 ] 

Kevin Conner commented on JBESB-3473:
-------------------------------------

Changing the MessageAwareListener code to the following should be sufficient.

		if (null != message)
		{
			final Message pipelineMessage = message ;
			final TransactionalRunner txRunner ;
			try
			{
				final Object txHandle = transactionStrategy.suspend();
				txRunner = new TransactionalRunner(pickUpCourier, pipelineMessage, txHandle);
			}
			catch (TransactionStrategyException ex)
			{
				_logger.warn("Caught transaction related exception: ", ex);
				cleanCourier(pickUpCourier);
				rollbackTransaction();
				return ;
			}
			
			updateThreadCount(+1);
			try
			{
				_execService.execute(txRunner);
			}
			catch (final RejectedExecutionException ree)
			{
				txRunner.run() ;
			}
		}

I'll trying to get a test case now, will then test and commit.

Also, using jms-jca-provider rather than jms-provider would use the JCA inflow codebase and not the MessageAwareListener codebase.


> Message Loss or premature execution termination in an action during shutdown
> ----------------------------------------------------------------------------
>
>                 Key: JBESB-3473
>                 URL: https://jira.jboss.org/browse/JBESB-3473
>             Project: JBoss ESB
>          Issue Type: Bug
>      Security Level: Public(Everyone can see) 
>    Affects Versions: 4.9
>         Environment: RHEL 64bit
>            Reporter: Dave Siracusa
>            Assignee: Kevin Conner
>             Fix For: 4.9 CP1
>
>
> I'm able to reproduce JMS message loss or premature action termination during shutdown.  I enqueued several thousand mesages into an ESB application.  While the messages were being processed I toggle (stop/start) lifecycle state for the acting jms listeners via JMX in order to simulate shutdown.  I noticed periodic warnings in the serverlog (listing 1).  The final tally indicated message loss.  Via code inspection the bug appears to be self-evident. 
> Basically when the doStop method (JMXConsole) is called it sets state to STOPPING and immediately terminates the executor thread (MessageAwareListener.java). If the thread was currently processing bad things happen. 
> I'm using transcated queues and it appears the JMS message transaction evaporates and the message is not returned to the queue. I added some code to AbstractThreadedManagedLifecycle.java to remedy the issue.   It uses the default terminationPeriod (60 seconds), unless it is specified in the jboss-esb.xml file.
> AbstractThreadedManagedLifecycle.java
> ---------------------------------------
> protected void doStop()
> throws ManagedLifecycleException
> {
> runningLock.lock() ;
> try
> {
> if (isRunning())
> {
> setRunning(ManagedLifecycleThreadStatae.STOPPING) ;
> }
> // Dave Siracusa -start
> if (!waitUntilStopped())
> {
> throw new ManagedLifecycleException("Thread still active!") ;
> } // Dave Siracusa - end 
> }
> finally
> {
> runningLock.unlock() ;
> }
> } 
> MessageAwareListener.java
> ---------------------------
> protected void doStop()
> throws ManagedLifecycleException
> {
> super.doStop();
> _execService.shutdown() ;
> } 
> Listing 1
> ----------
> 2010-09-01 15:04:40,924 WARN [org.jboss.soa.esb.listeners.lifecycle.AbstractThreadedManagedLifecycle] (Thread-200) Unexpected error from doRun()
> java.util.concurrent.RejectedExecutionException
> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecutionThreadPoolExecutor.java:1768)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at org.jboss.soa.esb.listeners.message.MessageAwareListener.waitForEventAndProcess(MessageAwareListener.java:359)
> at org.jboss.soa.esb.listeners.message.MessageAwareListener.doRun(MessageAwareListener.java:253)
> at org.jboss.soa.esb.listeners.lifecycle.AbstractThreadedManagedLifecycle.run(AbstractThreadedManagedLifecycle.java:115)
> at java.lang.Thread.run(Thread.java:619)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira