[infinispan-issues] [JBoss JIRA] (ISPN-9701) TransactionTable does not shutdown gracefully

[Dan Berindei (Jira)]

    [ https://issues.jboss.org/browse/ISPN-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13666159#comment-13666159 ] 

Dan Berindei commented on ISPN-9701:
------------------------------------

I haven't yet integrated the backport to 9.3.x because of a test failure:

{noformat}
java.lang.AssertionError
	at org.infinispan.lock.singlelock.AbstractInitiatorCrashTest.testInitiatorCrashesBeforeReleasingLock(AbstractInitiatorCrashTest.java:33)
{noformat}

I think it's an issue with the test though, it assumes that the local tx info is removed before the {{TxCompletionNotificationCommand}} is sent.

> TransactionTable does not shutdown gracefully
> ---------------------------------------------
>
>                 Key: ISPN-9701
>                 URL: https://issues.jboss.org/browse/ISPN-9701
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 9.2.4.Final, 9.3.5.Final, 9.4.1.Final
>            Reporter: Paul Ferraro
>            Assignee: Paul Ferraro
>            Priority: Critical
>             Fix For: 10.0.0.Alpha2, 9.4.3.Final
>
>
> Here's a sample stacktrace during shutdown:
> {noformat}
> 16:54:15,033 WARN  [org.wildfly.clustering.web.undertow] (default task-1) ISPN000472: Cache manager is stopping: org.infinispan.IllegalLifecycleStateException: ISPN000472: Cache manager is stopping
> 	at org.infinispan.marshall.core.GlobalMarshaller.getExternalizer(GlobalMarshaller.java:420)
> 	at org.infinispan.marshall.core.GlobalMarshaller.writeNonNullableObject(GlobalMarshaller.java:400)
> 	at org.infinispan.marshall.core.GlobalMarshaller.writeNullableObject(GlobalMarshaller.java:355)
> 	at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:183)
> 	at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:176)
> 	at org.infinispan.marshall.core.GlobalMarshaller.objectToBuffer(GlobalMarshaller.java:305)
> 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.marshallRequest(JGroupsTransport.java:1009)
> 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.sendCommand(JGroupsTransport.java:1209)
> 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.performAsyncRemoteInvocation(JGroupsTransport.java:1105)
> 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotelyAsync(JGroupsTransport.java:246)
> 	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotelyAsync(RpcManagerImpl.java:291)
> 	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:323)
> 	at org.infinispan.transaction.impl.TransactionTable.removeTransactionInfoRemotely(TransactionTable.java:900)
> 	at org.infinispan.transaction.impl.TransactionTable.releaseLocksForCompletedTransaction(TransactionTable.java:886)
> 	at org.infinispan.transaction.xa.XaTransactionTable.forgetSuccessfullyCompletedTransaction(XaTransactionTable.java:195)
> 	at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:128)
> 	at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68)
> 	at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419)
> 	at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466)
> 	at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335)
> 	at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110)
> {noformat}
> The problem seems to be that shutDownGracefully() first waits for the localTransactions map to be empty.  However, when the cache is clustered, releaseLocksForCompletedTransaction(...) removes the transaction from the localTransactions map *before* invoking removeTransactionInfoRemotely(...), which means that the subsequent TxCompletionNotificationCommand can fail to marshal (see above), or the transport might close before this command is sent.
> A naive fix would simply reorder the removeLocalTransaction(...) to happen after the call to removeTransactionInfoRemotely(...) within the releaseLocksForCompletedTransaction(...) method, but I'm sure there's more to it.



--
This message was sent by Atlassian Jira
(v7.12.1#712002)