[JBoss JIRA] (JBTM-2583) Use the local ActionStatusService to determine if a transaction containing XAResources is still in-flight before relying on orphan detection
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/JBTM-2583?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on JBTM-2583:
-----------------------------------------------
Petr Penicka <ppenicka(a)redhat.com> changed the Status of [bug 1310603|https://bugzilla.redhat.com/show_bug.cgi?id=1310603] from VERIFIED to CLOSED
> Use the local ActionStatusService to determine if a transaction containing XAResources is still in-flight before relying on orphan detection
> --------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: JBTM-2583
> URL: https://issues.jboss.org/browse/JBTM-2583
> Project: JBoss Transaction Manager
> Issue Type: Enhancement
> Components: Recovery
> Reporter: Tom Jenkinson
> Assignee: Tom Jenkinson
> Fix For: 4.17.34, 5.2.17.Final, 5.3.0.Final
>
>
> Currently we use a timeout based system to determine if prepared Xids that a ResourceManager knows about but where the transaction is not prepared yet are the result of a pre-prepare crash or whether it is just slow progress of the resources/transaction manager.
> This issue is to record an enhancement to the recovery manager for XAResources to attempt to contact the transaction manager to determine if an Xid is indoubt before rolling it back.
>
> * In the case where the recovery manager and transaction manager are co-located this negates the need for a timeout based process entirely
> * In the case where the recovery manager and transaction manager are not in the same JVM process, the current behaviour of timeout based orphan detection MUST still be employed
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (JBTM-2579) Throw XAException in XATerminator::commit if a wrapped resource fails transiently
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/JBTM-2579?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on JBTM-2579:
-----------------------------------------------
Petr Penicka <ppenicka(a)redhat.com> changed the Status of [bug 1329142|https://bugzilla.redhat.com/show_bug.cgi?id=1329142] from VERIFIED to CLOSED
> Throw XAException in XATerminator::commit if a wrapped resource fails transiently
> ---------------------------------------------------------------------------------
>
> Key: JBTM-2579
> URL: https://issues.jboss.org/browse/JBTM-2579
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Components: JTA
> Reporter: Tom Jenkinson
> Assignee: Tom Jenkinson
> Priority: Critical
> Fix For: 4.17.31, 5.2.9.Final
>
>
> It is possible for a resource that we are wrapping to return say XA_RETRY or XA_RMFAIL and therefore end up in the BasicAction failed list. However there is no error returned from commit in this circumstance as the recovery manager should ensure a consistent outcome.
> The reason this becomes a problem for JTA and XATerminator in particular is that as no error is returned a parent coordinator will assume the branch completed successfully. In the future when it calls XATerminator::recover though this branch will be returned and detected as an orphan.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (JBTM-2840) Add new suppressed exception test
by Tom Jenkinson (JIRA)
Tom Jenkinson created JBTM-2840:
-----------------------------------
Summary: Add new suppressed exception test
Key: JBTM-2840
URL: https://issues.jboss.org/browse/JBTM-2840
Project: JBoss Transaction Manager
Issue Type: Task
Reporter: Tom Jenkinson
Priority: Minor
Show that initCause messages in the XAException can be read and show how integration code could access them.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (JBTM-2837) Server being stuck during shudown when transaction probe op is called
by Tom Jenkinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-2837?page=com.atlassian.jira.plugin.... ]
Tom Jenkinson commented on JBTM-2837:
-------------------------------------
Regarding option 2, unless necessary we should avoid upgrading to Narayana 6 in EAP 7.1 just to resolve this issue. What would happen if we called periodicWorkSecondPass during shutdown with the normal interface (rather than the new interface)?
> Server being stuck during shudown when transaction probe op is called
> ---------------------------------------------------------------------
>
> Key: JBTM-2837
> URL: https://issues.jboss.org/browse/JBTM-2837
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Components: Recovery
> Affects Versions: 5.5.0.Final
> Reporter: Ondra Chaloupka
> Assignee: Michael Musgrove
> Attachments: server.log, stacktrace1.log
>
>
> I do experience server to be stuck (in intermittent way) during shutdown. I need to use afterwards {{kill -9}} to stop it.
> From my investigation it seems that it's caused by fact that {{jboss-cli}} operation {{:probe}} does cause {{XARecoveryModule.periodicWorkFirstPass}} being called
> https://github.com/jbosstm/narayana/blob/master/ArjunaJTA/jta/classes/com...
> That way {{scanState}} is left at value ScanStates.BETWEEN_PASSES [1]
> https://github.com/jbosstm/narayana/blob/master/ArjunaJTA/jta/classes/com...
> Now when shutdown of container is being run (you can check attached {{server.log}})
> the periodic recovery is stopped and the {{scanState}} is left as it is. At that point
> call of {{XARecoveryModule#removeXAResourceRecoveryHelper}} causes thread waits for state {{IDLE}} ifinintely. https://github.com/jbosstm/narayana/blob/master/ArjunaJTA/jta/classes/com...
> You can consult thread dump from time when server gets stuck {{stacktrace1.log}}.
> [1]
> {code}
> Thread [management-handler-thread - 5] (Suspended (breakpoint at line 149 in XARecoveryModule))
> owns: XARecoveryModule (id=219)
> owns: AtomicAction (id=844)
> owns: ObjStoreBrowser (id=845)
> XARecoveryModule.periodicWorkFirstPass() line: 149
> XARecoveryModule.getNewXAResource(Xid) line: 272
> XARecoveryModule.getNewXAResource(XAResourceRecord) line: 310
> XAResourceRecord.getNewXAResource() line: 1303
> XAResourceRecord.restore_state(InputObjectState, int) line: 1054
> AtomicAction(BasicAction).restore_state(InputObjectState, int) line: 1180
> AtomicAction(BasicAction).activate(String) line: 488
> AtomicAction(BasicAction).activate() line: 451
> ActionBean$GenericAtomicActionWrapper.activate() line: 391
> ActionBean.createWrapper(UidWrapper, boolean) line: 107
> ActionBean.<init>(UidWrapper) line: 93
> NativeConstructorAccessorImpl.newInstance0(Constructor<?>, Object[]) line: not available [native method]
> NativeConstructorAccessorImpl.newInstance(Object[]) line: 62
> DelegatingConstructorAccessorImpl.newInstance(Object[]) line: 45
> Constructor<T>.newInstance(Object...) line: 423
> UidWrapper.createMBean() line: 196
> ObjStoreBrowser.createBean(Uid, String) line: 481
> ObjStoreBrowser.probe() line: 435
> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> Method.invoke(Object, Object...) line: 498
> Trampoline.invoke(Method, Object, Object[]) line: 71
> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> Method.invoke(Object, Object...) line: 498
> MethodUtil.invoke(Method, Object, Object[]) line: 275
> StandardMBeanIntrospector.invokeM2(Method, Object, Object[], Object) line: 112
> StandardMBeanIntrospector.invokeM2(Object, Object, Object[], Object) line: 46
> StandardMBeanIntrospector(MBeanIntrospector<M>).invokeM(M, Object, Object[], Object) line: 237
> PerInterface<M>.invoke(Object, String, Object[], String[], Object) line: 138
> StandardMBeanSupport(MBeanSupport<M>).invoke(String, Object[], String[]) line: 252
> DefaultMBeanServerInterceptor.invoke(ObjectName, String, Object[], String[]) line: 819
> JmxMBeanServer.invoke(ObjectName, String, Object[], String[]) line: 801
> PluggableMBeanServerImpl$TcclMBeanServer.invoke(ObjectName, String, Object[], String[]) line: 1512
> PluggableMBeanServerImpl.invoke(ObjectName, String, Object[], String[]) line: 731
> LogStoreProbeHandler.probeTransactions(MBeanServer, boolean) line: 157
> LogStoreProbeHandler.execute(OperationContext, ModelNode) line: 186
> OperationContextImpl(AbstractOperationContext).executeStep(AbstractOperationContext$Step) line: 921
> OperationContextImpl(AbstractOperationContext).processStages() line: 664
> OperationContextImpl(AbstractOperationContext).executeOperation() line: 383
> OperationContextImpl.executeOperation() line: 1390
> ModelControllerImpl.internalExecute(ModelNode, OperationMessageHandler, ModelController$OperationTransactionControl, OperationAttachments, OperationStepHandler, boolean, boolean) line: 419
> ModelControllerImpl.lambda$execute$1(Operation, OperationMessageHandler, ModelController$OperationTransactionControl) line: 240
> 277244299.run() line: not available
> SecurityIdentity.runAs(PrivilegedAction<T>) line: 193
> ModelControllerImpl.execute(Operation, OperationMessageHandler, ModelController$OperationTransactionControl) line: 240
> ModelControllerClientOperationHandler$ExecuteRequestHandler.doExecute(ModelNode, int, ManagementRequestContext<Void>, CompletedCallback) line: 217
> ModelControllerClientOperationHandler$ExecuteRequestHandler.access$400(ModelControllerClientOperationHandler$ExecuteRequestHandler, ModelNode, int, ManagementRequestContext, ModelControllerClientOperationHandler$CompletedCallback) line: 137
> ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run() line: 161
> ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run() line: 157
> SecurityIdentity.runAs(PrivilegedExceptionAction<T>) line: 212
> AccessAuditContext.doAs(SecurityIdentity, InetAddress, PrivilegedExceptionAction<T>) line: 185
> ModelControllerClientOperationHandler$ExecuteRequestHandler$1.execute(ManagementRequestContext<Void>) line: 157
> ManagementRequestContextImpl$1.doExecute() line: 70
> ManagementRequestContextImpl$1(ManagementRequestContextImpl$AsyncTaskRunner).run() line: 160
> ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1142
> ThreadPoolExecutor$Worker.run() line: 617
> JBossThread(Thread).run() line: 745
> JBossThread.run() line: 320
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (JBTM-2837) Server being stuck during shudown when transaction probe op is called
by Michael Musgrove (JIRA)
[ https://issues.jboss.org/browse/JBTM-2837?page=com.atlassian.jira.plugin.... ]
Michael Musgrove commented on JBTM-2837:
----------------------------------------
Thanks for the thorough analysis [~ochaloup] As ondrej has discovered after periodicWorkFirstPass if periodic recovery has been asked to terminate we skip periodicWorkSecondPass so we do not get the opportunity to update the XARecoveryModule#scanState. There are two alternative fixes:
# notify each recovery module that periodic recovery has finished (and therefore periodicWorkSecondPass will not be called);
# if periodicWorkFirstPass has been called then always call periodicWorkSecondPass and only then check to see whether periodic recovery has finished
My preference is option 2) (the only drawback is that it will delay shutdown). Ideally we would call it immediately and also pass an argument to periodicWorkSecondPass indicating the operating mode of scanning process (https://github.com/jbosstm/narayana/blob/master/ArjunaCore/arjuna/classes...) which recovery modules would use to optimise the second pass. This is an interface change so we would need to use java 8 default methods feature ( https://docs.oracle.com/javase/tutorial/java/IandI/defaultmethods.html) so we should do that in the next major release.
If nobody has any objections I will implement option 2)
> Server being stuck during shudown when transaction probe op is called
> ---------------------------------------------------------------------
>
> Key: JBTM-2837
> URL: https://issues.jboss.org/browse/JBTM-2837
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Components: Recovery
> Affects Versions: 5.5.0.Final
> Reporter: Ondra Chaloupka
> Assignee: Michael Musgrove
> Attachments: server.log, stacktrace1.log
>
>
> I do experience server to be stuck (in intermittent way) during shutdown. I need to use afterwards {{kill -9}} to stop it.
> From my investigation it seems that it's caused by fact that {{jboss-cli}} operation {{:probe}} does cause {{XARecoveryModule.periodicWorkFirstPass}} being called
> https://github.com/jbosstm/narayana/blob/master/ArjunaJTA/jta/classes/com...
> That way {{scanState}} is left at value ScanStates.BETWEEN_PASSES [1]
> https://github.com/jbosstm/narayana/blob/master/ArjunaJTA/jta/classes/com...
> Now when shutdown of container is being run (you can check attached {{server.log}})
> the periodic recovery is stopped and the {{scanState}} is left as it is. At that point
> call of {{XARecoveryModule#removeXAResourceRecoveryHelper}} causes thread waits for state {{IDLE}} ifinintely. https://github.com/jbosstm/narayana/blob/master/ArjunaJTA/jta/classes/com...
> You can consult thread dump from time when server gets stuck {{stacktrace1.log}}.
> [1]
> {code}
> Thread [management-handler-thread - 5] (Suspended (breakpoint at line 149 in XARecoveryModule))
> owns: XARecoveryModule (id=219)
> owns: AtomicAction (id=844)
> owns: ObjStoreBrowser (id=845)
> XARecoveryModule.periodicWorkFirstPass() line: 149
> XARecoveryModule.getNewXAResource(Xid) line: 272
> XARecoveryModule.getNewXAResource(XAResourceRecord) line: 310
> XAResourceRecord.getNewXAResource() line: 1303
> XAResourceRecord.restore_state(InputObjectState, int) line: 1054
> AtomicAction(BasicAction).restore_state(InputObjectState, int) line: 1180
> AtomicAction(BasicAction).activate(String) line: 488
> AtomicAction(BasicAction).activate() line: 451
> ActionBean$GenericAtomicActionWrapper.activate() line: 391
> ActionBean.createWrapper(UidWrapper, boolean) line: 107
> ActionBean.<init>(UidWrapper) line: 93
> NativeConstructorAccessorImpl.newInstance0(Constructor<?>, Object[]) line: not available [native method]
> NativeConstructorAccessorImpl.newInstance(Object[]) line: 62
> DelegatingConstructorAccessorImpl.newInstance(Object[]) line: 45
> Constructor<T>.newInstance(Object...) line: 423
> UidWrapper.createMBean() line: 196
> ObjStoreBrowser.createBean(Uid, String) line: 481
> ObjStoreBrowser.probe() line: 435
> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> Method.invoke(Object, Object...) line: 498
> Trampoline.invoke(Method, Object, Object[]) line: 71
> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> Method.invoke(Object, Object...) line: 498
> MethodUtil.invoke(Method, Object, Object[]) line: 275
> StandardMBeanIntrospector.invokeM2(Method, Object, Object[], Object) line: 112
> StandardMBeanIntrospector.invokeM2(Object, Object, Object[], Object) line: 46
> StandardMBeanIntrospector(MBeanIntrospector<M>).invokeM(M, Object, Object[], Object) line: 237
> PerInterface<M>.invoke(Object, String, Object[], String[], Object) line: 138
> StandardMBeanSupport(MBeanSupport<M>).invoke(String, Object[], String[]) line: 252
> DefaultMBeanServerInterceptor.invoke(ObjectName, String, Object[], String[]) line: 819
> JmxMBeanServer.invoke(ObjectName, String, Object[], String[]) line: 801
> PluggableMBeanServerImpl$TcclMBeanServer.invoke(ObjectName, String, Object[], String[]) line: 1512
> PluggableMBeanServerImpl.invoke(ObjectName, String, Object[], String[]) line: 731
> LogStoreProbeHandler.probeTransactions(MBeanServer, boolean) line: 157
> LogStoreProbeHandler.execute(OperationContext, ModelNode) line: 186
> OperationContextImpl(AbstractOperationContext).executeStep(AbstractOperationContext$Step) line: 921
> OperationContextImpl(AbstractOperationContext).processStages() line: 664
> OperationContextImpl(AbstractOperationContext).executeOperation() line: 383
> OperationContextImpl.executeOperation() line: 1390
> ModelControllerImpl.internalExecute(ModelNode, OperationMessageHandler, ModelController$OperationTransactionControl, OperationAttachments, OperationStepHandler, boolean, boolean) line: 419
> ModelControllerImpl.lambda$execute$1(Operation, OperationMessageHandler, ModelController$OperationTransactionControl) line: 240
> 277244299.run() line: not available
> SecurityIdentity.runAs(PrivilegedAction<T>) line: 193
> ModelControllerImpl.execute(Operation, OperationMessageHandler, ModelController$OperationTransactionControl) line: 240
> ModelControllerClientOperationHandler$ExecuteRequestHandler.doExecute(ModelNode, int, ManagementRequestContext<Void>, CompletedCallback) line: 217
> ModelControllerClientOperationHandler$ExecuteRequestHandler.access$400(ModelControllerClientOperationHandler$ExecuteRequestHandler, ModelNode, int, ManagementRequestContext, ModelControllerClientOperationHandler$CompletedCallback) line: 137
> ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run() line: 161
> ModelControllerClientOperationHandler$ExecuteRequestHandler$1$1.run() line: 157
> SecurityIdentity.runAs(PrivilegedExceptionAction<T>) line: 212
> AccessAuditContext.doAs(SecurityIdentity, InetAddress, PrivilegedExceptionAction<T>) line: 185
> ModelControllerClientOperationHandler$ExecuteRequestHandler$1.execute(ManagementRequestContext<Void>) line: 157
> ManagementRequestContextImpl$1.doExecute() line: 70
> ManagementRequestContextImpl$1(ManagementRequestContextImpl$AsyncTaskRunner).run() line: 160
> ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1142
> ThreadPoolExecutor$Worker.run() line: 617
> JBossThread(Thread).run() line: 745
> JBossThread.run() line: 320
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years