[jboss-jira] [JBoss JIRA] (WFLY-12583) DB Connection is not getting removed from pool when transaction is rooled back during database fail over in Wildfly-17

[ https://issues.redhat.com/browse/WFLY-12583?page=com.atlassian.jira.plugi... ] James Perkins updated WFLY-12583: --------------------------------- Description: I am using Wildfly-17 server. I am testing the fail over scenario on Oracle database during on going API traffic in Wildfly. The steps I performed: - Oracle RAC DB setup is done and scan ip (Virtual IP) is configured in domain.xml. Also, created 2 database services (named as "pri" and "sec") in database with preferred service configuration. - Wildfly server is up and running with my project binaries, where DS configuration is: {code} <datasource jta="true" jndi-name="java:jboss/datasources/jdbc/InventoryDS" pool-name="jdbc/InventoryDS" enabled="true" use-java-context="true" spy="false" use-ccm="true" statistics-enabled="true"> <connection-url>jdbc:oracle:thin:@10.151.5.16:1521/pri</connection-url> <connection-property name="AutoCommit">false</connection-property> <driver>ojdbc8</driver> <pool> <min-pool-size>1</min-pool-size> <initial-pool-size>1</initial-pool-size> <max-pool-size>10</max-pool-size> <prefill>false</prefill> <fair>false</fair> <use-strict-min>false</use-strict-min> <flush-strategy>FailingConnectionOnly</flush-strategy> </pool> <security> <security-domain>InventoryDS</security-domain> </security> <validation> <validate-on-match>false</validate-on-match> <background-validation>false</background-validation> <use-fast-fail>false</use-fast-fail> <exception-sorter class-name="org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter"/> </validation> <statement> <track-statements>false</track-statements> </statement> </datasource> {code} - Started load (REST API) using JMeter. All current Db connection is pointing to "pri" service - During ongoing traffic, I rebooted one DB node (having "pri" service) Expectation: The existing DB connections will be removed from pool and new connection will be pointing to "sec" service. Hence, minimal impact on ongoing API traffic Actual result : It is not removing DB connection from pool. Next time the same connection is getting used. Due to which the API is getting failed. Observation: The code is written in such a way that for any exception occures during API execution, the code set to call setRollbackOnly() method for transaction (it is container managed transaction in EJB). Due to the transaction rollback, the server is not able to remove connection from pool. When I removed setRollbackOnly() from code, then it is able to remove connection from pool and next request will get new connection from "sec" DB service. Logs for the reference: ======================= Logs with setRollbackOnly() called from API: -------------------------------------------- {code} 2019-09-13 11:01:44,464 TRACE (default task-15) IJTRACER-jdbc/InventoryDS-4467852c-1046-93-2069714450032188-105a159d-73db02c0-org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction@75128fe0 2019-09-13 11:01:44,464 TRACE (default task-15) IJTRACER-jdbc/InventoryDS-4467852c-1046-41-2069714450059032-105a159d-73db02c0- 2019-09-13 11:01:44,464 TRACE (default task-15) [TxConnectionListener@105a159d] unregisterConnection: 0 handles left ([]) 2019-09-13 11:01:44,464 TRACE (default task-15) isManagedConnectionFree=false mc=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@33bd6c05 2019-09-13 11:01:44,465 TRACE (default task-15) popped object: org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction@75128fe0 2019-09-13 11:01:44,465 TRACE (default task-15) IJTRACER-CachedConnectionManager-NONE-1046-91-2069714451688504-NONE-org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction@75128fe0-java.lang.Throwable:_CALLSTACK|\at_org.jboss.jca.core.connectionmanager.ccm.CachedConnectionManagerImpl.popMetaAwareObject(CachedConnectionManagerImpl.java:283)|\at_org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction.teardown(CachedConnectionManagerSetupProcessor.java:107)|\at_org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:60)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)|\at_org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)|\at_org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)|\at_org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)|\at_org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)|\at_org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)|\at_org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)|\at_com.demo.inventory.ejb3.inventorymgmt.interfaces.IInventoryDetailSessionLocal$$$view355.searchInventory(Unknown_Source)|\at_com.demo.inventory.ejb3.inventorymgmt.sessionfacadeintapi.InventoryOperations.searchInventory(InventoryOperations.java:529)|\at_sun.reflect.GeneratedMethodAccessor1021.invoke(Unknown_Source)|\at_sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)|\at_java.lang.reflect.Method.invoke(Method.java:497)|\at_org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)|\at_org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70)|\at_org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80)|\at_org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93)|\at_org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)|\at_com.demo.inventory.aspect.PrePostPlugin.processPrePostEvent(PrePostPlugin.java:125)|\at_sun.reflect.GeneratedMethodAccessor1020.invoke(Unknown_Source)|\at_sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)|\at_java.lang.reflect.Method.invoke(Method.java:497)|\at_org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)|\at_org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)|\at_org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)|\at_org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275)|\at_org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)|\at_org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:79)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:44)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)|\at_org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)|\at_org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)|\at_org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)|\at_org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)|\at_org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:266)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:184)|\at_org.jboss.ejb.client.EJBObjectInterceptor.handleInvocation(EJBObjectInterceptor.java:58)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)|\at_org.jboss.ejb.client.EJBHomeInterceptor.handleInvocation(EJBHomeInterceptor.java:83)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)|\at_org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:42)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)|\at_org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:138)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)|\at_org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:255)|\at_org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:200)|\at_org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183)|\at_org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)|\at_com.sun.proxy.$Proxy250.searchInventory(Unknown_Source)|\at_com.demo.rest.inventory.delegator.InventoryOperationsDelegator.searchInventory(InventoryOperationsDelegator.java:70)|\at_com.demo.rest.inventory.service.InventoryOperationsService.searchInventory(InventoryOperationsService.java:61)|\at_com.demo.rest.inventory.resource.InventoryOperationsResource.searchInventory(InventoryOperationsResource.java:69)|\at_sun.reflect.GeneratedMethodAccessor1019.invoke(Unknown_Source)|\at_sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)|\at_java.lang.reflect.Method.invoke(Method.java:497)|\at_org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139)|\at_org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)|\at_org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)|\at_org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236)|\at_org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402)|\at_org.jboss.resteasy.core.SynchronousDispatcher.invokePropagateNotFound(SynchronousDispatcher.java:247)|\at_org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:225)|\at_org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.doFilter(FilterDispatcher.java:62)|\at_io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)|\at_io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)|\at_com.demo.rest.core.filter.CORSFilter.doFilter(CORSFilter.java:51)|\at_io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)|\at_io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)|\at_io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)|\at_io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)|\at_io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)|\at_org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)|\at_io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)|\at_io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)|\at_io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)|\at_io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)|\at_io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)|\at_io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)|\at_io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)|\at_io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)|\at_io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)|\at_io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)|\at_io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)|\at_io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)|\at_io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)|\at_io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)|\at_io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805)|\at_java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)|\at_java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)|\at_java.lang.Thread.run(Thread.java:745)| 2019-09-13 11:01:44,466 TRACE (default task-15) BaseTransaction.setRollbackOnly 2019-09-13 11:01:44,466 TRACE (default task-15) TransactionImple.setRollbackOnly 2019-09-13 11:01:44,466 TRACE (default task-15) TransactionImple.getStatus: javax.transaction.Status.STATUS_MARKED_ROLLBACK 2019-09-13 11:01:44,466 TRACE (default task-15) BaseTransaction.rollback 2019-09-13 11:01:44,466 TRACE (default task-15) TransactionImple.rollbackAndDisassociate 2019-09-13 11:01:44,468 TRACE (default task-15) Unlock: HeldByCurrentThread: Yes, Locked: Yes, HoldCount: 1, QueueLength: 0 2019-09-13 11:01:44,468 TRACE (default task-15) Owner: Thread[default task-15,5,main] 2019-09-13 11:01:44,476 TRACE (default task-15) SynchronizationImple.afterCompletion - Class: class org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList HashCode: 40330594 toString: org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList@2676562 2019-09-13 11:01:44,476 TRACE (default task-15) afterCompletion(4) isTrackByTx=true for org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@105a159d[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@33bd6c05 connection handles=0 lastReturned=1568352703782 lastValidated=1568350845750 lastCheckedOut=1568352703818 trackByTx=true pool=org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject@20796ab0 mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@4467852c[pool=jdbc/InventoryDS] xaResource=LocalXAResourceImpl@1263d5dd[connectionListener=105a159d connectionManager=630b92ae warned=false currentXid=null productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production jndiName=java:jboss/datasources/jdbc/InventoryDS] txSync=null] 2019-09-13 11:01:44,476 TRACE (default task-15) [TxConnectionListener@105a159d] isManagedConnectionFree: true 2019-09-13 11:01:44,476 TRACE (default task-15) IJTRACER-jdbc/InventoryDS-4467852c-1046-34-2069714462871419-105a159d-- {code} Logs without setRollbackOnly() from API: ----------------------------------------- {code} 2019-09-13 11:01:44,479 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-93-2069714465486053-3c9e3b2f-23a7ccff-org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction@751461b5 2019-09-13 11:01:44,479 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-41-2069714465507047-3c9e3b2f-23a7ccff- 2019-09-13 11:01:44,479 TRACE (default task-17) [TxConnectionListener@3c9e3b2f] unregisterConnection: 0 handles left ([]) 2019-09-13 11:01:44,479 TRACE (default task-17) isManagedConnectionFree=false mc=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@110a223f 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 2019-09-13 11:01:44,479 TRACE (default task-17) BaseTransaction.commit 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.commitAndDisassociate 2019-09-13 11:01:44,479 TRACE (default task-17) SynchronizationImple.beforeCompletion - Class: class org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList HashCode: 101703171 toString: org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList@60fde03 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 2019-09-13 11:01:44,479 TRACE (default task-17) delistResource(LocalXAResourceImpl@332d0f3[connectionListener=3c9e3b2f connectionManager=3081dd0c warned=false currentXid=< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a278003:-28cc616e:5d7b227a:c51, node_name=1, branch_uid=0:ffff0a278003:-28cc616e:5d7b227a:c53, subordinatenodename=null, eis_name=java:jboss/datasources/jdbc/InventoryDS > productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production jndiName=java:jboss/datasources/jdbc/InventoryDS], TMSUCCESS) 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.delistResource ( LocalXAResourceImpl@332d0f3[connectionListener=3c9e3b2f connectionManager=3081dd0c warned=false currentXid=< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a278003:-28cc616e:5d7b227a:c51, node_name=1, branch_uid=0:ffff0a278003:-28cc616e:5d7b227a:c53, subordinatenodename=null, eis_name=java:jboss/datasources/jdbc/InventoryDS > productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production jndiName=java:jboss/datasources/jdbc/InventoryDS], 67108864 ) 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 2019-09-13 11:01:44,479 TRACE (default task-17) end(< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a278003:-28cc616e:5d7b227a:c51, node_name=1, branch_uid=0:ffff0a278003:-28cc616e:5d7b227a:c53, subordinatenodename=null, eis_name=java:jboss/datasources/jdbc/InventoryDS >,67108864) 2019-09-13 11:01:44,479 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-30-2069714465761476-3c9e3b2f-TransactionImple < ac, BasicAction: 0:ffff0a278003:_28cc616e:5d7b227a:c51 status: ActionStatus.RUNNING >- 2019-09-13 11:01:44,480 TRACE (default task-17) Unlock: HeldByCurrentThread: Yes, Locked: Yes, HoldCount: 1, QueueLength: 0 2019-09-13 11:01:44,480 TRACE (default task-17) Owner: Thread[default task-17,5,main] 2019-09-13 11:01:44,480 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-11-2069714466239757-3c9e3b2f-- 2019-09-13 11:01:44,480 TRACE (default task-17) returnConnection(3c9e3b2f, true) 2019-09-13 11:01:44,480 TRACE (default task-17) Destroying returned connection org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@3c9e3b2f[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@110a223f connection handles=0 lastReturned=1568352703828 lastValidated=1568350873622 lastCheckedOut=1568352703856 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject@7b955f13 mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@588c2c98[pool=jdbc/InventoryDS] xaResource=LocalXAResourceImpl@332d0f3[connectionListener=3c9e3b2f connectionManager=3081dd0c warned=false currentXid=null productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production jndiName=java:jboss/datasources/jdbc/InventoryDS] txSync=TransactionSynchronization@384438466{tx=TransactionImple < ac, BasicAction: 0:ffff0a278003:-28cc616e:5d7b227a:c51 status: ActionStatus.COMMITTING > wasTrackByTx=true enlisted=true cancel=false}] 2019-09-13 11:01:44,480 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-70-2069714466500063-3c9e3b2f-- {code} Please help to solve this issue. was: I am using Wildfly-17 server. I am testing the fail over scenario on Oracle database during on going API traffic in Wildfly. The steps I performed: - Oracle RAC DB setup is done and scan ip (Virtual IP) is configured in domain.xml. Also, created 2 database services (named as "pri" and "sec") in database with preferred service configuration. - Wildfly server is up and running with my project binaries, where DS configuration is: <datasource jta="true" jndi-name="java:jboss/datasources/jdbc/InventoryDS" pool-name="jdbc/InventoryDS" enabled="true" use-java-context="true" spy="false" use-ccm="true" statistics-enabled="true"> <connection-url>jdbc:oracle:thin:@10.151.5.16:1521/pri</connection-url> <connection-property name="AutoCommit">false</connection-property> <driver>ojdbc8</driver> <pool> <min-pool-size>1</min-pool-size> <initial-pool-size>1</initial-pool-size> <max-pool-size>10</max-pool-size> <prefill>false</prefill> <fair>false</fair> <use-strict-min>false</use-strict-min> <flush-strategy>FailingConnectionOnly</flush-strategy> </pool> <security> <security-domain>InventoryDS</security-domain> </security> <validation> <validate-on-match>false</validate-on-match> <background-validation>false</background-validation> <use-fast-fail>false</use-fast-fail> <exception-sorter class-name="org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter"/> </validation> <statement> <track-statements>false</track-statements> </statement> </datasource> - Started load (REST API) using JMeter. All current Db connection is pointing to "pri" service - During ongoing traffic, I rebooted one DB node (having "pri" service) Expectation: The existing DB connections will be removed from pool and new connection will be pointing to "sec" service. Hence, minimal impact on ongoing API traffic Actual result : It is not removing DB connection from pool. Next time the same connection is getting used. Due to which the API is getting failed. Observation: The code is written in such a way that for any exception occures during API execution, the code set to call setRollbackOnly() method for transaction (it is container managed transaction in EJB). Due to the transaction rollback, the server is not able to remove connection from pool. When I removed setRollbackOnly() from code, then it is able to remove connection from pool and next request will get new connection from "sec" DB service. Logs for the reference: ======================= Logs with setRollbackOnly() called from API: -------------------------------------------- 2019-09-13 11:01:44,464 TRACE (default task-15) IJTRACER-jdbc/InventoryDS-4467852c-1046-93-2069714450032188-105a159d-73db02c0-org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction@75128fe0 2019-09-13 11:01:44,464 TRACE (default task-15) IJTRACER-jdbc/InventoryDS-4467852c-1046-41-2069714450059032-105a159d-73db02c0- 2019-09-13 11:01:44,464 TRACE (default task-15) [TxConnectionListener@105a159d] unregisterConnection: 0 handles left ([]) 2019-09-13 11:01:44,464 TRACE (default task-15) isManagedConnectionFree=false mc=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@33bd6c05 2019-09-13 11:01:44,465 TRACE (default task-15) popped object: org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction@75128fe0 2019-09-13 11:01:44,465 TRACE (default task-15) IJTRACER-CachedConnectionManager-NONE-1046-91-2069714451688504-NONE-org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction@75128fe0-java.lang.Throwable:_CALLSTACK|\at_org.jboss.jca.core.connectionmanager.ccm.CachedConnectionManagerImpl.popMetaAwareObject(CachedConnectionManagerImpl.java:283)|\at_org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction.teardown(CachedConnectionManagerSetupProcessor.java:107)|\at_org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:60)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)|\at_org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)|\at_org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)|\at_org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)|\at_org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)|\at_org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)|\at_org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)|\at_com.demo.inventory.ejb3.inventorymgmt.interfaces.IInventoryDetailSessionLocal$$$view355.searchInventory(Unknown_Source)|\at_com.demo.inventory.ejb3.inventorymgmt.sessionfacadeintapi.InventoryOperations.searchInventory(InventoryOperations.java:529)|\at_sun.reflect.GeneratedMethodAccessor1021.invoke(Unknown_Source)|\at_sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)|\at_java.lang.reflect.Method.invoke(Method.java:497)|\at_org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)|\at_org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70)|\at_org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80)|\at_org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93)|\at_org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)|\at_com.demo.inventory.aspect.PrePostPlugin.processPrePostEvent(PrePostPlugin.java:125)|\at_sun.reflect.GeneratedMethodAccessor1020.invoke(Unknown_Source)|\at_sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)|\at_java.lang.reflect.Method.invoke(Method.java:497)|\at_org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)|\at_org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)|\at_org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)|\at_org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275)|\at_org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)|\at_org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:79)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:44)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)|\at_org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)|\at_org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)|\at_org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)|\at_org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)|\at_org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)|\at_org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)|\at_org.jboss.as.ejb3.remote.LocalEjbReceiver.processInvocation(LocalEjbReceiver.java:266)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:184)|\at_org.jboss.ejb.client.EJBObjectInterceptor.handleInvocation(EJBObjectInterceptor.java:58)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)|\at_org.jboss.ejb.client.EJBHomeInterceptor.handleInvocation(EJBHomeInterceptor.java:83)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)|\at_org.jboss.ejb.client.TransactionInterceptor.handleInvocation(TransactionInterceptor.java:42)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)|\at_org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:138)|\at_org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)|\at_org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:255)|\at_org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:200)|\at_org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183)|\at_org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)|\at_com.sun.proxy.$Proxy250.searchInventory(Unknown_Source)|\at_com.demo.rest.inventory.delegator.InventoryOperationsDelegator.searchInventory(InventoryOperationsDelegator.java:70)|\at_com.demo.rest.inventory.service.InventoryOperationsService.searchInventory(InventoryOperationsService.java:61)|\at_com.demo.rest.inventory.resource.InventoryOperationsResource.searchInventory(InventoryOperationsResource.java:69)|\at_sun.reflect.GeneratedMethodAccessor1019.invoke(Unknown_Source)|\at_sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)|\at_java.lang.reflect.Method.invoke(Method.java:497)|\at_org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139)|\at_org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)|\at_org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)|\at_org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236)|\at_org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402)|\at_org.jboss.resteasy.core.SynchronousDispatcher.invokePropagateNotFound(SynchronousDispatcher.java:247)|\at_org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:225)|\at_org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.doFilter(FilterDispatcher.java:62)|\at_io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)|\at_io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)|\at_com.demo.rest.core.filter.CORSFilter.doFilter(CORSFilter.java:51)|\at_io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)|\at_io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)|\at_io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)|\at_io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)|\at_io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)|\at_org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)|\at_io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)|\at_io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)|\at_io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)|\at_io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)|\at_io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)|\at_io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)|\at_io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)|\at_io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)|\at_io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)|\at_io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)|\at_io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)|\at_io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)|\at_io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)|\at_io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)|\at_io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)|\at_io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)|\at_io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805)|\at_java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)|\at_java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)|\at_java.lang.Thread.run(Thread.java:745)| 2019-09-13 11:01:44,466 TRACE (default task-15) BaseTransaction.setRollbackOnly 2019-09-13 11:01:44,466 TRACE (default task-15) TransactionImple.setRollbackOnly 2019-09-13 11:01:44,466 TRACE (default task-15) TransactionImple.getStatus: javax.transaction.Status.STATUS_MARKED_ROLLBACK 2019-09-13 11:01:44,466 TRACE (default task-15) BaseTransaction.rollback 2019-09-13 11:01:44,466 TRACE (default task-15) TransactionImple.rollbackAndDisassociate 2019-09-13 11:01:44,468 TRACE (default task-15) Unlock: HeldByCurrentThread: Yes, Locked: Yes, HoldCount: 1, QueueLength: 0 2019-09-13 11:01:44,468 TRACE (default task-15) Owner: Thread[default task-15,5,main] 2019-09-13 11:01:44,476 TRACE (default task-15) SynchronizationImple.afterCompletion - Class: class org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList HashCode: 40330594 toString: org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList@2676562 2019-09-13 11:01:44,476 TRACE (default task-15) afterCompletion(4) isTrackByTx=true for org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@105a159d[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@33bd6c05 connection handles=0 lastReturned=1568352703782 lastValidated=1568350845750 lastCheckedOut=1568352703818 trackByTx=true pool=org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject@20796ab0 mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@4467852c[pool=jdbc/InventoryDS] xaResource=LocalXAResourceImpl@1263d5dd[connectionListener=105a159d connectionManager=630b92ae warned=false currentXid=null productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production jndiName=java:jboss/datasources/jdbc/InventoryDS] txSync=null] 2019-09-13 11:01:44,476 TRACE (default task-15) [TxConnectionListener@105a159d] isManagedConnectionFree: true 2019-09-13 11:01:44,476 TRACE (default task-15) IJTRACER-jdbc/InventoryDS-4467852c-1046-34-2069714462871419-105a159d-- Logs without setRollbackOnly() from API: ----------------------------------------- 2019-09-13 11:01:44,479 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-93-2069714465486053-3c9e3b2f-23a7ccff-org.jboss.as.connector.deployers.ra.processors.CachedConnectionManagerSetupProcessor$CachedConnectionManagerSetupAction@751461b5 2019-09-13 11:01:44,479 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-41-2069714465507047-3c9e3b2f-23a7ccff- 2019-09-13 11:01:44,479 TRACE (default task-17) [TxConnectionListener@3c9e3b2f] unregisterConnection: 0 handles left ([]) 2019-09-13 11:01:44,479 TRACE (default task-17) isManagedConnectionFree=false mc=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@110a223f 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 2019-09-13 11:01:44,479 TRACE (default task-17) BaseTransaction.commit 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.commitAndDisassociate 2019-09-13 11:01:44,479 TRACE (default task-17) SynchronizationImple.beforeCompletion - Class: class org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList HashCode: 101703171 toString: org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList@60fde03 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 2019-09-13 11:01:44,479 TRACE (default task-17) delistResource(LocalXAResourceImpl@332d0f3[connectionListener=3c9e3b2f connectionManager=3081dd0c warned=false currentXid=< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a278003:-28cc616e:5d7b227a:c51, node_name=1, branch_uid=0:ffff0a278003:-28cc616e:5d7b227a:c53, subordinatenodename=null, eis_name=java:jboss/datasources/jdbc/InventoryDS > productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production jndiName=java:jboss/datasources/jdbc/InventoryDS], TMSUCCESS) 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.delistResource ( LocalXAResourceImpl@332d0f3[connectionListener=3c9e3b2f connectionManager=3081dd0c warned=false currentXid=< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a278003:-28cc616e:5d7b227a:c51, node_name=1, branch_uid=0:ffff0a278003:-28cc616e:5d7b227a:c53, subordinatenodename=null, eis_name=java:jboss/datasources/jdbc/InventoryDS > productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production jndiName=java:jboss/datasources/jdbc/InventoryDS], 67108864 ) 2019-09-13 11:01:44,479 TRACE (default task-17) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 2019-09-13 11:01:44,479 TRACE (default task-17) end(< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a278003:-28cc616e:5d7b227a:c51, node_name=1, branch_uid=0:ffff0a278003:-28cc616e:5d7b227a:c53, subordinatenodename=null, eis_name=java:jboss/datasources/jdbc/InventoryDS >,67108864) 2019-09-13 11:01:44,479 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-30-2069714465761476-3c9e3b2f-TransactionImple < ac, BasicAction: 0:ffff0a278003:_28cc616e:5d7b227a:c51 status: ActionStatus.RUNNING >- 2019-09-13 11:01:44,480 TRACE (default task-17) Unlock: HeldByCurrentThread: Yes, Locked: Yes, HoldCount: 1, QueueLength: 0 2019-09-13 11:01:44,480 TRACE (default task-17) Owner: Thread[default task-17,5,main] 2019-09-13 11:01:44,480 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-11-2069714466239757-3c9e3b2f-- 2019-09-13 11:01:44,480 TRACE (default task-17) returnConnection(3c9e3b2f, true) 2019-09-13 11:01:44,480 TRACE (default task-17) Destroying returned connection org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@3c9e3b2f[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@110a223f connection handles=0 lastReturned=1568352703828 lastValidated=1568350873622 lastCheckedOut=1568352703856 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject@7b955f13 mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool@588c2c98[pool=jdbc/InventoryDS] xaResource=LocalXAResourceImpl@332d0f3[connectionListener=3c9e3b2f connectionManager=3081dd0c warned=false currentXid=null productName=Oracle productVersion=Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production jndiName=java:jboss/datasources/jdbc/InventoryDS] txSync=TransactionSynchronization@384438466{tx=TransactionImple < ac, BasicAction: 0:ffff0a278003:-28cc616e:5d7b227a:c51 status: ActionStatus.COMMITTING > wasTrackByTx=true enlisted=true cancel=false}] 2019-09-13 11:01:44,480 TRACE (default task-17) IJTRACER-jdbc/InventoryDS-588c2c98-1051-70-2069714466500063-3c9e3b2f-- Please help to solve this issue. -- This message was sent by Atlassian Jira (v7.13.8#713008)