[JBoss JIRA] (WFLY-10392) Regression in Remote JCA scenario with JDBC store after Artemis upgrade
by Erich Duda (JIRA)
[ https://issues.jboss.org/browse/WFLY-10392?page=com.atlassian.jira.plugin... ]
Erich Duda commented on WFLY-10392:
-----------------------------------
Thanks [~fnigro]. It would be great to find out which change causes the regressions. So if you provide the version (in form of Artemis branch), I can run the tests with it and we will see.
> Regression in Remote JCA scenario with JDBC store after Artemis upgrade
> -----------------------------------------------------------------------
>
> Key: WFLY-10392
> URL: https://issues.jboss.org/browse/WFLY-10392
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Reporter: Erich Duda
> Assignee: Jeff Mesnil
> Priority: Critical
>
> After Artemis upgrade to 1.5.5.jbossorg-011 (WFLY-10139) I can see a regression in Remote JCA test case when JDBC persistent store is used. This issue is not present if Artemis file based journal is used.
> When I removed commits related to JDBC HA from Artemis upgrade, the test passed with both JDBC and file based store.
> *Remote JCA scenario:*
> * There are 4 Wildfly servers
> * Servers 1 and 3 are used as messaging brokers - they are called JMS servers
> * Servers 2 and 4 have MDBs and RA configured to connect to JMS servers. These servers are called MDB servers.
> * External standalone producer sends messages to server 1 to InQueue
> * MDBs on MBD severs receive messages from InQueue and send them to OutQueue
> * External standalone receiver receives messages from server 3 from OutQueue
> * During this scenario server 1 is several times killed and restarted.
> *Expectation:* All messages sent by the standalone producer are received by the standalone receiver. There are no lost or duplicated messages.
> *Reality:* After some kills of server 1, message flow coming from the standalone producer is broken and the receiver does not receive any messages in specified timeout.
> *Critical* priority was chosen because the regression has been detected only in this particular test case so far. We ran only nightly testing scope. After that we run full testing, we will know more about impact of this issue.
> *Technical details:*
> At some point following log messages start to occur in the test log. It seems that they are related to malfunction of communication among Artemis brokers what causes that message flow is broken.
> {code}
> 21:55:51,724 WARN [org.apache.activemq.artemis.core.server] (Thread-1 (ActiveMQ-client-global-threads)) AMQ222139: MessageFlowRecordImpl [nodeID=e53da514-5953-11e8-910a-fa163e48a89a, connector=TransportConfigur
> ation(name=connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?httpUpgradeEndpoint=acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=10080&host=rhel7-la
> rge-58597, queueName=sf.my-cluster.e53da514-5953-11e8-910a-fa163e48a89a, queue=QueueImpl[name=sf.my-cluster.e53da514-5953-11e8-910a-fa163e48a89a, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=
> d80bf5bd-5953-11e8-ac10-fa163e48a89a]]@756a727a, isClosed=false, reset=true]::Remote queue binding jms.queue.DLQe53da514-5953-11e8-910a-fa163e48a89a has already been bound in the post office. Most likely cause f
> or this is you have a loop in your cluster due to cluster max-hops being too large or you have multiple cluster connections to the same nodes using overlapping addresses
> {code}
> {code}
> 21:55:53,078 ERROR [org.apache.activemq.artemis.core.server] (Thread-6 (ActiveMQ-client-global-threads)) AMQ224037: cluster connection Failed to handle message: java.lang.IllegalStateException: Cannot find binding for jms.queue.InQueuee53da514-5953-11e8-910a-fa163e48a89a on ClusterConnectionImpl@14989488[nodeUUID=d80bf5bd-5953-11e8-ac10-fa163e48a89a, connector=TransportConfiguration(name=connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?httpUpgradeEndpoint=acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=8080&host=rhel7-large-58597, address=jms, server=ActiveMQServerImpl::serverUUID=d80bf5bd-5953-11e8-ac10-fa163e48a89a]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.doConsumerCreated(ClusterConnectionImpl.java:1294) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.handleNotificationMessage(ClusterConnectionImpl.java:1029) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.onMessage(ClusterConnectionImpl.java:1004) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1001) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:49) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1124) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (SWSQE-145) Modularize CI/CD Pipline
by Guilherme Baufaker Rêgo (JIRA)
[ https://issues.jboss.org/browse/SWSQE-145?page=com.atlassian.jira.plugin.... ]
Guilherme Baufaker Rêgo resolved SWSQE-145.
-------------------------------------------
Resolution: Done
> Modularize CI/CD Pipline
> ------------------------
>
> Key: SWSQE-145
> URL: https://issues.jboss.org/browse/SWSQE-145
> Project: Kiali QE
> Issue Type: Epic
> Reporter: Matt Mahoney
> Attachments: Kiali Pipeline Proposal.png
>
>
> The CI/CD pipeline has become more complex and challenging with which to build CI/CD usage-cases and to add new functionality:
> Use Case Examples:
> 1) Deploy Istio -> Deploy&build-kiali -> Deploy All Service Meshes -> Deploy Traffic Generator on all Service Meshes -> Run all test suites
> 2) Deploy built Kiali -> Run REST suite
> 3) Build and Deploy Kiali -> Run all test suites
> 4) Run UI suite on any Openshift Cluster
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (SWSQE-166) Create Test Meshes Phase (Step 3) of Pipeline
by Guilherme Baufaker Rêgo (JIRA)
[ https://issues.jboss.org/browse/SWSQE-166?page=com.atlassian.jira.plugin.... ]
Guilherme Baufaker Rêgo resolved SWSQE-166.
-------------------------------------------
Resolution: Done
> Create Test Meshes Phase (Step 3) of Pipeline
> ---------------------------------------------
>
> Key: SWSQE-166
> URL: https://issues.jboss.org/browse/SWSQE-166
> Project: Kiali QE
> Issue Type: Sub-task
> Reporter: Guilherme Baufaker Rêgo
> Assignee: Guilherme Baufaker Rêgo
>
> {panel:title=
> Step 3 (SWSQE-166)}
> - Refactor [Kiali-Test-Mesh|http://jenkins2.bc.jonqe.lab.eng.bos.redhat.com:8080/job/Kiali-Test-Mesh/] Jenkins Job
> - Refactor to deploy a subset of Test Meshes (It helps if you are using a small cluster and doesn't need all type of meshes)
> - Make it optional on the pipeline
> - kiali-test-breadth-sink should be kiali-test-breath-sink
>
> {panel}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (WFLY-10392) Regression in Remote JCA scenario with JDBC store after Artemis upgrade
by Francesco Nigro (JIRA)
[ https://issues.jboss.org/browse/WFLY-10392?page=com.atlassian.jira.plugin... ]
Francesco Nigro commented on WFLY-10392:
----------------------------------------
[~eduda] The most evident changes I have done the could impact Oracle are:
# [How concurrent creation of table would be handled|https://github.com/rh-messaging/jboss-activemq-artemis/commit/8f7...]
# [How to read empty blobs|https://github.com/rh-messaging/jboss-activemq-artemis/commit/8f759...]
I suspect that the first one is the one that could create some issue on concurrent creation of tables and I'm expecting some weird debug logs during the creation of indexes.
I can provide a version of it that uses the old behaviour and see if is responsible of the regression.
AFAIK there aren't any other changes on the code that could impact Oracle, beside using the SQL statements read from a property file: a feature coming from the original commits of [~jmesnil] and I suppose aren't responsible of the regression.
> Regression in Remote JCA scenario with JDBC store after Artemis upgrade
> -----------------------------------------------------------------------
>
> Key: WFLY-10392
> URL: https://issues.jboss.org/browse/WFLY-10392
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Reporter: Erich Duda
> Assignee: Jeff Mesnil
> Priority: Critical
>
> After Artemis upgrade to 1.5.5.jbossorg-011 (WFLY-10139) I can see a regression in Remote JCA test case when JDBC persistent store is used. This issue is not present if Artemis file based journal is used.
> When I removed commits related to JDBC HA from Artemis upgrade, the test passed with both JDBC and file based store.
> *Remote JCA scenario:*
> * There are 4 Wildfly servers
> * Servers 1 and 3 are used as messaging brokers - they are called JMS servers
> * Servers 2 and 4 have MDBs and RA configured to connect to JMS servers. These servers are called MDB servers.
> * External standalone producer sends messages to server 1 to InQueue
> * MDBs on MBD severs receive messages from InQueue and send them to OutQueue
> * External standalone receiver receives messages from server 3 from OutQueue
> * During this scenario server 1 is several times killed and restarted.
> *Expectation:* All messages sent by the standalone producer are received by the standalone receiver. There are no lost or duplicated messages.
> *Reality:* After some kills of server 1, message flow coming from the standalone producer is broken and the receiver does not receive any messages in specified timeout.
> *Critical* priority was chosen because the regression has been detected only in this particular test case so far. We ran only nightly testing scope. After that we run full testing, we will know more about impact of this issue.
> *Technical details:*
> At some point following log messages start to occur in the test log. It seems that they are related to malfunction of communication among Artemis brokers what causes that message flow is broken.
> {code}
> 21:55:51,724 WARN [org.apache.activemq.artemis.core.server] (Thread-1 (ActiveMQ-client-global-threads)) AMQ222139: MessageFlowRecordImpl [nodeID=e53da514-5953-11e8-910a-fa163e48a89a, connector=TransportConfigur
> ation(name=connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?httpUpgradeEndpoint=acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=10080&host=rhel7-la
> rge-58597, queueName=sf.my-cluster.e53da514-5953-11e8-910a-fa163e48a89a, queue=QueueImpl[name=sf.my-cluster.e53da514-5953-11e8-910a-fa163e48a89a, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=
> d80bf5bd-5953-11e8-ac10-fa163e48a89a]]@756a727a, isClosed=false, reset=true]::Remote queue binding jms.queue.DLQe53da514-5953-11e8-910a-fa163e48a89a has already been bound in the post office. Most likely cause f
> or this is you have a loop in your cluster due to cluster max-hops being too large or you have multiple cluster connections to the same nodes using overlapping addresses
> {code}
> {code}
> 21:55:53,078 ERROR [org.apache.activemq.artemis.core.server] (Thread-6 (ActiveMQ-client-global-threads)) AMQ224037: cluster connection Failed to handle message: java.lang.IllegalStateException: Cannot find binding for jms.queue.InQueuee53da514-5953-11e8-910a-fa163e48a89a on ClusterConnectionImpl@14989488[nodeUUID=d80bf5bd-5953-11e8-ac10-fa163e48a89a, connector=TransportConfiguration(name=connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?httpUpgradeEndpoint=acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=8080&host=rhel7-large-58597, address=jms, server=ActiveMQServerImpl::serverUUID=d80bf5bd-5953-11e8-ac10-fa163e48a89a]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.doConsumerCreated(ClusterConnectionImpl.java:1294) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.handleNotificationMessage(ClusterConnectionImpl.java:1029) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.onMessage(ClusterConnectionImpl.java:1004) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1001) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:49) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1124) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (WFLY-9896) Cluster-cluster EJB transactional invocation rolls back if stateful bean is used and JBOSS-LOCAL-USER auth is not possible
by David Lloyd (JIRA)
[ https://issues.jboss.org/browse/WFLY-9896?page=com.atlassian.jira.plugin.... ]
David Lloyd commented on WFLY-9896:
-----------------------------------
Not yet. We'll come back to it after the WildFly 13 tag.
> Cluster-cluster EJB transactional invocation rolls back if stateful bean is used and JBOSS-LOCAL-USER auth is not possible
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-9896
> URL: https://issues.jboss.org/browse/WFLY-9896
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, EJB, Transactions
> Affects Versions: 11.0.0.Final, 12.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: David Lloyd
> Priority: Critical
> Labels: downstream_dependency
>
> Scenario:
> - server-1 (intermediary)
> -- Running standalone-ha profile (important)
> -- Has a remote-outbound-connection pointing at server-2 through server-2's public IP address
> -- Doesn't matter if the intermediary bean is stateless or stateful
> - server-2 (target)
> -- On a different machine than server-1 (important, otherwise transaction manager uses JBOSS-LOCAL-USER auth)
> -- Running standalone-ha profile (important)
> -- Target bean needs to be stateful (important)
> - Standalone EJB client invokes an intermediary bean server-1, which as a result invokes target bean on server-2.
> The first invocation is handled by EJB on target server, but when the transaction should be committed, The following is the log observed on server-1 (and also on the client):
> {code}
> 03:58:27,439 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: JBoss EAP 7.1.0.GA (WildFly Core 3.0.10.Final-redhat-1) started in 12893ms - Started 1013 of 1226 services (458 services are lazy, passive or on-demand)
> 03:59:00,563 INFO [org.wildfly.naming] (default task-7) WildFly Naming version 1.0.7.Final-redhat-1
> 03:59:00,585 INFO [org.jboss.ejb.client] (default task-7) JBoss EJB Client version 4.0.9.Final-redhat-1
> 03:59:01,335 INFO [stdout] (default task-7) Intermediate Stateless WhoAmIBean.whoAmI called as anonymous
> 03:59:01,809 WARN [com.arjuna.ats.arjuna] (default task-7) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a105a36:-2ab20799:5a2f9a2b:1c, org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@24aead2a >: org.wildfly.transaction.client.SynchronizationException: javax.transaction.xa.XAException: WFTXN0034: Failed to acquire a connection for this operation
> at org.wildfly.transaction.client.XAOutflowedResources$1.beforeCompletion(XAOutflowedResources.java:83)
> at org.wildfly.transaction.client.AbstractTransaction.performConsumer(AbstractTransaction.java:209)
> at org.wildfly.transaction.client.AbstractTransaction.performConsumer(AbstractTransaction.java:220)
> at org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization.beforeCompletion(AbstractTransaction.java:265)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1289)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:73)
> at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:92)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:332)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:240)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
> at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
> at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
> at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609)
> at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
> at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
> at org.wildfly.security.auth.server.SecurityIdentity.runAsFunctionEx(SecurityIdentity.java:380)
> at org.jboss.as.ejb3.remote.AssociationImpl.invokeWithIdentity(AssociationImpl.java:535)
> at org.jboss.as.ejb3.remote.AssociationImpl.invokeMethod(AssociationImpl.java:530)
> at org.jboss.as.ejb3.remote.AssociationImpl.lambda$receiveInvocationRequest$0(AssociationImpl.java:193)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: javax.transaction.xa.XAException: WFTXN0034: Failed to acquire a connection for this operation
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentityXA(RemotingRemoteTransactionPeer.java:94)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.beforeCompletion(RemotingRemoteTransactionPeer.java:166)
> at org.wildfly.transaction.client.SubordinateXAResource.beforeCompletion(SubordinateXAResource.java:135)
> at org.wildfly.transaction.client.XAOutflowedResources$1.beforeCompletion(XAOutflowedResources.java:70)
> ... 56 more
> Caused by: org.wildfly.security.auth.AuthenticationException: JBREM000308: Authentication failed (no mechanisms left), tried:
> JBOSS-LOCAL-USER: org.wildfly.security.auth.AuthenticationException: JBREM000300: Authentication failed due to I/O error
> at org.jboss.remoting3.ConnectionPeerIdentityContext.doAuthenticate(ConnectionPeerIdentityContext.java:368)
> at org.jboss.remoting3.ConnectionPeerIdentityContext.authenticate(ConnectionPeerIdentityContext.java:174)
> at org.jboss.remoting3.EndpointImpl$3.handleDone(EndpointImpl.java:505)
> at org.jboss.remoting3.EndpointImpl$3.handleDone(EndpointImpl.java:494)
> at org.xnio.IoFuture$HandlingNotifier.notify(IoFuture.java:208)
> at org.xnio.AbstractIoFuture$NotifierRunnable.run(AbstractIoFuture.java:720)
> at org.xnio.IoUtils$2.execute(IoUtils.java:71)
> at org.xnio.AbstractIoFuture.runNotifier(AbstractIoFuture.java:693)
> at org.xnio.AbstractIoFuture$CompleteState.withNotifier(AbstractIoFuture.java:132)
> at org.xnio.AbstractIoFuture.addNotifier(AbstractIoFuture.java:570)
> at org.jboss.remoting3.EndpointImpl.doGetConnection(EndpointImpl.java:494)
> at org.jboss.remoting3.EndpointImpl.getConnectedIdentity(EndpointImpl.java:433)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentity(RemotingRemoteTransactionPeer.java:87)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentityXA(RemotingRemoteTransactionPeer.java:92)
> ... 59 more
> 03:59:01,820 WARN [com.arjuna.ats.jta] (default task-7) ARJUNA016129: Could not end XA resource Subordinate XAResource at remote+http://10.16.90.56:8080: javax.transaction.xa.XAException: WFTXN0034: Failed to acquire a connection for this operation
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentityXA(RemotingRemoteTransactionPeer.java:94)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.end(RemotingRemoteTransactionPeer.java:157)
> at org.wildfly.transaction.client.SubordinateXAResource.end(SubordinateXAResource.java:130)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.doEnd(TransactionImple.java:1089)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.endAssociation(TransactionImple.java:1060)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.endAssociation(XAResourceRecord.java:1287)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:313)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:3023)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:3002)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1674)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1615)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:99)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1289)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:73)
> at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:92)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:332)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:240)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
> at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
> at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
> at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609)
> at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
> at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
> at org.wildfly.security.auth.server.SecurityIdentity.runAsFunctionEx(SecurityIdentity.java:380)
> at org.jboss.as.ejb3.remote.AssociationImpl.invokeWithIdentity(AssociationImpl.java:535)
> at org.jboss.as.ejb3.remote.AssociationImpl.invokeMethod(AssociationImpl.java:530)
> at org.jboss.as.ejb3.remote.AssociationImpl.lambda$receiveInvocationRequest$0(AssociationImpl.java:193)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.wildfly.security.auth.AuthenticationException: JBREM000308: Authentication failed (no mechanisms left), tried:
> JBOSS-LOCAL-USER: org.wildfly.security.auth.AuthenticationException: JBREM000300: Authentication failed due to I/O error
> at org.jboss.remoting3.ConnectionPeerIdentityContext.doAuthenticate(ConnectionPeerIdentityContext.java:368)
> at org.jboss.remoting3.ConnectionPeerIdentityContext.authenticate(ConnectionPeerIdentityContext.java:174)
> at org.jboss.remoting3.EndpointImpl$3.handleDone(EndpointImpl.java:505)
> at org.jboss.remoting3.EndpointImpl$3.handleDone(EndpointImpl.java:494)
> at org.xnio.IoFuture$HandlingNotifier.notify(IoFuture.java:208)
> at org.xnio.AbstractIoFuture$NotifierRunnable.run(AbstractIoFuture.java:720)
> at org.xnio.IoUtils$2.execute(IoUtils.java:71)
> at org.xnio.AbstractIoFuture.runNotifier(AbstractIoFuture.java:693)
> at org.xnio.AbstractIoFuture$CompleteState.withNotifier(AbstractIoFuture.java:132)
> at org.xnio.AbstractIoFuture.addNotifier(AbstractIoFuture.java:570)
> at org.jboss.remoting3.EndpointImpl.doGetConnection(EndpointImpl.java:494)
> at org.jboss.remoting3.EndpointImpl.getConnectedIdentity(EndpointImpl.java:433)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentity(RemotingRemoteTransactionPeer.java:87)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentityXA(RemotingRemoteTransactionPeer.java:92)
> ... 61 more
> 03:59:01,829 WARN [com.arjuna.ats.jta] (default task-7) ARJUNA016045: attempted rollback of < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a105a36:-2ab20799:5a2f9a2b:f, node_name=1, branch_uid=0:ffff0a105a36:-2ab20799:5a2f9a2b:19, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://10.16.90.56:8080) failed with exception code XAException.XAER_RMFAIL: javax.transaction.xa.XAException: WFTXN0034: Failed to acquire a connection for this operation
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentityXA(RemotingRemoteTransactionPeer.java:94)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.rollback(RemotingRemoteTransactionPeer.java:148)
> at org.wildfly.transaction.client.SubordinateXAResource.rollback(SubordinateXAResource.java:147)
> at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:330)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:3023)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:3002)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1674)
> at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1615)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:99)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1289)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:73)
> at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:92)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:332)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:240)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
> at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
> at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
> at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609)
> at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
> at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
> at org.wildfly.security.auth.server.SecurityIdentity.runAsFunctionEx(SecurityIdentity.java:380)
> at org.jboss.as.ejb3.remote.AssociationImpl.invokeWithIdentity(AssociationImpl.java:535)
> at org.jboss.as.ejb3.remote.AssociationImpl.invokeMethod(AssociationImpl.java:530)
> at org.jboss.as.ejb3.remote.AssociationImpl.lambda$receiveInvocationRequest$0(AssociationImpl.java:193)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.wildfly.security.auth.AuthenticationException: JBREM000308: Authentication failed (no mechanisms left), tried:
> JBOSS-LOCAL-USER: org.wildfly.security.auth.AuthenticationException: JBREM000300: Authentication failed due to I/O error
> at org.jboss.remoting3.ConnectionPeerIdentityContext.doAuthenticate(ConnectionPeerIdentityContext.java:368)
> at org.jboss.remoting3.ConnectionPeerIdentityContext.authenticate(ConnectionPeerIdentityContext.java:174)
> at org.jboss.remoting3.EndpointImpl$3.handleDone(EndpointImpl.java:505)
> at org.jboss.remoting3.EndpointImpl$3.handleDone(EndpointImpl.java:494)
> at org.xnio.IoFuture$HandlingNotifier.notify(IoFuture.java:208)
> at org.xnio.AbstractIoFuture$NotifierRunnable.run(AbstractIoFuture.java:720)
> at org.xnio.IoUtils$2.execute(IoUtils.java:71)
> at org.xnio.AbstractIoFuture.runNotifier(AbstractIoFuture.java:693)
> at org.xnio.AbstractIoFuture$CompleteState.withNotifier(AbstractIoFuture.java:132)
> at org.xnio.AbstractIoFuture.addNotifier(AbstractIoFuture.java:570)
> at org.jboss.remoting3.EndpointImpl.doGetConnection(EndpointImpl.java:494)
> at org.jboss.remoting3.EndpointImpl.getConnectedIdentity(EndpointImpl.java:433)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentity(RemotingRemoteTransactionPeer.java:87)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentityXA(RemotingRemoteTransactionPeer.java:92)
> ... 58 more
> 03:59:01,833 WARN [com.arjuna.ats.arjuna] (default task-7) ARJUNA012091: Top-level abort of action 0:ffff0a105a36:-2ab20799:5a2f9a2b:f received TwoPhaseOutcome.FINISH_ERROR from com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord
> 03:59:01,838 ERROR [org.jboss.as.ejb3.invocation] (default task-7) WFLYEJB0034: EJB Invocation failed on component Intermediate for method public abstract java.lang.String example.ejb.WhoAmIBeanRemote.whoAmI(): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:138)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:118)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:332)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:240)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
> at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
> at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
> at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609)
> at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
> at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
> at org.wildfly.security.auth.server.SecurityIdentity.runAsFunctionEx(SecurityIdentity.java:380)
> at org.jboss.as.ejb3.remote.AssociationImpl.invokeWithIdentity(AssociationImpl.java:535)
> at org.jboss.as.ejb3.remote.AssociationImpl.invokeMethod(AssociationImpl.java:530)
> at org.jboss.as.ejb3.remote.AssociationImpl.lambda$receiveInvocationRequest$0(AssociationImpl.java:193)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1301)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:73)
> at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:92)
> ... 43 more
> Caused by: org.wildfly.transaction.client.SynchronizationException: javax.transaction.xa.XAException: WFTXN0034: Failed to acquire a connection for this operation
> at org.wildfly.transaction.client.XAOutflowedResources$1.beforeCompletion(XAOutflowedResources.java:83)
> at org.wildfly.transaction.client.AbstractTransaction.performConsumer(AbstractTransaction.java:209)
> at org.wildfly.transaction.client.AbstractTransaction.performConsumer(AbstractTransaction.java:220)
> at org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization.beforeCompletion(AbstractTransaction.java:265)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1289)
> ... 48 more
> Caused by: javax.transaction.xa.XAException: WFTXN0034: Failed to acquire a connection for this operation
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentityXA(RemotingRemoteTransactionPeer.java:94)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.beforeCompletion(RemotingRemoteTransactionPeer.java:166)
> at org.wildfly.transaction.client.SubordinateXAResource.beforeCompletion(SubordinateXAResource.java:135)
> at org.wildfly.transaction.client.XAOutflowedResources$1.beforeCompletion(XAOutflowedResources.java:70)
> ... 56 more
> Caused by: org.wildfly.security.auth.AuthenticationException: JBREM000308: Authentication failed (no mechanisms left), tried:
> JBOSS-LOCAL-USER: org.wildfly.security.auth.AuthenticationException: JBREM000300: Authentication failed due to I/O error
> at org.jboss.remoting3.ConnectionPeerIdentityContext.doAuthenticate(ConnectionPeerIdentityContext.java:368)
> at org.jboss.remoting3.ConnectionPeerIdentityContext.authenticate(ConnectionPeerIdentityContext.java:174)
> at org.jboss.remoting3.EndpointImpl$3.handleDone(EndpointImpl.java:505)
> at org.jboss.remoting3.EndpointImpl$3.handleDone(EndpointImpl.java:494)
> at org.xnio.IoFuture$HandlingNotifier.notify(IoFuture.java:208)
> at org.xnio.AbstractIoFuture$NotifierRunnable.run(AbstractIoFuture.java:720)
> at org.xnio.IoUtils$2.execute(IoUtils.java:71)
> at org.xnio.AbstractIoFuture.runNotifier(AbstractIoFuture.java:693)
> at org.xnio.AbstractIoFuture$CompleteState.withNotifier(AbstractIoFuture.java:132)
> at org.xnio.AbstractIoFuture.addNotifier(AbstractIoFuture.java:570)
> at org.jboss.remoting3.EndpointImpl.doGetConnection(EndpointImpl.java:494)
> at org.jboss.remoting3.EndpointImpl.getConnectedIdentity(EndpointImpl.java:433)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentity(RemotingRemoteTransactionPeer.java:87)
> at org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer.getPeerIdentityXA(RemotingRemoteTransactionPeer.java:92)
> ... 59 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (WFLY-10392) Regression in Remote JCA scenario with JDBC store after Artemis upgrade
by Erich Duda (JIRA)
[ https://issues.jboss.org/browse/WFLY-10392?page=com.atlassian.jira.plugin... ]
Erich Duda commented on WFLY-10392:
-----------------------------------
As I mentioned in the description, this issue was hit by nightly tests which have limited scope. I've triggered weekly jobs (which are still running) to get more information.
Some jobs from weekly scope already finished. I can see another regression in {{Lodh4TestCase}} with JDBC store. Tests in this scenario fail because of duplicated messages.
Variant of the test case with file based journal passes. Also when I ran it against modified build where JDBC HA work was reverted from the upgrade, the tests passed.
CC: [~fnigro] [~ataylor]
> Regression in Remote JCA scenario with JDBC store after Artemis upgrade
> -----------------------------------------------------------------------
>
> Key: WFLY-10392
> URL: https://issues.jboss.org/browse/WFLY-10392
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Reporter: Erich Duda
> Assignee: Jeff Mesnil
> Priority: Critical
>
> After Artemis upgrade to 1.5.5.jbossorg-011 (WFLY-10139) I can see a regression in Remote JCA test case when JDBC persistent store is used. This issue is not present if Artemis file based journal is used.
> When I removed commits related to JDBC HA from Artemis upgrade, the test passed with both JDBC and file based store.
> *Remote JCA scenario:*
> * There are 4 Wildfly servers
> * Servers 1 and 3 are used as messaging brokers - they are called JMS servers
> * Servers 2 and 4 have MDBs and RA configured to connect to JMS servers. These servers are called MDB servers.
> * External standalone producer sends messages to server 1 to InQueue
> * MDBs on MBD severs receive messages from InQueue and send them to OutQueue
> * External standalone receiver receives messages from server 3 from OutQueue
> * During this scenario server 1 is several times killed and restarted.
> *Expectation:* All messages sent by the standalone producer are received by the standalone receiver. There are no lost or duplicated messages.
> *Reality:* After some kills of server 1, message flow coming from the standalone producer is broken and the receiver does not receive any messages in specified timeout.
> *Critical* priority was chosen because the regression has been detected only in this particular test case so far. We ran only nightly testing scope. After that we run full testing, we will know more about impact of this issue.
> *Technical details:*
> At some point following log messages start to occur in the test log. It seems that they are related to malfunction of communication among Artemis brokers what causes that message flow is broken.
> {code}
> 21:55:51,724 WARN [org.apache.activemq.artemis.core.server] (Thread-1 (ActiveMQ-client-global-threads)) AMQ222139: MessageFlowRecordImpl [nodeID=e53da514-5953-11e8-910a-fa163e48a89a, connector=TransportConfigur
> ation(name=connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?httpUpgradeEndpoint=acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=10080&host=rhel7-la
> rge-58597, queueName=sf.my-cluster.e53da514-5953-11e8-910a-fa163e48a89a, queue=QueueImpl[name=sf.my-cluster.e53da514-5953-11e8-910a-fa163e48a89a, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=
> d80bf5bd-5953-11e8-ac10-fa163e48a89a]]@756a727a, isClosed=false, reset=true]::Remote queue binding jms.queue.DLQe53da514-5953-11e8-910a-fa163e48a89a has already been bound in the post office. Most likely cause f
> or this is you have a loop in your cluster due to cluster max-hops being too large or you have multiple cluster connections to the same nodes using overlapping addresses
> {code}
> {code}
> 21:55:53,078 ERROR [org.apache.activemq.artemis.core.server] (Thread-6 (ActiveMQ-client-global-threads)) AMQ224037: cluster connection Failed to handle message: java.lang.IllegalStateException: Cannot find binding for jms.queue.InQueuee53da514-5953-11e8-910a-fa163e48a89a on ClusterConnectionImpl@14989488[nodeUUID=d80bf5bd-5953-11e8-ac10-fa163e48a89a, connector=TransportConfiguration(name=connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?httpUpgradeEndpoint=acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=8080&host=rhel7-large-58597, address=jms, server=ActiveMQServerImpl::serverUUID=d80bf5bd-5953-11e8-ac10-fa163e48a89a]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.doConsumerCreated(ClusterConnectionImpl.java:1294) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.handleNotificationMessage(ClusterConnectionImpl.java:1029) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.onMessage(ClusterConnectionImpl.java:1004) [artemis-server-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1001) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:49) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1124) [artemis-core-client-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-011.jar:1.5.5.jbossorg-011]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months