[JBoss JIRA] (WFWIP-28) [Artemis 2.x upgrade] Unexptected crash of server in SOAK test
by Miroslav Novak (JIRA)
[ https://issues.jboss.org/browse/WFWIP-28?page=com.atlassian.jira.plugin.s... ]
Miroslav Novak commented on WFWIP-28:
-------------------------------------
[~fnigro] Started servers takes 4GB each, they have {{-Xms4g -Xmx4g}}. Then maven surefire has other 2GB. However provisioned Jenkins slave has only 8GB. I think this is the cause whey OOM killer is triggered.
Could you modify {{NewSoakTestCase.startServer}} method and replace:
{code}
s = s.replaceAll("-Xms.*-Xmx.*", "-Xms4g -Xmx4g");
{code}
by:
{code}
s = s.replaceAll("-Xms.*-Xmx.*", "-Xms2g -Xmx2g");
{code}
> [Artemis 2.x upgrade] Unexptected crash of server in SOAK test
> --------------------------------------------------------------
>
> Key: WFWIP-28
> URL: https://issues.jboss.org/browse/WFWIP-28
> Project: WildFly WIP
> Issue Type: Bug
> Components: Artemis
> Reporter: Miroslav Novak
> Assignee: Martyn Taylor
> Priority: Blocker
> Labels: feature-branch-blocker
> Attachments: sosreport-rvaisdebug.asd-20180807101658.tar.xz
>
>
> After ~13 hours there is unexpected crash of one server in SOAK test. There is no error/warning in the logs.
> Test Scenario:
> * Start 2 servers
> * Client sends messages to input queue. Messages then go through:
> * One server to another through MDB reading and sending them from remote container through resource adapter
> * Messages are forwarded from one server to another over JMS bridge and back over Core bridge
> * Messages have JMSReplyTo defined with a temporary queue, that is filled with responses for the client
> * Messages are read from the destination with stateless EJB and sent back to clients
> * Client reads the messages after the pass through all the soak modules.
> Pass Criteria: In the last step receiver consumes all messages sent by producer.
> Actual Result:
> After ~13 hours 1st server suddenly crashes. There is no error/warning in server logs.
> Issue was hit with Artemis 2.5.0 with https://github.com/jmesnil/wildfly/tree/WFLY-9407_upgrade_artemis_2.4.0_w... (commit 51dd8102f103ccb0470a3cfc8713d3f9bdb1b65d)
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFLY-10630) HttpSessionListener.sessionDestroyed() not called if session invalidated in another WAR
by Bartosz Baranowski (JIRA)
[ https://issues.jboss.org/browse/WFLY-10630?page=com.atlassian.jira.plugin... ]
Bartosz Baranowski edited comment on WFLY-10630 at 8/8/18 6:24 AM:
-------------------------------------------------------------------
This happens because session no longer( I think ) is invalidated, it is transfered under different ID to another context. At least is seems so.
This happens in https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr...
and ends up in https://github.com/undertow-io/undertow/blob/master/servlet/src/main/java...
As is, https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr... is being called only once. Oddly enough this is not true for inifinspan sessions that back this up.
Am not sure if its by design regression or unintentional.
[~swd847] ^^ ?
was (Author: baranowb):
This happens because session no longer( I think ) is invalidated, it is transfered under different ID to another context. At least is seems so.
This happens in https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr...
and ends up in https://github.com/undertow-io/undertow/blob/master/servlet/src/main/java...
As is, https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr... is being called only once. Oddly enough this is not true for inifinspan sessions that back this up.
[~swd847] ^^ ?
> HttpSessionListener.sessionDestroyed() not called if session invalidated in another WAR
> ---------------------------------------------------------------------------------------
>
> Key: WFLY-10630
> URL: https://issues.jboss.org/browse/WFLY-10630
> Project: WildFly
> Issue Type: Bug
> Components: Web (Undertow)
> Affects Versions: 10.0.0.Final, 13.0.0.Final
> Environment: Windows 10, Java 1.8.0_131
> Reproducible with both WildFly-10.0.0.Final and Wildfly-13.0.0.Final
> Reporter: Bernhard Kabelka
> Assignee: Bartosz Baranowski
> Attachments: LoginForm.png, roles.properties, sessionlistener-test.ear, standalone.xml, users.properties
>
>
> For sessions shared across different WARs in a single EAR, the notification of HttpSessionListener works differently in WildFly 10.0.0.Final (and Wildfly 13.0.0.Final) than it it used to work in WildFly 8.2.0.Final:
> I have an EAR containing two WARs with enabled session sharing across the WARs. Basically, one WAR contains the web UI, and the other WAR contains the REST interfaces for AJAX calls made by the UI. The user authenticates against the UI-WAR. On logout, a REST method in the AJAX-WAR is triggered which calls HttpSession.invalidate() on the user session.
> In WildFly 8.2.0.Final, a HttpSessionListener in the UI-WAR gets notified immediately about session creation and destruction.
> In WildFly 13.0.0.Final, however, a HttpSessionListener in either WAR only gets one of the two notifications:
> * In the UI-WAR, I get a notification about the created session immediately when the login form is loaded. However, I do not receive any notification about the session destruction (unless it times out).
> * In the AJAX-WAR, I do not get any notification about the session creation at all, but I immediately receive a notification about the session destruction.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFLY-10630) HttpSessionListener.sessionDestroyed() not called if session invalidated in another WAR
by Bartosz Baranowski (JIRA)
[ https://issues.jboss.org/browse/WFLY-10630?page=com.atlassian.jira.plugin... ]
Bartosz Baranowski edited comment on WFLY-10630 at 8/8/18 6:24 AM:
-------------------------------------------------------------------
This happens because session no longer( I think ) is invalidated, it is transfered under different ID to another context. At least is seems so.
This happens in https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr...
and ends up in https://github.com/undertow-io/undertow/blob/master/servlet/src/main/java...
As is, https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr... is being called only once. Oddly enough this is not true for inifinspan sessions that back this up.
I am not sure if its by design regression or unintentional.
[~swd847] ^^ ?
was (Author: baranowb):
This happens because session no longer( I think ) is invalidated, it is transfered under different ID to another context. At least is seems so.
This happens in https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr...
and ends up in https://github.com/undertow-io/undertow/blob/master/servlet/src/main/java...
As is, https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr... is being called only once. Oddly enough this is not true for inifinspan sessions that back this up.
Am not sure if its by design regression or unintentional.
[~swd847] ^^ ?
> HttpSessionListener.sessionDestroyed() not called if session invalidated in another WAR
> ---------------------------------------------------------------------------------------
>
> Key: WFLY-10630
> URL: https://issues.jboss.org/browse/WFLY-10630
> Project: WildFly
> Issue Type: Bug
> Components: Web (Undertow)
> Affects Versions: 10.0.0.Final, 13.0.0.Final
> Environment: Windows 10, Java 1.8.0_131
> Reproducible with both WildFly-10.0.0.Final and Wildfly-13.0.0.Final
> Reporter: Bernhard Kabelka
> Assignee: Bartosz Baranowski
> Attachments: LoginForm.png, roles.properties, sessionlistener-test.ear, standalone.xml, users.properties
>
>
> For sessions shared across different WARs in a single EAR, the notification of HttpSessionListener works differently in WildFly 10.0.0.Final (and Wildfly 13.0.0.Final) than it it used to work in WildFly 8.2.0.Final:
> I have an EAR containing two WARs with enabled session sharing across the WARs. Basically, one WAR contains the web UI, and the other WAR contains the REST interfaces for AJAX calls made by the UI. The user authenticates against the UI-WAR. On logout, a REST method in the AJAX-WAR is triggered which calls HttpSession.invalidate() on the user session.
> In WildFly 8.2.0.Final, a HttpSessionListener in the UI-WAR gets notified immediately about session creation and destruction.
> In WildFly 13.0.0.Final, however, a HttpSessionListener in either WAR only gets one of the two notifications:
> * In the UI-WAR, I get a notification about the created session immediately when the login form is loaded. However, I do not receive any notification about the session destruction (unless it times out).
> * In the AJAX-WAR, I do not get any notification about the session creation at all, but I immediately receive a notification about the session destruction.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFLY-10630) HttpSessionListener.sessionDestroyed() not called if session invalidated in another WAR
by Bartosz Baranowski (JIRA)
[ https://issues.jboss.org/browse/WFLY-10630?page=com.atlassian.jira.plugin... ]
Bartosz Baranowski commented on WFLY-10630:
-------------------------------------------
This happens because session no longer( I think ) is invalidated, it is transfered under different ID to another context. At least is seems so.
This happens in https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr...
and ends up in https://github.com/undertow-io/undertow/blob/master/servlet/src/main/java...
As is, https://github.com/wildfly/wildfly/blob/master/clustering/web/undertow/sr... is being called only once. Oddly enough this is not true for inifinspan sessions that back this up.
[~swd847] ^^ ?
> HttpSessionListener.sessionDestroyed() not called if session invalidated in another WAR
> ---------------------------------------------------------------------------------------
>
> Key: WFLY-10630
> URL: https://issues.jboss.org/browse/WFLY-10630
> Project: WildFly
> Issue Type: Bug
> Components: Web (Undertow)
> Affects Versions: 10.0.0.Final, 13.0.0.Final
> Environment: Windows 10, Java 1.8.0_131
> Reproducible with both WildFly-10.0.0.Final and Wildfly-13.0.0.Final
> Reporter: Bernhard Kabelka
> Assignee: Bartosz Baranowski
> Attachments: LoginForm.png, roles.properties, sessionlistener-test.ear, standalone.xml, users.properties
>
>
> For sessions shared across different WARs in a single EAR, the notification of HttpSessionListener works differently in WildFly 10.0.0.Final (and Wildfly 13.0.0.Final) than it it used to work in WildFly 8.2.0.Final:
> I have an EAR containing two WARs with enabled session sharing across the WARs. Basically, one WAR contains the web UI, and the other WAR contains the REST interfaces for AJAX calls made by the UI. The user authenticates against the UI-WAR. On logout, a REST method in the AJAX-WAR is triggered which calls HttpSession.invalidate() on the user session.
> In WildFly 8.2.0.Final, a HttpSessionListener in the UI-WAR gets notified immediately about session creation and destruction.
> In WildFly 13.0.0.Final, however, a HttpSessionListener in either WAR only gets one of the two notifications:
> * In the UI-WAR, I get a notification about the created session immediately when the login form is loaded. However, I do not receive any notification about the session destruction (unless it times out).
> * In the AJAX-WAR, I do not get any notification about the session creation at all, but I immediately receive a notification about the session destruction.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFCORE-4002) Some security tests fail on closed channel exception
by Jan Kalina (JIRA)
[ https://issues.jboss.org/browse/WFCORE-4002?page=com.atlassian.jira.plugi... ]
Jan Kalina commented on WFCORE-4002:
------------------------------------
1) AuditLog:
"syslog-test" resource is removed and connection to the syslog server closed before the server accept the message
Adding sleep Thread.sleep(500); into AuditLogHandlerBootEnabledTestCase#runSyslogTest():397 ensure green result.
Probably some sort of waiting before the syslog connection is closed is missing...
{code}
JAVA_HOME=/opt/jdk-11 mvn clean test -Dtest=org.jboss.as.domain.management.security.auditlog.AuditLogHandlerBootEnabledTestCase#testSyslogTlsOctetCountingClientAuth
{code}
> Some security tests fail on closed channel exception
> ----------------------------------------------------
>
> Key: WFCORE-4002
> URL: https://issues.jboss.org/browse/WFCORE-4002
> Project: WildFly Core
> Issue Type: Bug
> Components: Security, Test Suite
> Affects Versions: 6.0.0.Alpha5
> Reporter: Richard Opalka
> Assignee: Jan Kalina
> Priority: Blocker
> Labels: jdk11
> Fix For: 6.0.0.Alpha6
>
>
> With latest JDK11 (build 11-ea+25) the following tests fail due to closed channel exception:
> * domain-management/src/test/java/org/jboss/as/domain/management/security/auditlog/AuditLogHandlerBootEnabledTestCase.java
> * testsuite/domain/src/test/java/org/jboss/as/test/integration/domain/HTTPSManagementInterfaceTestCase.java
> * testsuite/elytron/src/test/java/org/wildfly/test/integration/elytron/sasl/mgmt/KerberosNativeMgmtSaslTestCase.java
> * testsuite/elytron/src/test/java/org/wildfly/test/integration/elytron/sasl/mgmt/ScramPlusMgmtSaslTestCase.java
> * testsuite/manualmode/src/test/java/org/wildfly/core/test/standalone/mgmt/HTTPSManagementInterfaceTestCase.java
> Is there a known SSL issue in JDK11? Could somebody from our security team investigate this issue?
> When these tests will start passing WFCORE test suite will be passing 100% of tests on JDK11.
> Potential investigator from our security team may want to include this PR:
> https://github.com/wildfly/wildfly-core/pull/3420
> to fix known JDK11 issues.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFLY-10474) AMQ119012: Timed out waiting to receive initial broadcast from cluster
by Jeff Mesnil (JIRA)
[ https://issues.jboss.org/browse/WFLY-10474?page=com.atlassian.jira.plugin... ]
Jeff Mesnil commented on WFLY-10474:
------------------------------------
[~pferraro] do you have an idea about the cause of the regression?
> AMQ119012: Timed out waiting to receive initial broadcast from cluster
> ----------------------------------------------------------------------
>
> Key: WFLY-10474
> URL: https://issues.jboss.org/browse/WFLY-10474
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Affects Versions: 13.0.0.Beta1
> Reporter: Erich Duda
> Assignee: Jeff Mesnil
> Priority: Blocker
> Labels: blocker-WF14
>
> In messaging HA scenarios we see that tests sometimes fail because of exception \[1\]. The exception is thrown from {{ServerLocatorImpl::createSessionFactory}} which waits until initial broadcast from cluster is received, but it is not received in 10 seconds timeout. In following scenario the issue causes split brain - both live and backup are active at the same time.
> *Scenario*
> * There are two Wildfly/Artemis servers configured as Live-Backup pair
> * Live server is killed
> * Backup server becomes new Live server
> * Live server is restarted and failback is performed
> *Expectation:* Failback performs successfully. Original Live becomes Live again and original Backup becomes Backup again.
> *Reality:* After the Live is restarted, it does not detect that there is already another active Live with the same nodeId and it activates (becomes Live) so both Live and Backup are active at the same time.
> *Technical details*
> I found out that original Live server does not detect that its Backup is active, because SharedNothingLiveActivation::isNodeIdUsed returns false. I added more logs to this method and found out that the {{false}} is returned from line \[2\] after that the exception \[1\] is thrown.
> *Investigation report*
> I tried to downgrade Artemis to {{1.5.5.jbossorg-010}} version but I hit the same issue. So the issue is not caused by recent Artemis upgrade.
> It is regression against EAP 7.1.0.GA but it is not regression against Wildfly 12.
> We see the issue only with JGroups. I tried to run the test with Netty discovery and it passed 50 times in row.
> \[1\]
> {code}
> ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=AMQ119012: Timed out waiting to receive initial broadcast from cluster]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:749) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:627) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connectNoWarnings(ServerLocatorImpl.java:633) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.isNodeIdUsed(SharedNothingLiveActivation.java:280) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.run(SharedNothingLiveActivation.java:90) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:539) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:485) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:413) [artemis-jms-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:205) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:64) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:99) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_171]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_171]
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> at org.jboss.threads.JBossThread.run(JBossThread.java:485) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> {code}
> \[2\] https://github.com/rh-messaging/jboss-activemq-artemis/blob/465eb8733e465...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFWIP-89) [Artemis 2.x Upgrade] Clients do not failover to backup when force-failover operation is used with JDBC store
by Francesco Nigro (JIRA)
[ https://issues.jboss.org/browse/WFWIP-89?page=com.atlassian.jira.plugin.s... ]
Francesco Nigro edited comment on WFWIP-89 at 8/8/18 5:27 AM:
--------------------------------------------------------------
I have noticed these logs on the node 2 log:
{code:java}
11:54:36,296 ERROR [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=00d22eb4-9712-11e8-bc7a-fa163e5855f4) AMQ224000: Failure in initialisation: java.lang.IllegalStateException: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:338) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.readSharedState(JdbcNodeManager.java:523) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.awaitLiveNode(JdbcNodeManager.java:380) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.SharedStoreBackupActivation.run(SharedStoreBackupActivation.java:77) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$ActivationThread.run(ActiveMQServerImpl.java:3236) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
Caused by: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.tryLock(BaseWrapperManagedConnection.java:402)
at org.jboss.jca.adapters.jdbc.WrappedConnection.lock(WrappedConnection.java:161)
at org.jboss.jca.adapters.jdbc.WrappedConnection.setTransactionIsolation(WrappedConnection.java:1007)
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:316) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
... 4 more
{code}
It shows that the backup node isn't able to come up because isn't able to read the state of the live node due to an issue of the EAP SQL connection pools.
It isn't a broker issue and it looks like a timing issue with the reaper thread used by the connection pool.
I believe that is intermittent, given that is dependent by timing.
[~mnovak] I think that it isn't a regression and not caused by Artemis.
was (Author: fnigro):
I have noticed these logs on the node 2 log:
{code:java}
11:54:36,296 ERROR [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=00d22eb4-9712-11e8-bc7a-fa163e5855f4) AMQ224000: Failure in initialisation: java.lang.IllegalStateException: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:338) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.readSharedState(JdbcNodeManager.java:523) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.awaitLiveNode(JdbcNodeManager.java:380) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.SharedStoreBackupActivation.run(SharedStoreBackupActivation.java:77) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$ActivationThread.run(ActiveMQServerImpl.java:3236) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
Caused by: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.tryLock(BaseWrapperManagedConnection.java:402)
at org.jboss.jca.adapters.jdbc.WrappedConnection.lock(WrappedConnection.java:161)
at org.jboss.jca.adapters.jdbc.WrappedConnection.setTransactionIsolation(WrappedConnection.java:1007)
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:316) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
... 4 more
{code}
It shows that the backup node isn't able to come up because isn't able to read the state of the live node due to an issue of the EAP SQL connection pools. I think that's not a broker issue and it looks like a timing issue with the reaper thread used by the connection pool.
I believe that is intermittent, given that is dependent by timing.
> [Artemis 2.x Upgrade] Clients do not failover to backup when force-failover operation is used with JDBC store
> -------------------------------------------------------------------------------------------------------------
>
> Key: WFWIP-89
> URL: https://issues.jboss.org/browse/WFWIP-89
> Project: WildFly WIP
> Issue Type: Bug
> Components: Artemis
> Reporter: Martin Styk
> Assignee: Martyn Taylor
> Attachments: org.jboss.qa.artemis.test.failover.DedicatedFailoverTestCase20.testForceFailoverOperation.zip
>
>
> Dedicated topology (1live, 1 backup) with JDBC store (oracle12c).
> Force failover operation is called on live node. Live artemis server stops, however backup doesn't start.
> This issue is intermittent.
> Attaching logs.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFCORE-3962) Starting WFLY scripts with JDK 11 blows up with ModuleNotFoundException java.se
by Rostislav Svoboda (JIRA)
[ https://issues.jboss.org/browse/WFCORE-3962?page=com.atlassian.jira.plugi... ]
Rostislav Svoboda commented on WFCORE-3962:
-------------------------------------------
Related PR - https://github.com/wildfly/wildfly-core/pull/3420
> Starting WFLY scripts with JDK 11 blows up with ModuleNotFoundException java.se
> -------------------------------------------------------------------------------
>
> Key: WFCORE-3962
> URL: https://issues.jboss.org/browse/WFCORE-3962
> Project: WildFly Core
> Issue Type: Bug
> Components: Management, Modules, Scripts
> Affects Versions: 6.0.0.Alpha3
> Reporter: Matej Novotny
> Assignee: Richard Opalka
> Priority: Blocker
> Labels: Java11, blocker-WF14
>
> As per request, copying MODULES-372 to WFCORE as well.
> Starting WFLY with JDK 11 using {{standalone.sh}} yields:
> {code}
> org.jboss.modules.ModuleNotFoundException: java.se
> at org.jboss.modules.Module.addPaths(Module.java:1266)
> at org.jboss.modules.Module.link(Module.java:1622)
> at org.jboss.modules.Module.relinkIfNecessary(Module.java:1650)
> at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:296)
> at org.jboss.modules.Main.main(Main.java:437)
> {code}
> Output of {{java -version}}:
> {code}
> openjdk version "11-ea" 2018-09-25
> OpenJDK Runtime Environment 18.9 (build 11-ea+21)
> OpenJDK 64-Bit Server VM 18.9 (build 11-ea+21, mixed mode)
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFWIP-89) [Artemis 2.x Upgrade] Clients do not failover to backup when force-failover operation is used with JDBC store
by Francesco Nigro (JIRA)
[ https://issues.jboss.org/browse/WFWIP-89?page=com.atlassian.jira.plugin.s... ]
Francesco Nigro edited comment on WFWIP-89 at 8/8/18 5:01 AM:
--------------------------------------------------------------
I have noticed these logs on the node 2 log:
{code:java}
11:54:36,296 ERROR [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=00d22eb4-9712-11e8-bc7a-fa163e5855f4) AMQ224000: Failure in initialisation: java.lang.IllegalStateException: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:338) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.readSharedState(JdbcNodeManager.java:523) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.awaitLiveNode(JdbcNodeManager.java:380) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.SharedStoreBackupActivation.run(SharedStoreBackupActivation.java:77) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$ActivationThread.run(ActiveMQServerImpl.java:3236) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
Caused by: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.tryLock(BaseWrapperManagedConnection.java:402)
at org.jboss.jca.adapters.jdbc.WrappedConnection.lock(WrappedConnection.java:161)
at org.jboss.jca.adapters.jdbc.WrappedConnection.setTransactionIsolation(WrappedConnection.java:1007)
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:316) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
... 4 more
{code}
It shows that the backup node isn't able to come up because isn't able to read the state of the live node due to an issue of the EAP SQL connection pools. I think that's not a broker issue and it looks like a timing issue with the reaper thread used by the connection pool.
I believe that is intermittent, given that is dependent by timing.
was (Author: fnigro):
I have noticed these logs on the node 2 logs:
{code:java}
11:54:36,296 ERROR [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=00d22eb4-9712-11e8-bc7a-fa163e5855f4) AMQ224000: Failure in initialisation: java.lang.IllegalStateException: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:338) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.readSharedState(JdbcNodeManager.java:523) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.awaitLiveNode(JdbcNodeManager.java:380) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.SharedStoreBackupActivation.run(SharedStoreBackupActivation.java:77) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$ActivationThread.run(ActiveMQServerImpl.java:3236) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
Caused by: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.tryLock(BaseWrapperManagedConnection.java:402)
at org.jboss.jca.adapters.jdbc.WrappedConnection.lock(WrappedConnection.java:161)
at org.jboss.jca.adapters.jdbc.WrappedConnection.setTransactionIsolation(WrappedConnection.java:1007)
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:316) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
... 4 more
{code}
It shows that the backup node isn't able to come up because isn't able to read the state of the live node due to an issue of the EAP SQL connection pools. I think that's not a broker issue and it looks like a timing issue with the reaper thread used by the connection pool.
I believe that is intermittent, given that is dependent by timing.
> [Artemis 2.x Upgrade] Clients do not failover to backup when force-failover operation is used with JDBC store
> -------------------------------------------------------------------------------------------------------------
>
> Key: WFWIP-89
> URL: https://issues.jboss.org/browse/WFWIP-89
> Project: WildFly WIP
> Issue Type: Bug
> Components: Artemis
> Reporter: Martin Styk
> Assignee: Martyn Taylor
> Attachments: org.jboss.qa.artemis.test.failover.DedicatedFailoverTestCase20.testForceFailoverOperation.zip
>
>
> Dedicated topology (1live, 1 backup) with JDBC store (oracle12c).
> Force failover operation is called on live node. Live artemis server stops, however backup doesn't start.
> This issue is intermittent.
> Attaching logs.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFWIP-89) [Artemis 2.x Upgrade] Clients do not failover to backup when force-failover operation is used with JDBC store
by Francesco Nigro (JIRA)
[ https://issues.jboss.org/browse/WFWIP-89?page=com.atlassian.jira.plugin.s... ]
Francesco Nigro commented on WFWIP-89:
--------------------------------------
I have noticed these logs on the node 2 logs:
{code:java}
11:54:36,296 ERROR [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=00d22eb4-9712-11e8-bc7a-fa163e5855f4) AMQ224000: Failure in initialisation: java.lang.IllegalStateException: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:338) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.readSharedState(JdbcNodeManager.java:523) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.awaitLiveNode(JdbcNodeManager.java:380) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.SharedStoreBackupActivation.run(SharedStoreBackupActivation.java:77) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$ActivationThread.run(ActiveMQServerImpl.java:3236) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
Caused by: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@6f9f8b97
at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.tryLock(BaseWrapperManagedConnection.java:402)
at org.jboss.jca.adapters.jdbc.WrappedConnection.lock(WrappedConnection.java:161)
at org.jboss.jca.adapters.jdbc.WrappedConnection.setTransactionIsolation(WrappedConnection.java:1007)
at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcSharedStateManager.readState(JdbcSharedStateManager.java:316) [artemis-server-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
... 4 more
{code}
It shows that the backup node isn't able to come up because isn't able to read the state of the live node due to an issue of the EAP SQL connection pools. I think that's not a broker issue and it looks like a timing issue with the reaper thread used by the connection pool.
I believe that is intermittent, given that is dependent by timing.
> [Artemis 2.x Upgrade] Clients do not failover to backup when force-failover operation is used with JDBC store
> -------------------------------------------------------------------------------------------------------------
>
> Key: WFWIP-89
> URL: https://issues.jboss.org/browse/WFWIP-89
> Project: WildFly WIP
> Issue Type: Bug
> Components: Artemis
> Reporter: Martin Styk
> Assignee: Martyn Taylor
> Attachments: org.jboss.qa.artemis.test.failover.DedicatedFailoverTestCase20.testForceFailoverOperation.zip
>
>
> Dedicated topology (1live, 1 backup) with JDBC store (oracle12c).
> Force failover operation is called on live node. Live artemis server stops, however backup doesn't start.
> This issue is intermittent.
> Attaching logs.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months