[jboss-jira] [JBoss JIRA] (WFLY-10068) [Artemis upgrade] Artemis creates and use NODE_MANAGER table even though HA is not configured

Thu Mar 22 06:15:00 EDT 2018

    [ https://issues.jboss.org/browse/WFLY-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13549553#comment-13549553 ] 

Francesco Nigro edited comment on WFLY-10068 at 3/22/18 6:14 AM:
-----------------------------------------------------------------

[~mnovak] Yes, so that's the explanation: the file-based and jdbc-based HA (with Shared-Store) works very similarly and indeed the file-based version is actually doing the same ie:
- try to create the lock file even if HA is not configured
- try to acquire the live lock on it

But with a subtle difference: the lock acquisition timeout is infinite and not configurable (ie DEFAULT_JOURNAL_LOCK_ACQUISITION_TIMEOUT = -1) and is common that the file lock created is owned exclusively: in both cases it means that exception won't be raised.
The jdbc version instead is making uses of a configurable lock timeout with a timeout of 60 seconds, making the same issue evident.
I'm seeing 3 ways to deal with this issue:
# create/validate that for the file-based version is happening the same and fix both by checking if any HA is configured on the broker instance [longer to fix but optimal solution]
# makes the jdbc lock timeout = -1 and not configurable anymore [simpler but suboptimal]
# when no JDBC HA is configured uses the file-based version instead [very very dirty]

I will ask [~ataylor] re how to check the HA config and to see which one is the better to implement, but probably need to raise the issue for the file-based version too.

was (Author: fnigro):
[~mnovak] Yes, so that's the explanation: the file-based and jdbc-based HA (with Shared-Store) works very similarly and indeed the file-based version is actually doing the same ie:
- try to create the lock file even if HA is not configured
- try to acquire the live lock on it

But with a subtle difference: the lock acquisition timeout is infinite and not configurable (ie DEFAULT_JOURNAL_LOCK_ACQUISITION_TIMEOUT = -1), hence it won't never throw that exception.
The jdbc version instead is making uses of a configurable lock timeout with a timeout of 60 seconds, making the same issue evident.
I'm seeing 3 ways to deal with this issue:
# create/validate that for the file-based version is happening the same and fix both by checking if any HA is configured on the broker instance [longer to fix but optimal solution]
# makes the jdbc lock timeout = -1 and not configurable anymore [simpler but suboptimal]
# when no JDBC HA is configured uses the file-based version instead [very very dirty]

I will ask [~ataylor] re how to check the HA config and to see which one is the better to implement, but probably need to raise the issue for the file-based version too.

> [Artemis upgrade] Artemis creates and use NODE_MANAGER table even though HA is not configured
> ---------------------------------------------------------------------------------------------
>
>                 Key: WFLY-10068
>                 URL: https://issues.jboss.org/browse/WFLY-10068
>             Project: WildFly
>          Issue Type: Bug
>          Components: JMS
>            Reporter: Miroslav Novak
>            Assignee: Jeff Mesnil
>            Priority: Blocker
>         Attachments: standalone-full-ha.xml
>
>
> If Artemis is not configured to use HA then it should not create and use journal-node-manager-store-table table which is normally used by live/backup pair. 
> Problem here is that especially if 2 servers are started and are using the database then administrator logically does not set journal-node-manager-store-table as HA is not used. However in the moment when both of the servers are started they use default values and both of them start to use the same table (default is NODE_MANAGER) then one of them fail with:
> {code}
> 09:20:03,246 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 74) AMQ221034: Waiting 60000 milliseconds to obtain live lock
> 09:21:03,517 ERROR [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 74) AMQ224000: Failure in initialisation: java.lang.Exception: timed out waiting for lock
> 	at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.lock(JdbcNodeManager.java:240) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.startLiveNode(JdbcNodeManager.java:346) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:65) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:522) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:461) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:376) [artemis-jms-server-2.5.0.jar:2.5.0]
> 	at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:205) [wildfly-messaging-activemq-13.0.0.Alpha1-SNAPSHOT.jar:13.0.0.Alpha1-SNAPSHOT]
> 	at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:64) [wildfly-messaging-activemq-13.0.0.Alpha1-SNAPSHOT.jar:13.0.0.Alpha1-SNAPSHOT]
> 	at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:99) [wildfly-messaging-activemq-13.0.0.Alpha1-SNAPSHOT.jar:13.0.0.Alpha1-SNAPSHOT]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_131]
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_131]
> 	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 	at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_131]
> 	at org.jboss.threads.JBossThread.run(JBossThread.java:485) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 09:21:03,520 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 74) MSC000001: Failed to start service jboss.messaging-activemq.default.jms.manager: org.jboss.msc.service.StartException in service jboss.messaging-activemq.default.jms.manager: java.lang.Exception: timed out waiting for lock
> 	at org.wildfly.extension.messaging.activemq.jms.JMSService.lambda$doStart$0(JMSService.java:141) [wildfly-messaging-activemq-13.0.0.Alpha1-SNAPSHOT.jar:13.0.0.Alpha1-SNAPSHOT]
> 	at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.callActivationFailureListeners(ActiveMQServerImpl.java:1908) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:82) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:522) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:461) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:376) [artemis-jms-server-2.5.0.jar:2.5.0]
> 	at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:205) [wildfly-messaging-activemq-13.0.0.Alpha1-SNAPSHOT.jar:13.0.0.Alpha1-SNAPSHOT]
> 	at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:64) [wildfly-messaging-activemq-13.0.0.Alpha1-SNAPSHOT.jar:13.0.0.Alpha1-SNAPSHOT]
> 	at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:99) [wildfly-messaging-activemq-13.0.0.Alpha1-SNAPSHOT.jar:13.0.0.Alpha1-SNAPSHOT]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_131]
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_131]
> 	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> 	at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_131]
> 	at org.jboss.threads.JBossThread.run(JBossThread.java:485) [jboss-threads-2.3.1.Final.jar:2.3.1.Final]
> Caused by: java.lang.Exception: timed out waiting for lock
> 	at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.lock(JdbcNodeManager.java:240) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.jdbc.JdbcNodeManager.startLiveNode(JdbcNodeManager.java:346) [artemis-server-2.5.0.jar:2.5.0]
> 	at org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:65) [artemis-server-2.5.0.jar:2.5.0]
> 	... 14 more
> {code}

--
This message was sent by Atlassian JIRA
(v7.5.0#75005)