[JBoss JIRA] (WFWIP-91) [Artemis 2.x upgrade] artemis: artemis/hornetq: memory exhaustion via UDP and JGroups discovery
by Miroslav Novak (JIRA)
Miroslav Novak created WFWIP-91:
-----------------------------------
Summary: [Artemis 2.x upgrade] artemis: artemis/hornetq: memory exhaustion via UDP and JGroups discovery
Key: WFWIP-91
URL: https://issues.jboss.org/browse/WFWIP-91
Project: WildFly WIP
Issue Type: Bug
Components: Artemis
Reporter: Miroslav Novak
Assignee: Martyn Taylor
Priority: Blocker
When 3rd party tool send multicast messages to Artemis which is configured with UDP Disovery and JGroups Discovery, Artemis creates a huge byte array at receiving the unexpected multicast message. So, it results in heap memory exhaustion, full GC and OutOfMemoryError in the worst case.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (SWSQE-374) When starting 40> more jenkins slaves some of them fail to start
by Filip Brychta (JIRA)
Filip Brychta created SWSQE-374:
-----------------------------------
Summary: When starting 40> more jenkins slaves some of them fail to start
Key: SWSQE-374
URL: https://issues.jboss.org/browse/SWSQE-374
Project: Kiali QE
Issue Type: QE Task
Reporter: Filip Brychta
Assignee: Filip Brychta
Priority: Minor
Some slaves fail to start with following errors:
from jenkins log:
ARNING: Error in provisioning; agent=KubernetesSlave name: jenkins-slave-kiali-ui-tests-fcdsz, template=PodTemplate{inheritFrom='', name='jenkins-slave-kiali-ui-tests', namespace='jenkins-slaves', label='python kiali-ui-tests', nodeSelector='', nodeUsageMode=NORMAL, workspaceVolume=EmptyDirWorkspaceVolume [memory=false], containers=[ContainerTemplate{name='jnlp', image='docker-registry.default.svc:5000/jenkins-slaves/jenkins-slave-kiali-ui-tests', alwaysPullImage=true, workingDir='/home/jenkins', command='', args='${computer.jnlpmac} ${computer.name} ', resourceRequestCpu='', resourceRequestMemory='', resourceLimitCpu='', resourceLimitMemory='', livenessProbe=org.csanchez.jenkins.plugins.kubernetes.ContainerLivenessProbe@3106f760}], yaml=}
java.lang.IllegalStateException: Node was deleted, computer is null
at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:177)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:292)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Aug 08, 2018 8:38:46 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for agent jenkins-slave-kiali-ui-tests-fcdsz
Aug 08, 2018 8:38:46 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
SEVERE: Computer for agent is null: jenkins-slave-kiali-ui-tests-fcdsz
WARNING: Unable to move atomically, falling back to non-atomic move.
java.nio.file.NoSuchFileException: /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/atomic9143280938281774341tmp -> /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/config.xml
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:191)
at hudson.XmlFile.write(XmlFile.java:198)
at jenkins.model.Nodes.persistNode(Nodes.java:175)
at jenkins.model.Nodes.addNode(Nodes.java:144)
at jenkins.model.Jenkins.addNode(Jenkins.java:2058)
at hudson.slaves.NodeProvisioner$2.run(NodeProvisioner.java:241)
at hudson.model.Queue._withLock(Queue.java:1380)
at hudson.model.Queue.withLock(Queue.java:1257)
at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:207)
at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:61)
at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(No
WARNING: Unable to move /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/atomic9143280938281774341tmp to /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/config.xml. Attempting to delete /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/atomic9143280938281774341tmp and abandoning.
Aug 08, 2018 8:38:34 AM hudson.slaves.NodeProvisioner$2 run
WARNING: Provisioned agent Kubernetes Pod Template failed to launch
java.nio.file.NoSuchFileException: /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/atomic9143280938281774341tmp
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:409)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:206)
at hudson.XmlFile.write(XmlFile.java:198)
at jenkins.model.Nodes.persistNode(Nodes.java:175)
at jenkins.model.Nodes.addNode(Nodes.java:144)
at jenkins.model.Jenkins.addNode(Jenkins.java:2058)
at hudson.slaves.NodeProvisioner$2.run(NodeProvisioner.java:241)
>From docker log:
INFO: Handshaking
Aug 08, 2018 12:38:43 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins2.bc.jonqe.lab.eng.bos.redhat.com:39765
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: b0:58:42:9c:19:76:a1:78:81:79:d9:fc:9a:e9:19:fd
Aug 08, 2018 12:38:44 PM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection to jenkins2.bc.jonqe.lab.eng.bos.redhat.com/10.16.23.71:39765] Local headers refused by remote: Unknown client name: jenkins-slave-kiali-ui-tests-fcdsz
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkins-slave-kiali-ui-tests-fcdsz
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:609)
at hudson.remoting.Engine.run(Engine.java:469)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkins-slave-kiali-ui-tests-fcdsz
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:378)
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:433)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.nio.channels.ClosedChannelException
... 7 more
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins2.bc.jonqe.lab.eng.bos.redhat.com:39765
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP3-connect not supported, skipping
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP2-connect not supported, skipping
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:670)
at hudson.remoting.Engine.innerRun(Engine.java:634)
at hudson.remoting.Engine.run(Engine.java:469)
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFLY-10474) AMQ119012: Timed out waiting to receive initial broadcast from cluster
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/WFLY-10474?page=com.atlassian.jira.plugin... ]
Paul Ferraro resolved WFLY-10474.
---------------------------------
Fix Version/s: 14.0.0.CR1
Resolution: Cannot Reproduce
Closing per reporter's comment.
> AMQ119012: Timed out waiting to receive initial broadcast from cluster
> ----------------------------------------------------------------------
>
> Key: WFLY-10474
> URL: https://issues.jboss.org/browse/WFLY-10474
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Affects Versions: 13.0.0.Beta1
> Reporter: Erich Duda
> Assignee: Jeff Mesnil
> Priority: Blocker
> Labels: blocker-WF14
> Fix For: 14.0.0.CR1
>
>
> In messaging HA scenarios we see that tests sometimes fail because of exception \[1\]. The exception is thrown from {{ServerLocatorImpl::createSessionFactory}} which waits until initial broadcast from cluster is received, but it is not received in 10 seconds timeout. In following scenario the issue causes split brain - both live and backup are active at the same time.
> *Scenario*
> * There are two Wildfly/Artemis servers configured as Live-Backup pair
> * Live server is killed
> * Backup server becomes new Live server
> * Live server is restarted and failback is performed
> *Expectation:* Failback performs successfully. Original Live becomes Live again and original Backup becomes Backup again.
> *Reality:* After the Live is restarted, it does not detect that there is already another active Live with the same nodeId and it activates (becomes Live) so both Live and Backup are active at the same time.
> *Technical details*
> I found out that original Live server does not detect that its Backup is active, because SharedNothingLiveActivation::isNodeIdUsed returns false. I added more logs to this method and found out that the {{false}} is returned from line \[2\] after that the exception \[1\] is thrown.
> *Investigation report*
> I tried to downgrade Artemis to {{1.5.5.jbossorg-010}} version but I hit the same issue. So the issue is not caused by recent Artemis upgrade.
> It is regression against EAP 7.1.0.GA but it is not regression against Wildfly 12.
> We see the issue only with JGroups. I tried to run the test with Netty discovery and it passed 50 times in row.
> \[1\]
> {code}
> ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=AMQ119012: Timed out waiting to receive initial broadcast from cluster]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:749) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:627) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connectNoWarnings(ServerLocatorImpl.java:633) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.isNodeIdUsed(SharedNothingLiveActivation.java:280) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.run(SharedNothingLiveActivation.java:90) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:539) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:485) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:413) [artemis-jms-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:205) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:64) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:99) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_171]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_171]
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> at org.jboss.threads.JBossThread.run(JBossThread.java:485) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> {code}
> \[2\] https://github.com/rh-messaging/jboss-activemq-artemis/blob/465eb8733e465...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (DROOLS-2860) Add second level header
by Juraj Soltes (JIRA)
[ https://issues.jboss.org/browse/DROOLS-2860?page=com.atlassian.jira.plugi... ]
Juraj Soltes updated DROOLS-2860:
---------------------------------
Sprint: 2018 Week 33-35 (was: 2018 Week 30-32)
> Add second level header
> -----------------------
>
> Key: DROOLS-2860
> URL: https://issues.jboss.org/browse/DROOLS-2860
> Project: Drools
> Issue Type: Task
> Components: Scenario Simulation and Testing
> Reporter: Gabriele Cardosi
> Assignee: Gabriele Cardosi
> Labels: ScenarioSimulation
> Original Estimate: 1 day
> Remaining Estimate: 1 day
>
> Inside the grid, the scenario-specific columns are grouped in two categories: "Given" and "Expected".
> All the "Given" columns should go below the main "Given" header, and all the "Expected" ones below "Expected".
> For each column there should be, then, a specific header
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFWIP-84) [Artemis 2.x upgrade] MDB deployed on EAP 7.2.0 with new Artemis cannot connect to queue deployed on EAP6
by Justin Bertram (JIRA)
[ https://issues.jboss.org/browse/WFWIP-84?page=com.atlassian.jira.plugin.s... ]
Justin Bertram reassigned WFWIP-84:
-----------------------------------
Assignee: Justin Bertram (was: Martyn Taylor)
> [Artemis 2.x upgrade] MDB deployed on EAP 7.2.0 with new Artemis cannot connect to queue deployed on EAP6
> ---------------------------------------------------------------------------------------------------------
>
> Key: WFWIP-84
> URL: https://issues.jboss.org/browse/WFWIP-84
> Project: WildFly WIP
> Issue Type: Bug
> Components: Artemis, JMS
> Reporter: Martin Švehla
> Assignee: Justin Bertram
> Priority: Blocker
> Labels: activemq, feature-branch-blocker
>
> I'm not able to connect with MDB deployed on WF14/EAP 7.2.0 with new Artemis to destination deployed on EAP6.
> CLI for configuration EAP7:
> {code}
> /socket-binding-group=standard-sockets/remote-destination-outbound-socket-binding=remote-eap6-binding:add(port=6445)
> /subsystem=messaging-activemq/server=default/remote-connector=remote-eap6-connector:add(socket-bidning=remote-eap6-binding)
> /subsystem=messaging-activemq/server=default/pooled-connection-factory=remote-eap6:add(connectors=[remote-eap6-connector], entries=["jms/remote-eap6", "java:jboss/exported/jms/remote-eap6"], protocol-manager-factory="org.apache.activemq.artemis.core.protocol.hornetq.client.HornetQClientProtocolManagerFactory)
> {code}
> The MDB is annotated with
> {code}
> @MessageDriven(activationConfig = {
> @ActivationConfigProperty(propertyName = "useJNDI", propertyValue = "false"),
> @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
> @ActivationConfigProperty(propertyName = "destination", propertyValue = "InQueue"),
> @ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge")
> })
> @ResourceAdapter("remote-eap6")
> {code}
> This setup worked with EAP 7.1.x and EAP 7.2.0.Beta with older Artemis
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFLY-10474) AMQ119012: Timed out waiting to receive initial broadcast from cluster
by Erich Duda (JIRA)
[ https://issues.jboss.org/browse/WFLY-10474?page=com.atlassian.jira.plugin... ]
Erich Duda edited comment on WFLY-10474 at 8/8/18 8:55 AM:
-----------------------------------------------------------
[~rhusar] yes I didn't set them to be kept because both runs passed. As I wrote in my comment I wasn't able to reproduce the issue with Wildfly master.
I think this issue can be marked as resolved.
was (Author: eduda):
[~rhusar] yes I didn't set them to be kept because both runs passed. As I wrote in my comment I wasn't able to reproduce the issue with Wildfly master.
> AMQ119012: Timed out waiting to receive initial broadcast from cluster
> ----------------------------------------------------------------------
>
> Key: WFLY-10474
> URL: https://issues.jboss.org/browse/WFLY-10474
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Affects Versions: 13.0.0.Beta1
> Reporter: Erich Duda
> Assignee: Jeff Mesnil
> Priority: Blocker
> Labels: blocker-WF14
>
> In messaging HA scenarios we see that tests sometimes fail because of exception \[1\]. The exception is thrown from {{ServerLocatorImpl::createSessionFactory}} which waits until initial broadcast from cluster is received, but it is not received in 10 seconds timeout. In following scenario the issue causes split brain - both live and backup are active at the same time.
> *Scenario*
> * There are two Wildfly/Artemis servers configured as Live-Backup pair
> * Live server is killed
> * Backup server becomes new Live server
> * Live server is restarted and failback is performed
> *Expectation:* Failback performs successfully. Original Live becomes Live again and original Backup becomes Backup again.
> *Reality:* After the Live is restarted, it does not detect that there is already another active Live with the same nodeId and it activates (becomes Live) so both Live and Backup are active at the same time.
> *Technical details*
> I found out that original Live server does not detect that its Backup is active, because SharedNothingLiveActivation::isNodeIdUsed returns false. I added more logs to this method and found out that the {{false}} is returned from line \[2\] after that the exception \[1\] is thrown.
> *Investigation report*
> I tried to downgrade Artemis to {{1.5.5.jbossorg-010}} version but I hit the same issue. So the issue is not caused by recent Artemis upgrade.
> It is regression against EAP 7.1.0.GA but it is not regression against Wildfly 12.
> We see the issue only with JGroups. I tried to run the test with Netty discovery and it passed 50 times in row.
> \[1\]
> {code}
> ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=AMQ119012: Timed out waiting to receive initial broadcast from cluster]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:749) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:627) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connectNoWarnings(ServerLocatorImpl.java:633) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.isNodeIdUsed(SharedNothingLiveActivation.java:280) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.run(SharedNothingLiveActivation.java:90) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:539) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:485) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:413) [artemis-jms-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:205) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:64) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:99) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_171]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_171]
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> at org.jboss.threads.JBossThread.run(JBossThread.java:485) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> {code}
> \[2\] https://github.com/rh-messaging/jboss-activemq-artemis/blob/465eb8733e465...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFLY-10474) AMQ119012: Timed out waiting to receive initial broadcast from cluster
by Erich Duda (JIRA)
[ https://issues.jboss.org/browse/WFLY-10474?page=com.atlassian.jira.plugin... ]
Erich Duda commented on WFLY-10474:
-----------------------------------
[~rhusar] yes I didn't set them to be kept because both runs passed. As I wrote in my comment I wasn't able to reproduce the issue with Wildfly master.
> AMQ119012: Timed out waiting to receive initial broadcast from cluster
> ----------------------------------------------------------------------
>
> Key: WFLY-10474
> URL: https://issues.jboss.org/browse/WFLY-10474
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Affects Versions: 13.0.0.Beta1
> Reporter: Erich Duda
> Assignee: Jeff Mesnil
> Priority: Blocker
> Labels: blocker-WF14
>
> In messaging HA scenarios we see that tests sometimes fail because of exception \[1\]. The exception is thrown from {{ServerLocatorImpl::createSessionFactory}} which waits until initial broadcast from cluster is received, but it is not received in 10 seconds timeout. In following scenario the issue causes split brain - both live and backup are active at the same time.
> *Scenario*
> * There are two Wildfly/Artemis servers configured as Live-Backup pair
> * Live server is killed
> * Backup server becomes new Live server
> * Live server is restarted and failback is performed
> *Expectation:* Failback performs successfully. Original Live becomes Live again and original Backup becomes Backup again.
> *Reality:* After the Live is restarted, it does not detect that there is already another active Live with the same nodeId and it activates (becomes Live) so both Live and Backup are active at the same time.
> *Technical details*
> I found out that original Live server does not detect that its Backup is active, because SharedNothingLiveActivation::isNodeIdUsed returns false. I added more logs to this method and found out that the {{false}} is returned from line \[2\] after that the exception \[1\] is thrown.
> *Investigation report*
> I tried to downgrade Artemis to {{1.5.5.jbossorg-010}} version but I hit the same issue. So the issue is not caused by recent Artemis upgrade.
> It is regression against EAP 7.1.0.GA but it is not regression against Wildfly 12.
> We see the issue only with JGroups. I tried to run the test with Netty discovery and it passed 50 times in row.
> \[1\]
> {code}
> ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=AMQ119012: Timed out waiting to receive initial broadcast from cluster]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:749) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:627) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connectNoWarnings(ServerLocatorImpl.java:633) [artemis-core-client-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.isNodeIdUsed(SharedNothingLiveActivation.java:280) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation.run(SharedNothingLiveActivation.java:90) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:539) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:485) [artemis-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:413) [artemis-jms-server-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:205) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:64) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:99) [wildfly-messaging-activemq-13.0.0.Beta2-SNAPSHOT.jar:13.0.0.Beta2-SNAPSHOT]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_171]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_171]
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> at org.jboss.threads.JBossThread.run(JBossThread.java:485) [jboss-threads-2.3.2.Final.jar:2.3.2.Final]
> {code}
> \[2\] https://github.com/rh-messaging/jboss-activemq-artemis/blob/465eb8733e465...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (WFWIP-86) [Artemis 2.x upgrade] Duplicated messages in scenario with stop/start JMS bridge
by Justin Bertram (JIRA)
[ https://issues.jboss.org/browse/WFWIP-86?page=com.atlassian.jira.plugin.s... ]
Justin Bertram commented on WFWIP-86:
-------------------------------------
bq. Ok, was with EAP 7.1 against EAP 7.1?
Yes. I just changed the test to use containers 1 & 2 rather than 1 & 3. Nothing else was changed.
{noformat}
diff --git a/jboss-hornetq-testsuite/common-tests/src/test/java/org/jboss/qa/hornetq/test/bridges/JMSBridgeTestCase.java b/jboss-hornetq-testsuite/common-tests/src/test/java/org/jboss/qa/hornetq/test/bridges/JMSBridgeTestCase.java
index c2304e768..fb7911b7b 100644
--- a/jboss-hornetq-testsuite/common-tests/src/test/java/org/jboss/qa/hornetq/test/bridges/JMSBridgeTestCase.java
+++ b/jboss-hornetq-testsuite/common-tests/src/test/java/org/jboss/qa/hornetq/test/bridges/JMSBridgeTestCase.java
@@ -243,7 +243,7 @@ public class JMSBridgeTestCase extends HornetQTestCase {
public void testStartStopJMSBridge() throws Exception {
Container inServer = container(1);
- Container outServer = container(3);
+ Container outServer = container(2);
prepareServers(inServer, outServer, Constants.QUALITY_OF_SERVICE.ONCE_AND_ONLY_ONCE);
outServer.start();
{noformat}
> [Artemis 2.x upgrade] Duplicated messages in scenario with stop/start JMS bridge
> --------------------------------------------------------------------------------
>
> Key: WFWIP-86
> URL: https://issues.jboss.org/browse/WFWIP-86
> Project: WildFly WIP
> Issue Type: Bug
> Components: Artemis
> Reporter: Miroslav Novak
> Assignee: Martyn Taylor
> Priority: Blocker
>
> There are duplicated messages in scenario where JMS bridge is stopped and started.
> Test Scenario:
> * Start 2 servers and configure JMS from node-1 to node-3
> * Start sending messages to queue InQueue to node-1
> * Stop JMS bridge using management operation (CLI)
> * Wait for 5 sec
> * Start JMS bridge using management operation
> * Start receiving mesage from node-3 from OutQueue
> Expected results:
> There are no lost/duplicated mesages.
> Actual Result:
> Receiver got duplicated messages.
> Link to failed Jenkins run:
> https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/eap-7.x-messag...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months