Intermittent failures in MixedDomainDeployment700TestCase --------------------------------------------------------- Key: WFLY-13684 URL: https://issues.redhat.com/browse/WFLY-13684 Project: WildFly Issue Type: Bug Components: JMS, Management Reporter: Brian Stansberry Assignee: Emmanuel Hugonnet Priority: Major Lots of failures starting July 16, 2020: https://ci.wildfly.org/project.html?projectId=WF&buildTypeId=&tab... This is the first failure in a set of 12-13 failures. Odd thing is nothing was merged into master after July 12, so why these started failing is unclear. WFLY-12649 was a similar issue-, but there it was known that the version of Artemis of EAP 7.0.0 did not work properly on OpenJ9-. This current issue is affecting various jobs running different OpenJDK versions. I'll send up a PR ignoring this test. The problem details: {code} java.lang.AssertionError: {"WFLYDC0074: Operation failed or was rolled back on all servers. Server failures:" => {"server-group" => {"other-server-group" => {"host" => {"slave" => {"server-one" => {"WFLYCTL0062: Composite operation failed and was rolled back. Steps that failed:" => {"Operation step-1" => { "WFLYCTL0180: Services with missing/unavailable dependencies" => ["jboss.naming.context.java.module.test.test.DefaultJMSConnectionFactory is missing [jboss.naming.context.java.jboss.DefaultJMSConnectionFactory]"], "WFLYCTL0288: One or more services were unable to start due to one or more indirect dependencies not being available." => { "Services that were unable to start:" => [ "jboss.deployment.unit.\"test.war\".CdiValidatorFactoryService", "jboss.deployment.unit.\"test.war\".WeldStartService", "jboss.deployment.unit.\"test.war\".component.\"com.sun.faces.config.ConfigureListener\".START", "jboss.deployment.unit.\"test.war\".component.\"com.sun.faces.config.ConfigureListener\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"javax.faces.webapp.FacetTag\".START", "jboss.deployment.unit.\"test.war\".component.\"javax.faces.webapp.FacetTag\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"javax.servlet.jsp.jstl.tlv.PermittedTaglibsTLV\".START", "jboss.deployment.unit.\"test.war\".component.\"javax.servlet.jsp.jstl.tlv.PermittedTaglibsTLV\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"javax.servlet.jsp.jstl.tlv.ScriptFreeTLV\".START", "jboss.deployment.unit.\"test.war\".component.\"javax.servlet.jsp.jstl.tlv.ScriptFreeTLV\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"org.jboss.weld.servlet.WeldInitialListener\".START", "jboss.deployment.unit.\"test.war\".component.\"org.jboss.weld.servlet.WeldInitialListener\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"org.jboss.weld.servlet.WeldTerminalListener\".START", "jboss.deployment.unit.\"test.war\".component.\"org.jboss.weld.servlet.WeldTerminalListener\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".deploymentCompleteService", "jboss.deployment.unit.\"test.war\".jndiDependencyService", "jboss.undertow.deployment.default-server.default-host./test", "jboss.undertow.deployment.default-server.default-host./test.UndertowDeploymentInfoService" ], "Services that may be the cause:" => [ "jboss.clustering.singleton.server.default", "jboss.naming.context.java.jboss.DefaultJMSConnectionFactory" ] } }}}}}}}}} {code} In the server.log of the /host=slave/server=server-one server: {code} 2020-07-17 08:48:06,358 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=true,journalDirectory=/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/journal,bindingsDirectory=/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/bindings,largeMessagesDirectory=/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/largemessages,pagingDirectory=/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/paging) 2020-07-17 08:48:06,378 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ221012: Using AIO Journal 2020-07-17 08:48:06,478 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE 2020-07-17 08:48:06,481 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ 2020-07-17 08:48:06,509 WARN [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ222010: Critical IO Error, shutting down the server. file=AIOSequentialFile:/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/journal/activemq-data-1.amq.tmp, message=Cannot open file:Invalid argument: java.io.IOException: Cannot open file:Invalid argument at org.apache.activemq.artemis.jlibaio.LibaioContext.open(Native Method) at org.apache.activemq.artemis.jlibaio.LibaioContext.openFile(LibaioContext.java:291) at org.apache.activemq.artemis.jlibaio.LibaioContext.openFile(LibaioContext.java:274) at org.apache.activemq.artemis.core.io.aio.AIOSequentialFile.open(AIOSequentialFile.java:133) at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.createFile0(JournalFilesRepository.java:590) at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.createFile(JournalFilesRepository.java:543) at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.ensureMinFiles(JournalFilesRepository.java:206) at org.apache.activemq.artemis.core.journal.impl.JournalImpl.setUpCurrentFile(JournalImpl.java:2652) at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1728) at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1126) at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1110) at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.loadMessageJournal(JournalStorageManager.java:1260) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:1701) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:1595) at org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:60) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:393) at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:381) at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:199) at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:63) at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:97) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.jboss.threads.JBossThread.run(JBossThread.java:320) {code}