Intermittent failures in MixedDomainDeployment700TestCase.testJsfWorks ---------------------------------------------------------------------- Key: WFLY-13684 URL: https://issues.redhat.com/browse/WFLY-13684 Project: WildFly Issue Type: Bug Components: JMS, Management Reporter: Brian Stansberry Assignee: Emmanuel Hugonnet Priority: Major Lots of failures starting July 16, 2020: https://ci.wildfly.org/project.html?projectId=WF&buildTypeId=&tab... This is the first failure in a set of 12-13 failures. Odd thing is nothing was merged into master after July 12, so why these started failing is unclear. WFLY-12649 was a similar issue, but there it was known that the version of Artemis of EAP 7.0.0 did not work properly on OpenJ9. This current issue is affecting various jobs running different OpenJDK versions. I'll send up a PR ignoring this test. The problem details: {code} java.lang.AssertionError: {"WFLYDC0074: Operation failed or was rolled back on all servers. Server failures:" => {"server-group" => {"other-server-group" => {"host" => {"slave" => {"server-one" => {"WFLYCTL0062: Composite operation failed and was rolled back. Steps that failed:" => {"Operation step-1" => { "WFLYCTL0180: Services with missing/unavailable dependencies" => ["jboss.naming.context.java.module.test.test.DefaultJMSConnectionFactory is missing [jboss.naming.context.java.jboss.DefaultJMSConnectionFactory]"], "WFLYCTL0288: One or more services were unable to start due to one or more indirect dependencies not being available." => { "Services that were unable to start:" => [ "jboss.deployment.unit.\"test.war\".CdiValidatorFactoryService", "jboss.deployment.unit.\"test.war\".WeldStartService", "jboss.deployment.unit.\"test.war\".component.\"com.sun.faces.config.ConfigureListener\".START", "jboss.deployment.unit.\"test.war\".component.\"com.sun.faces.config.ConfigureListener\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"javax.faces.webapp.FacetTag\".START", "jboss.deployment.unit.\"test.war\".component.\"javax.faces.webapp.FacetTag\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"javax.servlet.jsp.jstl.tlv.PermittedTaglibsTLV\".START", "jboss.deployment.unit.\"test.war\".component.\"javax.servlet.jsp.jstl.tlv.PermittedTaglibsTLV\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"javax.servlet.jsp.jstl.tlv.ScriptFreeTLV\".START", "jboss.deployment.unit.\"test.war\".component.\"javax.servlet.jsp.jstl.tlv.ScriptFreeTLV\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"org.jboss.weld.servlet.WeldInitialListener\".START", "jboss.deployment.unit.\"test.war\".component.\"org.jboss.weld.servlet.WeldInitialListener\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".component.\"org.jboss.weld.servlet.WeldTerminalListener\".START", "jboss.deployment.unit.\"test.war\".component.\"org.jboss.weld.servlet.WeldTerminalListener\".WeldInstantiator", "jboss.deployment.unit.\"test.war\".deploymentCompleteService", "jboss.deployment.unit.\"test.war\".jndiDependencyService", "jboss.undertow.deployment.default-server.default-host./test", "jboss.undertow.deployment.default-server.default-host./test.UndertowDeploymentInfoService" ], "Services that may be the cause:" => [ "jboss.clustering.singleton.server.default", "jboss.naming.context.java.jboss.DefaultJMSConnectionFactory" ] } }}}}}}}}} {code} In the server.log of the /host=slave/server=server-one server: {code} 2020-07-17 08:48:06,358 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=true,journalDirectory=/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/journal,bindingsDirectory=/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/bindings,largeMessagesDirectory=/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/largemessages,pagingDirectory=/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/paging) 2020-07-17 08:48:06,378 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ221012: Using AIO Journal 2020-07-17 08:48:06,478 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE 2020-07-17 08:48:06,481 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ 2020-07-17 08:48:06,509 WARN [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 42) AMQ222010: Critical IO Error, shutting down the server. file=AIOSequentialFile:/store/work/tc-work/f5da3564a57e9d74/testsuite/mixed-domain/target/domains/MixedDomain700TestSuite/slave/servers/server-one/data/activemq/journal/activemq-data-1.amq.tmp, message=Cannot open file:Invalid argument: java.io.IOException: Cannot open file:Invalid argument at org.apache.activemq.artemis.jlibaio.LibaioContext.open(Native Method) at org.apache.activemq.artemis.jlibaio.LibaioContext.openFile(LibaioContext.java:291) at org.apache.activemq.artemis.jlibaio.LibaioContext.openFile(LibaioContext.java:274) at org.apache.activemq.artemis.core.io.aio.AIOSequentialFile.open(AIOSequentialFile.java:133) at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.createFile0(JournalFilesRepository.java:590) at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.createFile(JournalFilesRepository.java:543) at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.ensureMinFiles(JournalFilesRepository.java:206) at org.apache.activemq.artemis.core.journal.impl.JournalImpl.setUpCurrentFile(JournalImpl.java:2652) at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1728) at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1126) at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1110) at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.loadMessageJournal(JournalStorageManager.java:1260) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:1701) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:1595) at org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:60) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:393) at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:381) at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:199) at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:63) at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:97) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.jboss.threads.JBossThread.run(JBossThread.java:320) {code}