[infinispan-issues] [JBoss JIRA] (ISPN-4988) Infinispan state transfer tests fail with Azul JDK

Infinispan state transfer tests fail with Azul JDK -------------------------------------------------- Key: ISPN-4988 URL: https://issues.jboss.org/browse/ISPN-4988 Project: Infinispan Issue Type: Bug Components: State Transfer Affects Versions: 7.0.1.Final Reporter: Vitalii Chepeliuk Labels: testsuite_stability Lot of ST tests fail with {noformat} 2014-11-16 10:16:50,867 ERROR [UnitTestTestNGListener] (testng-DistSyncNonTxStateTransferTest) Test testRemoveIfMatchFail(org.infinispan.xsite.statetransfer.DistSyncNonTxStateTransferTest) failed. java.lang.AssertionError: expected:<ok> but was:<Unable to pushState to 'NYC'. org.infinispan.util.concurrent.TimeoutException: Timed out after 20 minutes waiting for a response from NYC (sync, timeout=1200000)> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101) at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108) at org.infinispan.xsite.statetransfer.BaseStateTransferTest.startStateTransfer(BaseStateTransferTest.java:447) at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testStateTransferWithNoReplicatedOperation(BaseStateTransferTest.java:372) at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testRemoveIfMatchFail(BaseStateTransferTest.java:167) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80) at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at org.testng.TestRunner.privateRun(TestRunner.java:767) at org.testng.TestRunner.run(TestRunner.java:617) at org.testng.SuiteRunner.runTest(SuiteRunner.java:334) at org.testng.SuiteRunner.access$000(SuiteRunner.java:37) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368) at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Test suite progress: tests succeeded: 21, failed: 140, skipped: 0. {noformat} or {noformat} java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from DistSyncOnePhaseTxStateTransferTest-NodeAJ-18700, see cause for remote stack trace at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at org.infinispan.commons.util.concurrent.NotifyingFutureImpl.get(NotifyingFutureImpl.java:73) at org.infinispan.xsite.BackupReceiverImpl.invokeRemotelyInLocalSite(BackupReceiverImpl.java:215) at org.infinispan.xsite.BackupReceiverImpl.handleStateTransferControl(BackupReceiverImpl.java:89) at org.infinispan.xsite.BackupReceiverDelegator.handleStateTransferControl(BackupReceiverDelegator.java:37) at org.infinispan.xsite.statetransfer.XSiteStateTransferControlCommand.performInLocalSite(XSiteStateTransferControlCommand.java:41) at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher$3.run(CommandAwareRpcDispatcher.java:250) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from DistSyncOnePhaseTxStateTransferTest-NodeAJ-18700, see cause for remote stack trace at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:44) at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:364) at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:165) at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:525) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:290) at org.infinispan.remoting.rpc.RpcManagerImpl$2.call(RpcManagerImpl.java:325) at org.infinispan.remoting.rpc.RpcManagerImpl$2.call(RpcManagerImpl.java:321) at java.util.concurrent.FutureTask.run(FutureTask.java:262) ... 3 more Caused by: org.infinispan.commons.CacheException: Already receiving state from LON at org.infinispan.xsite.statetransfer.XSiteStateConsumerImpl.startStateTransfer(XSiteStateConsumerImpl.java:64) at org.infinispan.xsite.statetransfer.XSiteStateTransferControlCommand.perform(XSiteStateTransferControlCommand.java:59) at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:92) at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:208) at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:81) at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:265) at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:209) at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:460) at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377) at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:250) at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:677) at org.jgroups.JChannel.up(JChannel.java:733) at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1029) at org.jgroups.protocols.relay.RELAY2.up(RELAY2.java:419) at org.jgroups.protocols.RSVP.up(RSVP.java:201) at org.jgroups.protocols.FRAG2.up(FRAG2.java:165) at org.jgroups.protocols.FlowControl.up(FlowControl.java:394) at org.jgroups.protocols.FlowControl.up(FlowControl.java:383) at org.jgroups.protocols.tom.TOA.up(TOA.java:121) at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1042) at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234) at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064) at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779) at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426) at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:619) at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:297) at org.jgroups.protocols.Discovery.up(Discovery.java:277) at org.jgroups.protocols.TP.passMessageUp(TP.java:1572) at org.jgroups.protocols.TP$MyHandler.run(TP.java:1791) ... 3 more {noformat} or {noformat} 2014-11-15 22:02:52,185 ERROR [UnitTestTestNGListener] (testng-DistSyncOnePhaseTxStateTransferTest) Test testRemoveNonExisting(org.infinispan.xsite.statetransfer.DistSyncOnePhaseTxStateTransferTest) failed. java.util.concurrent.TimeoutException: Timed out waiting for event before-start at org.infinispan.test.fwk.CheckPoint.awaitStrict(CheckPoint.java:43) at org.infinispan.test.fwk.CheckPoint.awaitStrict(CheckPoint.java:33) at org.infinispan.xsite.statetransfer.BaseStateTransferTest$XSiteStateProviderControl.await(BaseStateTransferTest.java:783) at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testConcurrentOperation(BaseStateTransferTest.java:300) at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testRemoveNonExisting(BaseStateTransferTest.java:179) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80) at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at org.testng.TestRunner.privateRun(TestRunner.java:767) at org.testng.TestRunner.run(TestRunner.java:617) at org.testng.SuiteRunner.runTest(SuiteRunner.java:334) at org.testng.SuiteRunner.access$000(SuiteRunner.java:37) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368) at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {noformat} More info from jenkins jobs here https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuit... https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuit...