[infinispan-issues] [JBoss JIRA] (ISPN-4988) TopologyAwareDistAsyncFuncTest fails with SIGSEGV exception with Azul JDK

Vitalii Chepeliuk (JIRA) issues at jboss.org
Tue Nov 18 06:41:43 EST 2014 

     [ https://issues.jboss.org/browse/ISPN-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vitalii Chepeliuk updated ISPN-4988:
------------------------------------
    Description: 
Lot of ST tests fail with
{noformat}
Test suite progress: tests succeeded: 2602, failed: 0, skipped: 0.
2014-11-16 04:38:48,750 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
2014-11-16 04:38:48,750 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
2014-11-16 04:38:48,829 WARN  [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
2014-11-16 04:38:48,945 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
2014-11-16 04:38:48,945 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
2014-11-16 04:38:49,023 WARN  [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
2014-11-16 04:38:49,249 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
2014-11-16 04:38:49,249 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
2014-11-16 04:38:49,326 WARN  [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
Signum: [11] - Exiting due to unhandled SIGSEGV exception.
0: rip=0x0000000020d9eb48 @rip=[0xffffffffffffffff] (hotspot_os_backtrace_callback+40) [gcc frame, calls gcc]
1: rip=0x00007f526459da3f @rip=[0x0000440000bffb38] (os_backtrace+31) [gcc frame, calls gcc]
2: rip=0x0000000020d99ef5 @rip=[0x0000440000bffba8] (jvm_unexpected_exception_handler+165) [gcc frame, calls gcc]
3: rip=0x00007f526459cf52 @rip=[0x0000440000bffc58] (jvm_unexpected_exception_handler_wrapper+82) [gcc frame, calls gcc]
4: rip=0x0000000020838c30 @rip=[0x0000440000bffc78] (GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+112) [gcc frame, calls gcc]
5: rip=0x0000000020876355 @rip=[0x0000440000bffcd8] (void GPGC_MarkAlgorithm::drain_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+149) [gcc frame, calls gcc]
6: rip=0x000000002087748c @rip=[0x0000440000bffda8] (void GPGC_MarkAlgorithm::drain_and_steal_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+28) [gcc frame, calls gcc]
7: rip=0x00000000209ba1ed @rip=[0x0000440000bffe68] (PGCTaskThread::run()+589) [gcc frame, calls gcc]
8: rip=0x00007f526459fc49 @rip=[0x0000440000bfff88] (alternate_stack_create+153) [gcc frame, calls gcc]
#
# An unexpected error has been detected by Java Runtime Environment:
#
#  Segmentation fault (0xb) at pc=0x20838c30, pid=25165, tid=25171
#
# Java VM: Zing 64-Bit Tiered VM (1.7.0-zing_5.10.1.0-b9-product-azlinuxM-X86_64, mixed mode)
# Problematic frame:
# C  [libjvm.so+0x438c30]  GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+0x70
#
# An error report file with more information is saved as:
# /qa/hudson_workspace/workspace/jdg-63-ispn-testsuite-rhel-azul/c6098cff/infinispan/core/hs_err_pid25165.log
2014-11-16 04:38:49,693 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
2014-11-16 04:38:49,693 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
#
# If you would like to submit a bug report, please visit:
#   http://www.azulsystems.com/support/
#

More info from jenkins jobs here 
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuite-rhel-azul/63/USE_JDK=Azul,jdk=java17_default,label=edg-perf09/console
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuite-rhel-azul/62/USE_JDK=Azul,jdk=java17_default,label=edg-perf09/console


  was:
Lot of ST tests fail with
{noformat}
2014-11-16 10:16:50,867 ERROR [UnitTestTestNGListener] (testng-DistSyncNonTxStateTransferTest) Test testRemoveIfMatchFail(org.infinispan.xsite.statetransfer.DistSyncNonTxStateTransferTest) failed.
java.lang.AssertionError: expected:<ok> but was:<Unable to pushState to 'NYC'. org.infinispan.util.concurrent.TimeoutException: Timed out after 20 minutes waiting for a response from NYC (sync, timeout=1200000)>
	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
	at org.infinispan.xsite.statetransfer.BaseStateTransferTest.startStateTransfer(BaseStateTransferTest.java:447)
	at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testStateTransferWithNoReplicatedOperation(BaseStateTransferTest.java:372)
	at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testRemoveIfMatchFail(BaseStateTransferTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Test suite progress: tests succeeded: 21, failed: 140, skipped: 0.
{noformat}
or 
{noformat}
java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from DistSyncOnePhaseTxStateTransferTest-NodeAJ-18700, see cause for remote stack trace
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:188)
	at org.infinispan.commons.util.concurrent.NotifyingFutureImpl.get(NotifyingFutureImpl.java:73)
	at org.infinispan.xsite.BackupReceiverImpl.invokeRemotelyInLocalSite(BackupReceiverImpl.java:215)
	at org.infinispan.xsite.BackupReceiverImpl.handleStateTransferControl(BackupReceiverImpl.java:89)
	at org.infinispan.xsite.BackupReceiverDelegator.handleStateTransferControl(BackupReceiverDelegator.java:37)
	at org.infinispan.xsite.statetransfer.XSiteStateTransferControlCommand.performInLocalSite(XSiteStateTransferControlCommand.java:41)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher$3.run(CommandAwareRpcDispatcher.java:250)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from DistSyncOnePhaseTxStateTransferTest-NodeAJ-18700, see cause for remote stack trace
	at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:44)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:364)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:165)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:525)
	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:290)
	at org.infinispan.remoting.rpc.RpcManagerImpl$2.call(RpcManagerImpl.java:325)
	at org.infinispan.remoting.rpc.RpcManagerImpl$2.call(RpcManagerImpl.java:321)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	... 3 more
Caused by: org.infinispan.commons.CacheException: Already receiving state from LON
	at org.infinispan.xsite.statetransfer.XSiteStateConsumerImpl.startStateTransfer(XSiteStateConsumerImpl.java:64)
	at org.infinispan.xsite.statetransfer.XSiteStateTransferControlCommand.perform(XSiteStateTransferControlCommand.java:59)
	at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:92)
	at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:208)
	at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:81)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:265)
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:209)
	at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:460)
	at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377)
	at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:250)
	at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:677)
	at org.jgroups.JChannel.up(JChannel.java:733)
	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1029)
	at org.jgroups.protocols.relay.RELAY2.up(RELAY2.java:419)
	at org.jgroups.protocols.RSVP.up(RSVP.java:201)
	at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
	at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
	at org.jgroups.protocols.FlowControl.up(FlowControl.java:383)
	at org.jgroups.protocols.tom.TOA.up(TOA.java:121)
	at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1042)
	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
	at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
	at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
	at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:619)
	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:297)
	at org.jgroups.protocols.Discovery.up(Discovery.java:277)
	at org.jgroups.protocols.TP.passMessageUp(TP.java:1572)
	at org.jgroups.protocols.TP$MyHandler.run(TP.java:1791)
	... 3 more
{noformat}
or
{noformat}
2014-11-15 22:02:52,185 ERROR [UnitTestTestNGListener] (testng-DistSyncOnePhaseTxStateTransferTest) Test testRemoveNonExisting(org.infinispan.xsite.statetransfer.DistSyncOnePhaseTxStateTransferTest) failed.
java.util.concurrent.TimeoutException: Timed out waiting for event before-start
	at org.infinispan.test.fwk.CheckPoint.awaitStrict(CheckPoint.java:43)
	at org.infinispan.test.fwk.CheckPoint.awaitStrict(CheckPoint.java:33)
	at org.infinispan.xsite.statetransfer.BaseStateTransferTest$XSiteStateProviderControl.await(BaseStateTransferTest.java:783)
	at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testConcurrentOperation(BaseStateTransferTest.java:300)
	at org.infinispan.xsite.statetransfer.BaseStateTransferTest.testRemoveNonExisting(BaseStateTransferTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
{noformat}

More info from jenkins jobs here 
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuite-rhel-azul/63/USE_JDK=Azul,jdk=java17_default,label=edg-perf09/console
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuite-rhel-azul/62/USE_JDK=Azul,jdk=java17_default,label=edg-perf09/console




> TopologyAwareDistAsyncFuncTest fails with SIGSEGV exception with Azul JDK
> -------------------------------------------------------------------------
>
>                 Key: ISPN-4988
>                 URL: https://issues.jboss.org/browse/ISPN-4988
>             Project: Infinispan
>          Issue Type: Bug
>          Components: State Transfer
>    Affects Versions: 7.0.1.Final
>            Reporter: Vitalii Chepeliuk
>            Assignee: Dan Berindei
>              Labels: testsuite_stability
>
> Lot of ST tests fail with
> {noformat}
> Test suite progress: tests succeeded: 2602, failed: 0, skipped: 0.
> 2014-11-16 04:38:48,750 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:48,750 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:48,829 WARN  [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> 2014-11-16 04:38:48,945 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:48,945 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:49,023 WARN  [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> 2014-11-16 04:38:49,249 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:49,249 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:49,326 WARN  [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> Signum: [11] - Exiting due to unhandled SIGSEGV exception.
> 0: rip=0x0000000020d9eb48 @rip=[0xffffffffffffffff] (hotspot_os_backtrace_callback+40) [gcc frame, calls gcc]
> 1: rip=0x00007f526459da3f @rip=[0x0000440000bffb38] (os_backtrace+31) [gcc frame, calls gcc]
> 2: rip=0x0000000020d99ef5 @rip=[0x0000440000bffba8] (jvm_unexpected_exception_handler+165) [gcc frame, calls gcc]
> 3: rip=0x00007f526459cf52 @rip=[0x0000440000bffc58] (jvm_unexpected_exception_handler_wrapper+82) [gcc frame, calls gcc]
> 4: rip=0x0000000020838c30 @rip=[0x0000440000bffc78] (GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+112) [gcc frame, calls gcc]
> 5: rip=0x0000000020876355 @rip=[0x0000440000bffcd8] (void GPGC_MarkAlgorithm::drain_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+149) [gcc frame, calls gcc]
> 6: rip=0x000000002087748c @rip=[0x0000440000bffda8] (void GPGC_MarkAlgorithm::drain_and_steal_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+28) [gcc frame, calls gcc]
> 7: rip=0x00000000209ba1ed @rip=[0x0000440000bffe68] (PGCTaskThread::run()+589) [gcc frame, calls gcc]
> 8: rip=0x00007f526459fc49 @rip=[0x0000440000bfff88] (alternate_stack_create+153) [gcc frame, calls gcc]
> #
> # An unexpected error has been detected by Java Runtime Environment:
> #
> #  Segmentation fault (0xb) at pc=0x20838c30, pid=25165, tid=25171
> #
> # Java VM: Zing 64-Bit Tiered VM (1.7.0-zing_5.10.1.0-b9-product-azlinuxM-X86_64, mixed mode)
> # Problematic frame:
> # C  [libjvm.so+0x438c30]  GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+0x70
> #
> # An error report file with more information is saved as:
> # /qa/hudson_workspace/workspace/jdg-63-ispn-testsuite-rhel-azul/c6098cff/infinispan/core/hs_err_pid25165.log
> 2014-11-16 04:38:49,693 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:49,693 WARN  [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> #
> # If you would like to submit a bug report, please visit:
> #   http://www.azulsystems.com/support/
> #
> More info from jenkins jobs here 
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuite-rhel-azul/63/USE_JDK=Azul,jdk=java17_default,label=edg-perf09/console
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuite-rhel-azul/62/USE_JDK=Azul,jdk=java17_default,label=edg-perf09/console



--
This message was sent by Atlassian JIRA
(v6.3.8#6338)

More information about the infinispan-issues mailing list