[JBoss JIRA] (ISPN-5493) SiteProviderTopologyChangeTest.testXSiteSTDuringLeave random failures
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-5493?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-5493:
------------------------------
Fix Version/s: 8.1.0.Final
(was: 8.0.0.Final)
> SiteProviderTopologyChangeTest.testXSiteSTDuringLeave random failures
> ---------------------------------------------------------------------
>
> Key: ISPN-5493
> URL: https://issues.jboss.org/browse/ISPN-5493
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Cross-Site Replication, Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Assignee: Pedro Ruivo
> Priority: Blocker
> Labels: testsuite_failure
> Fix For: 8.1.0.Final
>
> Attachments: SiteProviderTopologyChangeTest.log.gz
>
>
> Looks like the node is killed before the {{XSiteStateTransferControlCommand(START_SEND)}} command was replicated to all the nodes in the source cluster, and the {{SuspectException}} stops the state push:
> {noformat}
> 23:33:22,834 DEBUG (testng-SiteProviderTopologyChangeTest:) [XSiteAdminOperations] Unable to pushState to 'NYC'.java.lang.Exception: java.util.concurrent.ExecutionException: org.infinispan.remoting.transport.jgroups.SuspectException: Suspected member: SiteProviderTopologyChangeTest-NodeBG-25313
> at org.infinispan.xsite.statetransfer.XSiteStateTransferManagerImpl.startPushState(XSiteStateTransferManagerImpl.java:151)
> at org.infinispan.xsite.XSiteAdminOperations.pushState(XSiteAdminOperations.java:238)
> at org.infinispan.xsite.statetransfer.failures.AbstractTopologyChangeTest.startStateTransfer(AbstractTopologyChangeTest.java:144)
> at org.infinispan.xsite.statetransfer.failures.SiteProviderTopologyChangeTest.doXSiteStateTransferDuringTopologyChange(SiteProviderTopologyChangeTest.java:241)
> at org.infinispan.xsite.statetransfer.failures.SiteProviderTopologyChangeTest.testXSiteSTDuringLeave(SiteProviderTopologyChangeTest.java:78)
> Caused by: java.util.concurrent.ExecutionException: org.infinispan.remoting.transport.jgroups.SuspectException: Suspected member: SiteProviderTopologyChangeTest-NodeBG-25313
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.infinispan.commons.util.concurrent.NotifyingFutureImpl.get(NotifyingFutureImpl.java:77)
> at org.infinispan.xsite.statetransfer.XSiteStateTransferManagerImpl.invokeRemotelyInLocalSite(XSiteStateTransferManagerImpl.java:376)
> at org.infinispan.xsite.statetransfer.XSiteStateTransferManagerImpl.controlStateTransferOnLocalSite(XSiteStateTransferManagerImpl.java:335)
> at org.infinispan.xsite.statetransfer.XSiteStateTransferManagerImpl.startPushState(XSiteStateTransferManagerImpl.java:142)
> ... 24 more
> Caused by: org.infinispan.remoting.transport.jgroups.SuspectException: Suspected member: SiteProviderTopologyChangeTest-NodeBG-25313
> at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:74)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:586)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:287)
> at org.infinispan.remoting.rpc.RpcManagerImpl$2.call(RpcManagerImpl.java:382)
> at org.infinispan.remoting.rpc.RpcManagerImpl$2.call(RpcManagerImpl.java:378)
> ... 4 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 8 months
[JBoss JIRA] (ISPN-5495) ConcurrentStartTest.testConcurrentStart random failures
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-5495?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-5495:
------------------------------
Fix Version/s: 8.1.0.Final
(was: 8.0.0.Final)
> ConcurrentStartTest.testConcurrentStart random failures
> -------------------------------------------------------
>
> Key: ISPN-5495
> URL: https://issues.jboss.org/browse/ISPN-5495
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.1.0.Final
>
>
> {noformat}
> org.testng.internal.thread.ThreadTimeoutException: Method org.testng.internal.TestNGMethod.testConcurrentStart() didn't finish within the time-out 60000
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
> at org.infinispan.test.TestingUtil.waitForRehashToComplete(TestingUtil.java:253)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 8 months
[JBoss JIRA] (ISPN-5498) ClientClusterFailoverEventsTest.testEventReplayWithAndWithoutStateAfterFailover random failures
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-5498?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-5498:
------------------------------
Fix Version/s: 8.1.0.Final
(was: 8.0.0.Final)
> ClientClusterFailoverEventsTest.testEventReplayWithAndWithoutStateAfterFailover random failures
> -----------------------------------------------------------------------------------------------
>
> Key: ISPN-5498
> URL: https://issues.jboss.org/browse/ISPN-5498
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.1.0.Final
>
>
> It appears the cluster event notifier wants to send some events to a node that no longer exists, and the exception is propagated all the way to the HR client:
> {noformat}
> 12:04:03,088 ERROR (HotRodServerWorker-78-1:) [InvocationContextInterceptor] ISPN000136: Execution errorjava.lang.IllegalArgumentException: Target node ClientClusterFailoverEventsTest-NodeA-33510 is not a cluster member, members are [ClientClusterFailoverEventsTest-NodeB-14182]
> at org.infinispan.distexec.DefaultExecutorService.submit(DefaultExecutorService.java:414)
> at org.infinispan.distexec.DefaultExecutorService.submit(DefaultExecutorService.java:403)
> at org.infinispan.distexec.DistributedExecutionCompletionService.submit(DistributedExecutionCompletionService.java:178)
> at org.infinispan.notifications.cachelistener.cluster.impl.BatchingClusterEventManagerImpl$UnicastEventContext.sendToTargets(BatchingClusterEventManagerImpl.java:97)
> at org.infinispan.notifications.cachelistener.cluster.impl.BatchingClusterEventManagerImpl.sendEvents(BatchingClusterEventManagerImpl.java:50)
> at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryCreated(CacheNotifierImpl.java:292)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.notifyCommitEntry(ClusteringDependentLogic.java:138)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$DistributionLogic.commitSingleEntry(ClusteringDependentLogic.java:493)
> at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:108)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:367)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:545)
> at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:344)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:418)
> at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:449)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:195)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:88)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:55)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:324)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:256)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:115)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1617)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1097)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1089)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:522)
> at org.infinispan.server.hotrod.CacheDecodeContext.put(CacheDecodeContext.scala:216)
> at org.infinispan.server.hotrod.HotRodDecoder.org$infinispan$server$hotrod$HotRodDecoder$$decodeValue(HotRodDecoder.scala:132)
> at org.infinispan.server.hotrod.HotRodDecoder$$anonfun$decode$1.apply$mcV$sp(HotRodDecoder.scala:50)
> at org.infinispan.server.hotrod.HotRodDecoder.wrapSecurity(HotRodDecoder.scala:208)
> at org.infinispan.server.hotrod.HotRodDecoder.decode(HotRodDecoder.scala:45)
> org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[922] returned server error (status=0x85): java.lang.IllegalArgumentException: Target node ClientClusterFailoverEventsTest-NodeA-33510 is not a cluster member, members are [ClientClusterFailoverEventsTest-NodeB-14182]
> at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:336)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:126)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:112)
> at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56)
> at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:57)
> at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:31)
> at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:20)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:52)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:247)
> at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79)
> at org.infinispan.client.hotrod.event.ClientClusterFailoverEventsTest.testEventReplayWithAndWithoutStateAfterFailover(ClientClusterFailoverEventsTest.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 8 months
[JBoss JIRA] (ISPN-5481) ConfigurationOverrideTest random failures
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-5481?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-5481:
------------------------------
Fix Version/s: 8.1.0.Final
(was: 8.0.0.Final)
> ConfigurationOverrideTest random failures
> -----------------------------------------
>
> Key: ISPN-5481
> URL: https://issues.jboss.org/browse/ISPN-5481
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Priority: Critical
> Fix For: 8.1.0.Final
>
>
> {{ConfigurationOverrideTest}} uses the default global configuration, and it fails when another test has already registered a cache manager MBean in JMX with the same name:
> {noformat}
> org.infinispan.jmx.JmxDomainConflictException: ISPN000034: There's already a JMX MBean instance type=CacheManager,name="DefaultCacheManager" already registered under 'org.infinispan' JMX domain. If you want to allow multiple instances configured with same JMX domain enable 'allowDuplicateDomains' attribute in 'globalJmxStatistics' config element
> at org.infinispan.jmx.JmxUtil.buildJmxDomain(JmxUtil.java:51)
> at org.infinispan.jmx.CacheManagerJmxRegistration.updateDomain(CacheManagerJmxRegistration.java:79)
> at org.infinispan.jmx.CacheManagerJmxRegistration.buildRegistrar(CacheManagerJmxRegistration.java:73)
> at org.infinispan.jmx.AbstractJmxRegistration.registerMBeans(AbstractJmxRegistration.java:37)
> at org.infinispan.jmx.CacheManagerJmxRegistration.start(CacheManagerJmxRegistration.java:41)
> at org.infinispan.manager.DefaultCacheManager.start(DefaultCacheManager.java:625)
> at org.infinispan.manager.DefaultCacheManager.<init>(DefaultCacheManager.java:218)
> at org.infinispan.manager.DefaultCacheManager.<init>(DefaultCacheManager.java:199)
> at org.infinispan.configuration.ConfigurationOverrideTest.testOverrideWithStore(ConfigurationOverrideTest.java:80)
> {noformat}
> We should verify the other tests as well, to make sure they all use the {{PerThreadMBeanServerLookup}} and/or a unique JMX domain.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 8 months
[JBoss JIRA] (ISPN-5499) SizeTest.testPersistentDistributedCacheSize random failures
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-5499?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-5499:
------------------------------
Fix Version/s: 8.1.0.Final
(was: 8.0.0.Final)
> SizeTest.testPersistentDistributedCacheSize random failures
> -----------------------------------------------------------
>
> Key: ISPN-5499
> URL: https://issues.jboss.org/browse/ISPN-5499
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.1.0.Final
>
>
> {noformat}
> 16:04:28,678 ERROR (testng-SizeTest:) [UnitTestTestNGListener] Test testPersistentDistributedCacheSize(org.infinispan.client.hotrod.SizeTest) failed.
> java.lang.AssertionError: expected:<20> but was:<38>
> at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
> at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364)
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80)
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:245)
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:252)
> at org.infinispan.client.hotrod.SizeTest.testPersistentDistributedCacheSize(SizeTest.java:59)
> {noformat}
> I have been able to make the test fail reliably by replacing the assertion on line 57 with this:
> {code}
> for (int i = 0; i < SIZE; i++) {
> assertEquals(SIZE, clients.get(0).getCache(cacheName).size());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 8 months
[JBoss JIRA] (ISPN-5476) Cross-site tests should run in parallel
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-5476?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-5476:
------------------------------
Fix Version/s: 8.1.0.Final
(was: 8.0.0.Final)
> Cross-site tests should run in parallel
> ---------------------------------------
>
> Key: ISPN-5476
> URL: https://issues.jboss.org/browse/ISPN-5476
> Project: Infinispan
> Issue Type: Task
> Components: Core, Cross-Site Replication, Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Fix For: 8.1.0.Final
>
>
> Currently the cross-site tests have to run in a single thread, and that means they're much slower than the regular core tests.
> It also means they need to run with a separate Maven profile, and that (combined with their duration) makes it very unlikely for devs to run the xsite tests on their machines.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 8 months
[JBoss JIRA] (ISPN-5470) Remote-executor threads should not block to acquire locks
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-5470?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-5470:
------------------------------
Fix Version/s: 8.1.0.Final
(was: 8.0.0.Final)
> Remote-executor threads should not block to acquire locks
> ---------------------------------------------------------
>
> Key: ISPN-5470
> URL: https://issues.jboss.org/browse/ISPN-5470
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Fix For: 8.1.0.Final
>
>
> Currently, enabling the queue on the remote-executor thread pool can cause deadlocks, because a CommitCommand/1PCPrepareCommand could end up in the queue while a remote-executor thread is busy waiting to acquire the same lock that this commit would release.
> If trying to acquire a lock would free the thread until the key has been acquired, we could enable the queue on the remote-executor/OOB thread pools, and we would need a lot less threads for the same load.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 8 months
[JBoss JIRA] (ISPN-5475) Narayana should be configured to use a volatile store by default
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-5475?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-5475:
------------------------------
Fix Version/s: 8.1.0.Final
(was: 8.0.0.Final)
> Narayana should be configured to use a volatile store by default
> ----------------------------------------------------------------
>
> Key: ISPN-5475
> URL: https://issues.jboss.org/browse/ISPN-5475
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 8.1.0.Final
>
>
> The {{jbossts-properties.xml}} configuration file in the core module configures a file store by default, and tests have to call {{TestCacheManagerFactory.markAsTransactional()}} (or one of the methods that calls it) to configure a volatile store instead.
> Furthermore, the {{jbossts-properties.xml}} file is explicitly filtered out of the core tests jar, so other modules can't use it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
8 years, 8 months