[JBoss JIRA] (ISPN-10238) RemoteCacheManager.stop() hangs if a client thread is waiting for a server response
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-10238?page=com.atlassian.jira.plugin... ]
Tristan Tarrant updated ISPN-10238:
-----------------------------------
Fix Version/s: 10.0.0.Final
(was: 10.0.0.CR3)
> RemoteCacheManager.stop() hangs if a client thread is waiting for a server response
> -----------------------------------------------------------------------------------
>
> Key: ISPN-10238
> URL: https://issues.jboss.org/browse/ISPN-10238
> Project: Infinispan
> Issue Type: Bug
> Components: Server, Test Suite - Server
> Affects Versions: 10.0.0.Beta3, 9.4.14.Final
> Reporter: Dan Berindei
> Priority: Major
> Labels: testsuite_stability
> Fix For: 10.0.0.Final
>
>
> One of our integration tests performs a blocking {{RemoteCache.size()}} operation on the thread where another asynchronous operation was completed (a {{HotRod-client-async-pool}} thread):
> {code:title=EvictionIT}
> CompletableFuture res = rc.putAllAsync(entries);
> res.thenRun(() -> assertEquals(3, rc.size()));
> {code}
> The test then finishes, but doesn't stop the {{RemoteCacheManager}}. When I changed the test to stop the {{RemoteCacheManager}}, the test started hanging:
> {noformat}
> "HotRod-client-async-pool-139-1" #2880 daemon prio=5 os_prio=0 cpu=434.56ms elapsed=1621.24s tid=0x00007f43a6b99800 nid=0x19c0 waiting on condition [0x00007f42ec9fd000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base(a)11.0.3/Native Method)
> - parking to wait for <0x00000000d3321350> (a java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.3/LockSupport.java:234)
> at java.util.concurrent.CompletableFuture$Signaller.block(java.base@11.0.3/CompletableFuture.java:1798)
> at java.util.concurrent.ForkJoinPool.managedBlock(java.base@11.0.3/ForkJoinPool.java:3128)
> at java.util.concurrent.CompletableFuture.timedGet(java.base@11.0.3/CompletableFuture.java:1868)
> at java.util.concurrent.CompletableFuture.get(java.base@11.0.3/CompletableFuture.java:2021)
> at org.infinispan.client.hotrod.impl.Util.await(Util.java:46)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.size(RemoteCacheImpl.java:307)
> at org.infinispan.server.test.eviction.EvictionIT.lambda$testPutAllAsyncEviction$0(EvictionIT.java:73)
> at org.infinispan.server.test.eviction.EvictionIT$$Lambda$347/0x000000010074a440.run(Unknown Source)
> at java.util.concurrent.CompletableFuture$UniRun.tryFire(java.base@11.0.3/CompletableFuture.java:783)
> at java.util.concurrent.CompletableFuture.postComplete(java.base@11.0.3/CompletableFuture.java:506)
> at java.util.concurrent.CompletableFuture.complete(java.base@11.0.3/CompletableFuture.java:2073)
> at org.infinispan.client.hotrod.impl.operations.HotRodOperation.complete(HotRodOperation.java:162)
> at org.infinispan.client.hotrod.impl.operations.PutAllOperation.acceptResponse(PutAllOperation.java:83)
> at org.infinispan.client.hotrod.impl.transport.netty.HeaderDecoder.decode(HeaderDecoder.java:144)
> at org.infinispan.client.hotrod.impl.transport.netty.HintedReplayingDecoder.callDecode(HintedReplayingDecoder.java:94)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:799)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:421)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:321)
> at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.3/ThreadPoolExecutor.java:1128)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.3/ThreadPoolExecutor.java:628)
> at java.lang.Thread.run(java.base@11.0.3/Thread.java:834)
> Locked ownable synchronizers:
> - <0x00000000ca248c30> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> "main" #1 prio=5 os_prio=0 cpu=37300.10ms elapsed=2911.99s tid=0x00007f43a4023000 nid=0x37f7 in Object.wait() [0x00007f43a9c21000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(java.base(a)11.0.3/Native Method)
> - waiting on <no object reference available>
> at java.lang.Object.wait(java.base@11.0.3/Object.java:328)
> at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:231)
> - waiting to re-lock in wait() <0x00000000ca174af8> (a io.netty.util.concurrent.DefaultPromise)
> at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:33)
> at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:32)
> at org.infinispan.client.hotrod.impl.transport.netty.ChannelFactory.destroy(ChannelFactory.java:216)
> at org.infinispan.client.hotrod.RemoteCacheManager.stop(RemoteCacheManager.java:365)
> at org.infinispan.client.hotrod.RemoteCacheManager.close(RemoteCacheManager.java:513)
> at org.infinispan.commons.junit.ClassResource.lambda$new$0(ClassResource.java:24)
> at org.infinispan.commons.junit.ClassResource$$Lambda$286/0x0000000100573040.accept(Unknown Source)
> at org.infinispan.commons.junit.ClassResource.after(ClassResource.java:41)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:50)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:167)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
> at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Locked ownable synchronizers:
> - None
> {noformat}
> {{HotRod-client-async-pool}} threads are not appropriate for doing blocking cache operations at any time, but we need to do more than just change the test:
> * We need an asynchronous {{RemoteCache.size()}} alternative
> * Currently blocking operations like {{size()}} wait for a response from the server for 1 day, they should wait for a much smaller (and configurable) timeout.
> * {{RemoteCacheManager.stop()}} should have a timeout as well, but more importantly it should cancel any pending operation.
> * We should consider running all application code on a separate thread pool.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 6 months
[JBoss JIRA] (ISPN-10236) Hotrod client error releasing channel after server cache stop
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-10236?page=com.atlassian.jira.plugin... ]
Tristan Tarrant updated ISPN-10236:
-----------------------------------
Fix Version/s: 10.0.0.Final
(was: 10.0.0.CR3)
> Hotrod client error releasing channel after server cache stop
> -------------------------------------------------------------
>
> Key: ISPN-10236
> URL: https://issues.jboss.org/browse/ISPN-10236
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 10.0.0.Beta3
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Major
> Labels: testsuite_stability
> Fix For: 10.0.0.Final
>
> Attachments: ISPN-10137_package_private_scope_20190524-1732_ServerFailureRetryTest-infinispan-client-hotrod.log.gz
>
>
> Random failure in {{ServerFailureRetryTest.testRetryCacheStopped}} caused by an assert statement in {{ChannelPool.release()}}.
> {noformat}
> 17:37:36,562 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.client.hotrod.retry.ServerFailureRetryTest.testRetryCacheStopped
> java.lang.AssertionError: Error releasing [id: 0x5d9755e6, L:/127.0.0.1:42472 ! R:127.0.0.1/127.0.0.1:44865]
> at org.infinispan.client.hotrod.impl.transport.netty.ChannelPool.release(ChannelPool.java:170) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.transport.netty.ChannelFactory.releaseChannel(ChannelFactory.java:309) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.operations.HotRodOperation.releaseChannel(HotRodOperation.java:105) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.invoke(RetryOnFailureOperation.java:80) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.transport.netty.ChannelPool.activateChannel(ChannelPool.java:217) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.transport.netty.ChannelPool.acquire(ChannelPool.java:86) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.transport.netty.ChannelFactory.fetchChannelAndInvoke(ChannelFactory.java:259) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.transport.netty.ChannelFactory.fetchChannelAndInvoke(ChannelFactory.java:297) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.operations.AbstractKeyOperation.fetchChannelAndInvoke(AbstractKeyOperation.java:41) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:61) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.putAsync(RemoteCacheImpl.java:366) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:334) ~[classes/:?]
> at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79) ~[classes/:?]
> at org.infinispan.client.hotrod.retry.ServerFailureRetryTest.testRetryCacheStopped(ServerFailureRetryTest.java:63) ~[test-classes/:?]
> {noformat}
> I investigated a bit and I couldn't find an obvious mistake in the way {{ChannelPool.created}} is incremented and decremented, but I think it would help if access to it and {{ChannelPool.active}} were centralized in a smaller number of methods.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 6 months
[JBoss JIRA] (ISPN-10192) Non-blocking cross-site requests
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-10192?page=com.atlassian.jira.plugin... ]
Tristan Tarrant updated ISPN-10192:
-----------------------------------
Fix Version/s: (was: 10.0.0.CR3)
> Non-blocking cross-site requests
> --------------------------------
>
> Key: ISPN-10192
> URL: https://issues.jboss.org/browse/ISPN-10192
> Project: Infinispan
> Issue Type: Enhancement
> Components: Cross-Site Replication
> Reporter: Pedro Ruivo
> Assignee: Pedro Ruivo
> Priority: Major
> Fix For: 10.0.0.Final
>
>
> Current implementation blocks the command in the interceptor chain waiting for it to be replicated to the remote site. This should be changed to a non-blocking approach.
> Returning an {{InvocationStage}} should solve the issue.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 6 months
[JBoss JIRA] (ISPN-10185) hotrod-client tests hang on Windows
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-10185?page=com.atlassian.jira.plugin... ]
Tristan Tarrant updated ISPN-10185:
-----------------------------------
Fix Version/s: 10.0.0.Final
(was: 10.0.0.CR3)
> hotrod-client tests hang on Windows
> -----------------------------------
>
> Key: ISPN-10185
> URL: https://issues.jboss.org/browse/ISPN-10185
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 9.4.13.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Major
> Labels: testsuite_stability
> Fix For: 10.0.0.Final
>
> Attachments: ReplFailOverRemoteIteratorTest.txt
>
>
> All the {{ForkJoin.commonPool}} in the thread dump are busy doing blocking operations:
> {noformat}
> "ForkJoinPool.commonPool-worker-1" #66 prio=0 tid=0x42 nid=NA timed_waiting
> java.lang.Thread.State: TIMED_WAITING
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x75d9efe2> (a java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1695)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3313)
> at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1775)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
> at org.infinispan.client.hotrod.impl.Util.await(Util.java:21)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:335)
> at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79)
> at org.infinispan.client.hotrod.impl.iteration.AbstractRemoteIteratorTest.lambda$populateCache$0(AbstractRemoteIteratorTest.java:34)
> {noformat}
> {{ForkJoinPool.commonPool}} is supposed to add new threads to maintain the default parallelism level (= number of cpus), but that doesn't seem to work because {{RemoteCacheManagerTest}} is trying to start a cache and it didn't get a {{commonPool}} thread:
> {noformat}
> "testng-RemoteCacheManagerTest" #61 prio=0 tid=0x3d nid=NA waiting
> java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x23a92f0a> (a java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at org.infinispan.client.hotrod.RemoteCacheManagerTest.testStartStopAsync(RemoteCacheManagerTest.java:55)
> {noformat}
> Server threads are all free. The first test to time out was {{ProtobufRemoteIteratorIndexingTest}}, but most put operations on {{ForkJoinPool.commonPool}} are issued by {{ReplFailOverRemoteIteratorTest}}:
> {noformat}
> "testng-ReplFailOverRemoteIteratorTest" #62 prio=0 tid=0x3e nid=NA waiting
> java.lang.Thread.State: WAITING
> at java.lang.Object.wait(Native Method)
> - waiting on <0x404a8e70> (a java.util.stream.ForEachOps$ForEachTask)
> at java.util.concurrent.ForkJoinTask.externalAwaitDone(ForkJoinTask.java:334)
> at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:405)
> at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
> at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
> at java.util.stream.ForEachOps$ForEachOp$OfInt.evaluateParallel(ForEachOps.java:189)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
> at java.util.stream.IntPipeline.forEach(IntPipeline.java:404)
> at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:560)
> at org.infinispan.client.hotrod.impl.iteration.AbstractRemoteIteratorTest.populateCache(AbstractRemoteIteratorTest.java:34)
> at org.infinispan.client.hotrod.impl.iteration.BaseIterationFailOverTest.testFailOver(BaseIterationFailOverTest.java:38)
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 6 months
[JBoss JIRA] (ISPN-10183) Unable to add cache configuration via server CLI
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-10183?page=com.atlassian.jira.plugin... ]
Tristan Tarrant updated ISPN-10183:
-----------------------------------
Fix Version/s: 10.0.0.Final
(was: 10.0.0.CR3)
> Unable to add cache configuration via server CLI
> ------------------------------------------------
>
> Key: ISPN-10183
> URL: https://issues.jboss.org/browse/ISPN-10183
> Project: Infinispan
> Issue Type: Bug
> Components: Server
> Affects Versions: 10.0.0.Beta3, 9.4.13.Final
> Reporter: Ryan Emerson
> Assignee: Ryan Emerson
> Priority: Major
> Fix For: 10.0.0.Final, 9.4.17.Final
>
>
> {code}
> [standalone@localhost:9990 /] cd subsystem=datagrid-infinispan/cache-container=local/configurations=CONFIGURATIONS
> [standalone@localhost:9990 configurations=CONFIGURATIONS] ./local-cache-configuration=txCache:add()
> {"outcome" => "success"}
> [standalone@localhost:9990 configurations=CONFIGURATIONS] ls local-cache-configuration=txCache
> {"WFLYCTL0062: Composite operation failed and was rolled back. Steps that failed:" => {"Operation step-2" => "WFLYCTL0216: Management resource '[
> (\"subsystem\" => \"datagrid-infinispan\"),
> (\"cache-container\" => \"local\"),
> (\"configurations\" => \"CONFIGURATIONS\"),
> (\"local-cache-configuration\" => \"txCache\"),
> (\"persistence\" => \"PERSISTENCE\")
> ]' not found"}}
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 6 months
[JBoss JIRA] (ISPN-10368) All thread pools should have a queue
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-10368?page=com.atlassian.jira.plugin... ]
Tristan Tarrant updated ISPN-10368:
-----------------------------------
Fix Version/s: 10.0.0.Final
(was: 10.0.0.CR3)
> All thread pools should have a queue
> ------------------------------------
>
> Key: ISPN-10368
> URL: https://issues.jboss.org/browse/ISPN-10368
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 10.0.0.Beta3
> Reporter: Dan Berindei
> Assignee: Will Burns
> Priority: Major
> Labels: testsuite_stability
> Fix For: 10.0.0.Final
>
>
> Random failures in {{DistSyncStoreNotSharedTest}} because the cpu executor ({{ASYNC_OPERATIONS_EXECUTOR}}) rejected a task:
> {noformat}
> java.util.concurrent.CompletionException: java.lang.AssertionError: Thread name is: persistence-thread-DistSyncStoreNotSharedTest-NodeB-p16499-t5
> at org.infinispan.util.concurrent.CompletionStages.join(CompletionStages.java:75)
> at org.infinispan.stream.impl.AbstractCacheStream.performPublisherOperation(AbstractCacheStream.java:290)
> at org.infinispan.stream.impl.DistributedCacheStream.anyMatch(DistributedCacheStream.java:328)
> at org.infinispan.cache.impl.CacheImpl.isEmpty(CacheImpl.java:502)
> at org.infinispan.cache.impl.CacheImpl.isEmpty(CacheImpl.java:498)
> at org.infinispan.cache.impl.AbstractDelegatingCache.isEmpty(AbstractDelegatingCache.java:379)
> at org.infinispan.distribution.DistSyncStoreNotSharedTest.prepareClearTest(DistSyncStoreNotSharedTest.java:348)
> at org.infinispan.distribution.DistSyncStoreNotSharedTest.testClearWithFlag(DistSyncStoreNotSharedTest.java:305)
> at org.infinispan.commons.test.TestNGLongTestsHook.run(TestNGLongTestsHook.java:24)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.lang.AssertionError: Thread name is: persistence-thread-DistSyncStoreNotSharedTest-NodeB-p16499-t5
> at org.infinispan.persistence.manager.PersistenceManagerImpl.publishEntries(PersistenceManagerImpl.java:699)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.publishEntries(PersistenceManagerImpl.java:120)
> at org.infinispan.interceptors.impl.CacheLoaderInterceptor$WrappedEntrySet.getCacheEntryPublisher(CacheLoaderInterceptor.java:726)
> at org.infinispan.interceptors.impl.CacheLoaderInterceptor$WrappedEntrySet.localPublisher(CacheLoaderInterceptor.java:713)
> at org.infinispan.reactive.publisher.impl.LocalPublisherManagerImpl.lambda$exactlyOnceSequential$7(LocalPublisherManagerImpl.java:372)
> at org.infinispan.util.rxjava.FlowableFromIntSetFunction$IteratorSubscription.slowPath(FlowableFromIntSetFunction.java:223)
> at org.infinispan.util.rxjava.FlowableFromIntSetFunction$BaseRangeSubscription.request(FlowableFromIntSetFunction.java:127)
> at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drainLoop(FlowableFlatMap.java:546)
> at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drain(FlowableFlatMap.java:366)
> at io.reactivex.internal.operators.flowable.FlowableFlatMap$InnerSubscriber.onComplete(FlowableFlatMap.java:678)
> at io.reactivex.internal.subscriptions.DeferredScalarSubscription.complete(DeferredScalarSubscription.java:119)
> at io.reactivex.processors.AsyncProcessor.onComplete(AsyncProcessor.java:201)
> at org.infinispan.reactive.RxJavaInterop.lambda$static$5(RxJavaInterop.java:120)
> at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
> at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
> at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
> at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
> at io.reactivex.internal.observers.ConsumerSingleObserver.onSuccess(ConsumerSingleObserver.java:62)
> at io.reactivex.internal.operators.flowable.FlowableAnySingle$AnySubscriber.onComplete(FlowableAnySingle.java:109)
> at io.reactivex.internal.operators.flowable.FlowableDoOnEach$DoOnEachSubscriber.onComplete(FlowableDoOnEach.java:135)
> at io.reactivex.internal.operators.flowable.FlowableConcatArray$ConcatArraySubscriber.onComplete(FlowableConcatArray.java:112)
> at io.reactivex.internal.subscribers.BasicFuseableSubscriber.onComplete(BasicFuseableSubscriber.java:120)
> at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.checkTerminated(FlowableObserveOn.java:215)
> at io.reactivex.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:399)
> at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:176)
> at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:260)
> at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:225)
> at org.infinispan.commons.util.concurrent.CallerRunsRejectOnShutdownPolicy.rejectedExecution(CallerRunsRejectOnShutdownPolicy.java:19)
> at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825)
> at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355)
> at org.infinispan.executors.LazyInitializingExecutorService.execute(LazyInitializingExecutorService.java:138)
> at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.schedule(ExecutorScheduler.java:143)
> at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.trySchedule(FlowableObserveOn.java:166)
> at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.onComplete(FlowableObserveOn.java:135)
> at io.reactivex.internal.operators.flowable.FlowableSubscribeOn$SubscribeOnSubscriber.onComplete(FlowableSubscribeOn.java:108)
> at io.reactivex.internal.operators.flowable.FlowableUsing$UsingSubscriber.onComplete(FlowableUsing.java:148)
> at io.reactivex.internal.subscribers.BasicFuseableSubscriber.onComplete(BasicFuseableSubscriber.java:120)
> at io.reactivex.internal.subscribers.BasicFuseableConditionalSubscriber.onComplete(BasicFuseableConditionalSubscriber.java:119)
> at io.reactivex.internal.subscribers.BasicFuseableConditionalSubscriber.onComplete(BasicFuseableConditionalSubscriber.java:119)
> at io.reactivex.internal.subscriptions.EmptySubscription.complete(EmptySubscription.java:69)
> {noformat}
> https://ci.infinispan.org/job/Infinispan/job/master/1269/
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 6 months
[JBoss JIRA] (ISPN-10369) InvalidatedNearCacheTest random failures
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-10369?page=com.atlassian.jira.plugin... ]
Tristan Tarrant updated ISPN-10369:
-----------------------------------
Fix Version/s: 10.0.0.Final
(was: 10.0.0.CR3)
> InvalidatedNearCacheTest random failures
> ----------------------------------------
>
> Key: ISPN-10369
> URL: https://issues.jboss.org/browse/ISPN-10369
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 10.0.0.Beta3
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Major
> Labels: testsuite_stability
> Fix For: 10.0.0.Final
>
>
> {{InvalidatedNearCacheTest.testGetNearCacheAfterConnect}} has 2 clients, performing some operations and waiting for the corresponding near cache events on client 1 and then resetting the events on client 2.
> This is not correct, because client 2 may not have received the events yet, in which case {{resetEvents()}} doesn't do anything.
> {noformat}
> 18:48:47,813 INFO (testng-InvalidatedNearCacheTest:[]) [TestSuiteProgress] Test succeeded: org.infinispan.client.hotrod.near.InvalidatedNearCacheTest.testGetNearCacheAfterConnect
> 18:48:47,814 INFO (testng-InvalidatedNearCacheTest:[]) [TestSuiteProgress] Test starting: org.infinispan.client.hotrod.near.InvalidatedNearCacheTest.testGetUpdatesNearCache
> 18:48:47,821 ERROR (testng-InvalidatedNearCacheTest:[]) [TestSuiteProgress] Test failed: org.infinispan.client.hotrod.near.InvalidatedNearCacheTest.testGetUpdatesNearCache
> 18:48:47,821 ERROR (testng-InvalidatedNearCacheTest:[]) [TestSuiteProgress] Test failed: org.infinispan.client.hotrod.near.InvalidatedNearCacheTest.testGetUpdatesNearCache
> java.lang.AssertionError: [org.infinispan.client.hotrod.near.MockNearCacheService$MockRemoveEvent@514f3f0c, org.infinispan.client.hotrod.near.MockNearCacheService$MockRemoveEvent@76bf7455] expected:<0> but was:<2>
> at org.testng.AssertJUnit.fail(AssertJUnit.java:59) ~[testng-6.14.3.jar:?]
> at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364) ~[testng-6.14.3.jar:?]
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80) ~[testng-6.14.3.jar:?]
> at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:245) ~[testng-6.14.3.jar:?]
> at org.infinispan.client.hotrod.near.AssertsNearCache.expectNoNearEvents(AssertsNearCache.java:123) ~[test-classes/:?]
> at org.infinispan.client.hotrod.near.InvalidatedNearCacheTest.testGetUpdatesNearCache(InvalidatedNearCacheTest.java:153) ~[test-classes/:?]
> {noformat}
> https://ci.infinispan.org/job/Infinispan/job/master/1240/
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 6 months