[JBoss JIRA] (ISPN-7977) Clean up NullPointerExceptions in the core test logs
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-7977?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-7977:
-------------------------------
Sprint: Sprint 9.3.0.Beta1
> Clean up NullPointerExceptions in the core test logs
> ----------------------------------------------------
>
> Key: ISPN-7977
> URL: https://issues.jboss.org/browse/ISPN-7977
> Project: Infinispan
> Issue Type: Task
> Components: Core, Test Suite - Core
> Affects Versions: 9.1.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Attachments: ClusterTopologyManagerImpl_NullPointerException.txt
>
>
> Even benign {{NullPointerException}} s are a bad smell, so tests should never trigger them. But they seem pretty common in the test logs nowadays. Some examples from the core test suite:
> {noformat}
> 23:30:26,503 ERROR (stateTransferExecutor-thread-SingleJoinTest[DIST_SYNC, tx=true]-NodeC-p583-t3:[StateRequest-dist]) [RpcManagerImpl] ISPN000073: Unexpected error while replicating
> java.lang.NullPointerException: null
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotelyAsync(JGroupsTransport.java:650) ~[classes/:?]
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotelyAsync(RpcManagerImpl.java:127) ~[classes/:?]
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:168) ~[classes/:?]
> at org.infinispan.statetransfer.InboundTransferTask.requestSegments(InboundTransferTask.java:132) ~[classes/:?]
> at org.infinispan.statetransfer.StateConsumerImpl.lambda$addTransfer$7(StateConsumerImpl.java:1055) ~[classes/:?]
> at org.infinispan.executors.LimitedExecutor.lambda$executeAsync$1(LimitedExecutor.java:101) ~[classes/:?]
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144) [classes/:?]
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33) [classes/:?]
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174) [classes/:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> {noformat}
> {noformat}
> 23:37:52,617 WARN (testng-AsynchronousInvocationTest:[]) [TrianglePerCacheInboundInvocationHandler] ISPN000071: Caught exception when handling command StreamRequestCommand{cacheName='___defaultcache'}
> java.lang.NullPointerException: null
> at org.infinispan.stream.impl.StreamRequestCommand.invokeAsync(StreamRequestCommand.java:89) ~[classes/:?]
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokeCommand(BasePerCacheInboundInvocationHandler.java:101) ~[classes/:?]
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.runSync(BaseBlockingRunnable.java:54) [classes/:?]
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:38) [classes/:?]
> at org.infinispan.remoting.inboundhandler.TrianglePerCacheInboundInvocationHandler.handleDefaultCommand(TrianglePerCacheInboundInvocationHandler.java:183) [classes/:?]
> at org.infinispan.remoting.inboundhandler.TrianglePerCacheInboundInvocationHandler.handle(TrianglePerCacheInboundInvocationHandler.java:112) [classes/:?]
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleCacheRpcCommand(GlobalInboundInvocationHandler.java:120) [classes/:?]
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleFromCluster(GlobalInboundInvocationHandler.java:79) [classes/:?]
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:175) [classes/:?]
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:149) [classes/:?]
> at org.infinispan.remoting.AsynchronousInvocationTest.assertDispatchForCommand(AsynchronousInvocationTest.java:176) [test-classes/:?]
> at org.infinispan.remoting.AsynchronousInvocationTest.testCacheRpcCommands(AsynchronousInvocationTest.java:140) [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) [testng-6.8.8.jar:?]
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) [testng-6.8.8.jar:?]
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) [testng-6.8.8.jar:?]
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) [testng-6.8.8.jar:?]
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) [testng-6.8.8.jar:?]
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) [testng-6.8.8.jar:?]
> at org.testng.TestRunner.privateRun(TestRunner.java:767) [testng-6.8.8.jar:?]
> at org.testng.TestRunner.run(TestRunner.java:617) [testng-6.8.8.jar:?]
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:348) [testng-6.8.8.jar:?]
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:38) [testng-6.8.8.jar:?]
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382) [testng-6.8.8.jar:?]
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64) [testng-6.8.8.jar:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> {noformat}
> {noformat}
> 23:37:54,821 TRACE (ForkThread-2,ThreadLocalLeakTest:[]) [InvocationContextInterceptor] Exception while executing code, but we're shutting down so failing silently.
> org.infinispan.persistence.spi.PersistenceException: java.lang.NullPointerException
> at org.infinispan.persistence.file.SingleFileStore.write(SingleFileStore.java:365) ~[classes/:?]
> at org.infinispan.persistence.manager.PersistenceManagerImpl.lambda$writeToAllNonTxStores$9(PersistenceManagerImpl.java:518) ~[classes/:?]
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:1.8.0_121]
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_121]
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_121]
> at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_121]
> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) ~[?:1.8.0_121]
> at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_121]
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) ~[?:1.8.0_121]
> at org.infinispan.persistence.manager.PersistenceManagerImpl.writeToAllNonTxStores(PersistenceManagerImpl.java:518) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.storeEntry(CacheWriterInterceptor.java:452) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.lambda$visitPutKeyValueCommand$1(CacheWriterInterceptor.java:187) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:109) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.visitPutKeyValueCommand(CacheWriterInterceptor.java:179) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheLoaderInterceptor.visitDataCommand(CacheLoaderInterceptor.java:208) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:135) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:102) ~[classes/:?]
> at org.infinispan.interceptors.impl.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:689) ~[classes/:?]
> at org.infinispan.interceptors.impl.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:310) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndFinally(BaseAsyncInterceptor.java:154) ~[classes/:?]
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:136) ~[classes/:?]
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:38) ~[classes/:?]
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:86) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:197) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:162) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) [classes/:?]
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:97) [classes/:?]
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) [classes/:?]
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1606) [classes/:?]
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1271) [classes/:?]
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1720) [classes/:?]
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:276) [classes/:?]
> at org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:358) [classes/:?]
> at org.infinispan.cache.impl.EncoderCache.put(EncoderCache.java:652) [classes/:?]
> at org.infinispan.util.ThreadLocalLeakTest$2.run(ThreadLocalLeakTest.java:115) [test-classes/:?]
> at org.infinispan.test.AbstractInfinispanTest$RunnableWrapper.run(AbstractInfinispanTest.java:510) [test-classes/:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> Caused by: java.lang.NullPointerException
> at org.infinispan.persistence.file.SingleFileStore.allocate(SingleFileStore.java:232) ~[classes/:?]
> at org.infinispan.persistence.file.SingleFileStore.write(SingleFileStore.java:332) ~[classes/:?]
> ... 46 more
> {noformat}
> {noformat}
> 23:38:21,145 ERROR (transport-thread-OptimisticTxFailureAfterLockingTest-NodeD-p33826-t2:[Topology-___defaultcache]) [LocalTopologyManagerImpl] ISPN000452: Failed to update topology for cache ___defaultcache
> java.lang.NullPointerException: null
> at org.infinispan.topology.PersistentUUIDManagerImpl.mapAddresses(PersistentUUIDManagerImpl.java:70) ~[classes/:?]
> at org.infinispan.topology.LocalTopologyManagerImpl.doHandleTopologyUpdate(LocalTopologyManagerImpl.java:348) ~[classes/:?]
> at org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleTopologyUpdate$1(LocalTopologyManagerImpl.java:266) ~[classes/:?]
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144) [classes/:?]
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33) [classes/:?]
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174) [classes/:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> {noformat}
> {noformat}
> 23:39:19,342 ERROR (transport-thread-ConcurrentStartChanelLookupTest-NodeA-p40485-t3:[Merge-1]) [ClusterCacheStatus] ISPN000228: Failed to recover cache repl state after the current node became the coordinator
> java.lang.NullPointerException: null
> at org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy.lambda$static$0(PreferAvailabilityStrategy.java:34) ~[classes/:?]
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) ~[?:1.8.0_121]
> at java.util.TimSort.sort(TimSort.java:220) ~[?:1.8.0_121]
> at java.util.Arrays.sort(Arrays.java:1512) ~[?:1.8.0_121]
> at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:348) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_121]
> at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_121]
> at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_121]
> at org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy.onPartitionMerge(PreferAvailabilityStrategy.java:122) ~[classes/:?]
> at org.infinispan.topology.ClusterCacheStatus.doMergePartitions(ClusterCacheStatus.java:570) ~[classes/:?]
> at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$recoverClusterStatus$4(ClusterTopologyManagerImpl.java:499) ~[classes/:?]
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144) [classes/:?]
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33) [classes/:?]
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174) [classes/:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (ISPN-9008) RpcManager.invokeCommandOnAll ignores cache member that are not in the cluster view
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-9008?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-9008:
-------------------------------
Sprint: Sprint 9.3.0.Beta1
> RpcManager.invokeCommandOnAll ignores cache member that are not in the cluster view
> -----------------------------------------------------------------------------------
>
> Key: ISPN-9008
> URL: https://issues.jboss.org/browse/ISPN-9008
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.3.0.Beta1
>
>
> {{RpcManager.invokeCommandOnAll}} broadcasts the command to all the members of the JGroups cluster view, and the {{ResponseCollector}} is not notified if any members of the cache are not in the cluster view.
> This is a problem in replicated caches with partition handling enabled, because it means a write can succeed in a minority partition in the time interval between {{JGroupsTransport}} seeing the minority cluster view and {{DistributionManagerImpl}} installing the {{DEGRADED_MODE}} cache topology.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (ISPN-9062) JGroupsTransport should only send messages to nodes in the cluster view
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-9062?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-9062:
-------------------------------
Sprint: Sprint 9.3.0.Beta1
> JGroupsTransport should only send messages to nodes in the cluster view
> -----------------------------------------------------------------------
>
> Key: ISPN-9062
> URL: https://issues.jboss.org/browse/ISPN-9062
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.2.2.Final, 9.3.0.Alpha1
>
>
> {{JGroupsTransport}} only waits for responses from nodes in the JGroups cluster view, but it still sends messages to all the nodes specified as a target. The idea was to optimize the common case by avoiding a {{HashSet.contains()}} call.
> However, when a node is not in the view, messages to it still pass through the entire JGroups stack, and UNICAST3 keeps those messages in a send table for a long time ({{UNICAST3.conn_expiry_timeout}}, changed with ISPN-9038 from {{0}} (unlimited) to 2 minutes (JGroups default)). Having a potentially unlimited number of messages of non-members, each with its own send table, makes it much harder to estimate memory usage.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (ISPN-8962) PreferAvailabilityStrategy: Rely less on the stable topology
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8962?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8962:
-------------------------------
Sprint: Sprint 9.3.0.Beta1
> PreferAvailabilityStrategy: Rely less on the stable topology
> ------------------------------------------------------------
>
> Key: ISPN-8962
> URL: https://issues.jboss.org/browse/ISPN-8962
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.0.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.2.2.Final, 9.3.0.Beta1
>
>
> {{PreferAvailabilityStrategy}} checks the size of the stable topology, and only considers cache topologies that are derived from the biggest topology (in size) when picking a post-merge topology.
> Unfortunately, in some situations this algorithm fails pretty badly. If a node has a very long GC pause, when it comes back it will report the old topology *and* the old stable topology. If the rest of the cluster rebalanced, it now has both a smaller current topology and a smaller stable topology.
> Furthermore, the stable topology is updated asynchronously, independent from the current topology. So even if there's a split and the minority partition installs a current topology with fewer members, it may take some time for its stable topology to be updated with fewer members. In fact, it appears that when a rebalance is not needed (e.g. because the partition has a single node), the stable topology is never updated!
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (ISPN-9087) Timeout during put operation when a node is blocked
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-9087?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-9087:
-------------------------------
Sprint: Sprint 9.3.0.Beta1
> Timeout during put operation when a node is blocked
> ---------------------------------------------------
>
> Key: ISPN-9087
> URL: https://issues.jboss.org/browse/ISPN-9087
> Project: Infinispan
> Issue Type: Bug
> Reporter: Diego Lovison
> Assignee: Dan Berindei
>
> {noformat}
> 2018-04-17 13:30:02.782 ERROR 14932 --- [timeout-thread--p3-t1] o.i.i.impl.InvocationContextInterceptor : ISPN000136: Error executing command PutKeyValueCommand, writing keys [5db796a3-3f65-468a-b86a-6d5ef8b4b330]
>
> org.infinispan.util.concurrent.TimeoutException: ISPN000427: Timeout after 15 seconds waiting for acks. Id=100000
> at org.infinispan.util.concurrent.CommandAckCollector.createTimeoutException(CommandAckCollector.java:188) ~[infinispan-embedded-8.5.0.Final-redhat-6.jar:8.5.0.Final-redhat-6]
> at org.infinispan.util.concurrent.CommandAckCollector.access$300(CommandAckCollector.java:51) ~[infinispan-embedded-8.5.0.Final-redhat-6.jar:8.5.0.Final-redhat-6]
> at org.infinispan.util.concurrent.CommandAckCollector$BaseCollector.call(CommandAckCollector.java:214) [infinispan-embedded-8.5.0.Final-redhat-6.jar:8.5.0.Final-redhat-6]
> at org.infinispan.util.concurrent.CommandAckCollector$BaseCollector.call(CommandAckCollector.java:191) [infinispan-embedded-8.5.0.Final-redhat-6.jar:8.5.0.Final-redhat-6]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_161]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_161]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_161]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> {noformat}
> After some investigation together with [~dan.berindei], we found that the FD_ALL is just too slow.
> {noformat}
> <FD_ALL timeout="60000"
> interval="15000"
> timeout_check_interval="5000"
> />
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (ISPN-8981) Generate Hot Rod parser automatically
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-8981?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-8981:
----------------------------------
Sprint: (was: Sprint 9.3.0.Beta1)
> Generate Hot Rod parser automatically
> -------------------------------------
>
> Key: ISPN-8981
> URL: https://issues.jboss.org/browse/ISPN-8981
> Project: Infinispan
> Issue Type: Enhancement
> Components: Server
> Affects Versions: 9.2.0.Final
> Reporter: Radim Vansa
> Assignee: Radim Vansa
> Fix For: 9.3.0.Final
>
>
> This JIRA has two objectives:
> 1. reduce number of allocated objects
> 2. improve the parsing on server side to avoid chains of lambda mappings
> Manual parsing of Hot Rod protocol, invoking recursive methods that return {{Optional}}s or {{Optional<Optional<...>}}s seems to generate a lot of garbage. A better approach would be a finite state automaton that would read the byte stream and invoke callbacks.
> Such automaton can be generated from a high-level grammar as part of the build process.
> Along with these changes we can remove the {{Response}} abstraction and write responses directly as {{ByteBuf}}s.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (ISPN-9094) ArrayIndexOutOfBoundsException on server using scattered cache
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-9094?page=com.atlassian.jira.plugin.... ]
Radim Vansa updated ISPN-9094:
------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/5937
> ArrayIndexOutOfBoundsException on server using scattered cache
> ---------------------------------------------------------------
>
> Key: ISPN-9094
> URL: https://issues.jboss.org/browse/ISPN-9094
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.1.Final
> Reporter: Paul Ferraro
> Assignee: Radim Vansa
>
> We hit ArrayIndexOutOfBoundsException when running tests for RFE EAP7-867.
> EAP distribution was built from {{https://github.com/pferraro/wildfly/tree/scattered}} .
> Test description: Positive stress test (no failover), 4-node EAP cluster, clients: starting with 400 clients in the beginning, raising the number of clients to 6000 in the end of the test.
> Error occured on server dev215 around 7th iteration (can be seen in the performance report, link below):
> {code}
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) Exception in thread "transport-thread--p15-t25" java.lang.ArrayIndexOutOfBoundsException: 129
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at org.infinispan.scattered.impl.ScatteredVersionManagerImpl.lambda$tryRegularInvalidations$4(ScatteredVersionManagerImpl.java:413)
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at java.lang.Thread.run(Thread.java:748)
> {code}
> Clients were getting "SocketTimeoutException: Read timed out" exceptions even before the ArrayIndexOutOfBoundsException ocurred, but also after.
> Performance report (accessible only when connected to VPN):
> http://download.eng.brq.redhat.com/scratch/mvinkler/reports/2018-04-19_15...
> One can observe that dev215 CPU usage and network usage dropped after 7th iteration.
> dev215 server log link:
> https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-se...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (ISPN-9099) Staggered remote get throws IllegalStateException
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-9099?page=com.atlassian.jira.plugin.... ]
Ryan Emerson updated ISPN-9099:
-------------------------------
Sprint: Sprint 9.3.0.Beta1
> Staggered remote get throws IllegalStateException
> -------------------------------------------------
>
> Key: ISPN-9099
> URL: https://issues.jboss.org/browse/ISPN-9099
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.3.0.Beta1
>
>
> {{StaggeredRequest.sendNextMessage()}} throws {{IllegalStateException}} if the last target has left the cluster and the next-to-last target didn't reply within the stagger timeout. This causes a random failure in {{TwoWaySplitAndMergeTest}}:
> {noformat}
> 12:29:10,960 ERROR (testng-TwoWaySplitAndMergeTest[DIST_SYNC]:[]) [TestSuiteProgress] Test failed: org.infinispan.partitionhandling.TwoWaySplitAndMergeTest.testSplitAndMerge4[DIST_SYNC]
> org.infinispan.commons.CacheException: java.lang.IllegalStateException: Request should have been completed already.
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:134) ~[classes/:?]
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.lambda$new$0(InvocationContextInterceptor.java:62) ~[classes/:?]
> at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_171]
> at org.infinispan.remoting.transport.AbstractRequest.completeExceptionally(AbstractRequest.java:74) ~[classes/:?]
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.sendNextMessage(StaggeredRequest.java:106) ~[classes/:?]
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.onTimeout(StaggeredRequest.java:66) ~[classes/:?]
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) ~[classes/:?]
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) ~[classes/:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_171]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_171]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
> Suppressed: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.IllegalStateException: Request should have been completed already.
> at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) ~[?:1.8.0_171]
> at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:82) ~[classes/:?]
> at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:37) ~[classes/:?]
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:485) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:478) ~[classes/:?]
> at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:348) ~[classes/:?]
> at org.infinispan.cache.impl.EncoderCache.get(EncoderCache.java:658) ~[classes/:?]
> at org.infinispan.partitionhandling.BasePartitionHandlingTest.assertKeyAvailableForRead(BasePartitionHandlingTest.java:396) ~[test-classes/:?]
> at org.infinispan.partitionhandling.BasePartitionHandlingTest$Partition.assertKeyAvailableForRead(BasePartitionHandlingTest.java:325) ~[test-classes/:?]
> at org.infinispan.partitionhandling.TwoWaySplitAndMergeTest.lambda$testSplitAndMerge$1(TwoWaySplitAndMergeTest.java:96) ~[test-classes/:?]
> at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110) ~[?:1.8.0_171]
> at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557) ~[?:1.8.0_171]
> at org.infinispan.partitionhandling.TwoWaySplitAndMergeTest.testSplitAndMerge(TwoWaySplitAndMergeTest.java:95) ~[test-classes/:?]
> at org.infinispan.partitionhandling.TwoWaySplitAndMergeTest.testSplitAndMerge4(TwoWaySplitAndMergeTest.java:43) ~[test-classes/:?]
> Caused by: org.infinispan.commons.CacheException: java.lang.IllegalStateException: Request should have been completed already.
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:134) ~[classes/:?]
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.lambda$new$0(InvocationContextInterceptor.java:62) ~[classes/:?]
> at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_171]
> at org.infinispan.remoting.transport.AbstractRequest.completeExceptionally(AbstractRequest.java:74) ~[classes/:?]
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.sendNextMessage(StaggeredRequest.java:106) ~[classes/:?]
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.onTimeout(StaggeredRequest.java:66) ~[classes/:?]
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) ~[classes/:?]
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) ~[classes/:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_171]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_171]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_171]
> ... 3 more
> Caused by: java.lang.IllegalStateException: Request should have been completed already.
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.sendNextMessage(StaggeredRequest.java:88) ~[classes/:?]
> ... 9 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (ISPN-9099) Staggered remote get throws IllegalStateException
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-9099?page=com.atlassian.jira.plugin.... ]
Ryan Emerson resolved ISPN-9099.
--------------------------------
Resolution: Done
> Staggered remote get throws IllegalStateException
> -------------------------------------------------
>
> Key: ISPN-9099
> URL: https://issues.jboss.org/browse/ISPN-9099
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.3.0.Beta1
>
>
> {{StaggeredRequest.sendNextMessage()}} throws {{IllegalStateException}} if the last target has left the cluster and the next-to-last target didn't reply within the stagger timeout. This causes a random failure in {{TwoWaySplitAndMergeTest}}:
> {noformat}
> 12:29:10,960 ERROR (testng-TwoWaySplitAndMergeTest[DIST_SYNC]:[]) [TestSuiteProgress] Test failed: org.infinispan.partitionhandling.TwoWaySplitAndMergeTest.testSplitAndMerge4[DIST_SYNC]
> org.infinispan.commons.CacheException: java.lang.IllegalStateException: Request should have been completed already.
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:134) ~[classes/:?]
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.lambda$new$0(InvocationContextInterceptor.java:62) ~[classes/:?]
> at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_171]
> at org.infinispan.remoting.transport.AbstractRequest.completeExceptionally(AbstractRequest.java:74) ~[classes/:?]
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.sendNextMessage(StaggeredRequest.java:106) ~[classes/:?]
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.onTimeout(StaggeredRequest.java:66) ~[classes/:?]
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) ~[classes/:?]
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) ~[classes/:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_171]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_171]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
> Suppressed: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.IllegalStateException: Request should have been completed already.
> at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) ~[?:1.8.0_171]
> at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:82) ~[classes/:?]
> at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:37) ~[classes/:?]
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:485) ~[classes/:?]
> at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:478) ~[classes/:?]
> at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:348) ~[classes/:?]
> at org.infinispan.cache.impl.EncoderCache.get(EncoderCache.java:658) ~[classes/:?]
> at org.infinispan.partitionhandling.BasePartitionHandlingTest.assertKeyAvailableForRead(BasePartitionHandlingTest.java:396) ~[test-classes/:?]
> at org.infinispan.partitionhandling.BasePartitionHandlingTest$Partition.assertKeyAvailableForRead(BasePartitionHandlingTest.java:325) ~[test-classes/:?]
> at org.infinispan.partitionhandling.TwoWaySplitAndMergeTest.lambda$testSplitAndMerge$1(TwoWaySplitAndMergeTest.java:96) ~[test-classes/:?]
> at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110) ~[?:1.8.0_171]
> at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557) ~[?:1.8.0_171]
> at org.infinispan.partitionhandling.TwoWaySplitAndMergeTest.testSplitAndMerge(TwoWaySplitAndMergeTest.java:95) ~[test-classes/:?]
> at org.infinispan.partitionhandling.TwoWaySplitAndMergeTest.testSplitAndMerge4(TwoWaySplitAndMergeTest.java:43) ~[test-classes/:?]
> Caused by: org.infinispan.commons.CacheException: java.lang.IllegalStateException: Request should have been completed already.
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:134) ~[classes/:?]
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.lambda$new$0(InvocationContextInterceptor.java:62) ~[classes/:?]
> at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_171]
> at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_171]
> at org.infinispan.remoting.transport.AbstractRequest.completeExceptionally(AbstractRequest.java:74) ~[classes/:?]
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.sendNextMessage(StaggeredRequest.java:106) ~[classes/:?]
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.onTimeout(StaggeredRequest.java:66) ~[classes/:?]
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) ~[classes/:?]
> at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) ~[classes/:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_171]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_171]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_171]
> ... 3 more
> Caused by: java.lang.IllegalStateException: Request should have been completed already.
> at org.infinispan.remoting.transport.jgroups.StaggeredRequest.sendNextMessage(StaggeredRequest.java:88) ~[classes/:?]
> ... 9 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months