[JBoss JIRA] (ISPN-9087) Timeout during put operation when a node is blocked
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-9087?page=com.atlassian.jira.plugin.... ]
Ryan Emerson resolved ISPN-9087.
--------------------------------
Fix Version/s: 9.3.0.CR1
Resolution: Done
> Timeout during put operation when a node is blocked
> ---------------------------------------------------
>
> Key: ISPN-9087
> URL: https://issues.jboss.org/browse/ISPN-9087
> Project: Infinispan
> Issue Type: Bug
> Reporter: Diego Lovison
> Assignee: Dan Berindei
> Fix For: 9.3.0.CR1
>
>
> {noformat}
> 2018-04-17 13:30:02.782 ERROR 14932 --- [timeout-thread--p3-t1] o.i.i.impl.InvocationContextInterceptor : ISPN000136: Error executing command PutKeyValueCommand, writing keys [5db796a3-3f65-468a-b86a-6d5ef8b4b330]
>
> org.infinispan.util.concurrent.TimeoutException: ISPN000427: Timeout after 15 seconds waiting for acks. Id=100000
> at org.infinispan.util.concurrent.CommandAckCollector.createTimeoutException(CommandAckCollector.java:188) ~[infinispan-embedded-8.5.0.Final-redhat-6.jar:8.5.0.Final-redhat-6]
> at org.infinispan.util.concurrent.CommandAckCollector.access$300(CommandAckCollector.java:51) ~[infinispan-embedded-8.5.0.Final-redhat-6.jar:8.5.0.Final-redhat-6]
> at org.infinispan.util.concurrent.CommandAckCollector$BaseCollector.call(CommandAckCollector.java:214) [infinispan-embedded-8.5.0.Final-redhat-6.jar:8.5.0.Final-redhat-6]
> at org.infinispan.util.concurrent.CommandAckCollector$BaseCollector.call(CommandAckCollector.java:191) [infinispan-embedded-8.5.0.Final-redhat-6.jar:8.5.0.Final-redhat-6]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_161]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_161]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_161]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> {noformat}
> After some investigation together with [~dan.berindei], we found that the FD_ALL is just too slow.
> {noformat}
> <FD_ALL timeout="60000"
> interval="15000"
> timeout_check_interval="5000"
> />
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-7977) Clean up NullPointerExceptions in the core test logs
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-7977?page=com.atlassian.jira.plugin.... ]
Ryan Emerson resolved ISPN-7977.
--------------------------------
Fix Version/s: 9.3.0.CR1
Resolution: Done
> Clean up NullPointerExceptions in the core test logs
> ----------------------------------------------------
>
> Key: ISPN-7977
> URL: https://issues.jboss.org/browse/ISPN-7977
> Project: Infinispan
> Issue Type: Task
> Components: Core, Test Suite - Core
> Affects Versions: 9.1.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.3.0.CR1
>
> Attachments: ClusterTopologyManagerImpl_NullPointerException.txt
>
>
> Even benign {{NullPointerException}} s are a bad smell, so tests should never trigger them. But they seem pretty common in the test logs nowadays. Some examples from the core test suite:
> {noformat}
> 23:30:26,503 ERROR (stateTransferExecutor-thread-SingleJoinTest[DIST_SYNC, tx=true]-NodeC-p583-t3:[StateRequest-dist]) [RpcManagerImpl] ISPN000073: Unexpected error while replicating
> java.lang.NullPointerException: null
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotelyAsync(JGroupsTransport.java:650) ~[classes/:?]
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotelyAsync(RpcManagerImpl.java:127) ~[classes/:?]
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:168) ~[classes/:?]
> at org.infinispan.statetransfer.InboundTransferTask.requestSegments(InboundTransferTask.java:132) ~[classes/:?]
> at org.infinispan.statetransfer.StateConsumerImpl.lambda$addTransfer$7(StateConsumerImpl.java:1055) ~[classes/:?]
> at org.infinispan.executors.LimitedExecutor.lambda$executeAsync$1(LimitedExecutor.java:101) ~[classes/:?]
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144) [classes/:?]
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33) [classes/:?]
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174) [classes/:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> {noformat}
> {noformat}
> 23:37:52,617 WARN (testng-AsynchronousInvocationTest:[]) [TrianglePerCacheInboundInvocationHandler] ISPN000071: Caught exception when handling command StreamRequestCommand{cacheName='___defaultcache'}
> java.lang.NullPointerException: null
> at org.infinispan.stream.impl.StreamRequestCommand.invokeAsync(StreamRequestCommand.java:89) ~[classes/:?]
> at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokeCommand(BasePerCacheInboundInvocationHandler.java:101) ~[classes/:?]
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.runSync(BaseBlockingRunnable.java:54) [classes/:?]
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:38) [classes/:?]
> at org.infinispan.remoting.inboundhandler.TrianglePerCacheInboundInvocationHandler.handleDefaultCommand(TrianglePerCacheInboundInvocationHandler.java:183) [classes/:?]
> at org.infinispan.remoting.inboundhandler.TrianglePerCacheInboundInvocationHandler.handle(TrianglePerCacheInboundInvocationHandler.java:112) [classes/:?]
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleCacheRpcCommand(GlobalInboundInvocationHandler.java:120) [classes/:?]
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleFromCluster(GlobalInboundInvocationHandler.java:79) [classes/:?]
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:175) [classes/:?]
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:149) [classes/:?]
> at org.infinispan.remoting.AsynchronousInvocationTest.assertDispatchForCommand(AsynchronousInvocationTest.java:176) [test-classes/:?]
> at org.infinispan.remoting.AsynchronousInvocationTest.testCacheRpcCommands(AsynchronousInvocationTest.java:140) [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) [testng-6.8.8.jar:?]
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) [testng-6.8.8.jar:?]
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) [testng-6.8.8.jar:?]
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) [testng-6.8.8.jar:?]
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) [testng-6.8.8.jar:?]
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) [testng-6.8.8.jar:?]
> at org.testng.TestRunner.privateRun(TestRunner.java:767) [testng-6.8.8.jar:?]
> at org.testng.TestRunner.run(TestRunner.java:617) [testng-6.8.8.jar:?]
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:348) [testng-6.8.8.jar:?]
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:38) [testng-6.8.8.jar:?]
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382) [testng-6.8.8.jar:?]
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64) [testng-6.8.8.jar:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> {noformat}
> {noformat}
> 23:37:54,821 TRACE (ForkThread-2,ThreadLocalLeakTest:[]) [InvocationContextInterceptor] Exception while executing code, but we're shutting down so failing silently.
> org.infinispan.persistence.spi.PersistenceException: java.lang.NullPointerException
> at org.infinispan.persistence.file.SingleFileStore.write(SingleFileStore.java:365) ~[classes/:?]
> at org.infinispan.persistence.manager.PersistenceManagerImpl.lambda$writeToAllNonTxStores$9(PersistenceManagerImpl.java:518) ~[classes/:?]
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:1.8.0_121]
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_121]
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_121]
> at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_121]
> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) ~[?:1.8.0_121]
> at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_121]
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) ~[?:1.8.0_121]
> at org.infinispan.persistence.manager.PersistenceManagerImpl.writeToAllNonTxStores(PersistenceManagerImpl.java:518) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.storeEntry(CacheWriterInterceptor.java:452) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.lambda$visitPutKeyValueCommand$1(CacheWriterInterceptor.java:187) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:109) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.visitPutKeyValueCommand(CacheWriterInterceptor.java:179) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheLoaderInterceptor.visitDataCommand(CacheLoaderInterceptor.java:208) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:135) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:102) ~[classes/:?]
> at org.infinispan.interceptors.impl.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:689) ~[classes/:?]
> at org.infinispan.interceptors.impl.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:310) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndFinally(BaseAsyncInterceptor.java:154) ~[classes/:?]
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:136) ~[classes/:?]
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:38) ~[classes/:?]
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:86) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:197) ~[classes/:?]
> at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:162) ~[classes/:?]
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:74) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) [classes/:?]
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:97) [classes/:?]
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) [classes/:?]
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1606) [classes/:?]
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1271) [classes/:?]
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1720) [classes/:?]
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:276) [classes/:?]
> at org.infinispan.cache.impl.AbstractDelegatingCache.put(AbstractDelegatingCache.java:358) [classes/:?]
> at org.infinispan.cache.impl.EncoderCache.put(EncoderCache.java:652) [classes/:?]
> at org.infinispan.util.ThreadLocalLeakTest$2.run(ThreadLocalLeakTest.java:115) [test-classes/:?]
> at org.infinispan.test.AbstractInfinispanTest$RunnableWrapper.run(AbstractInfinispanTest.java:510) [test-classes/:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> Caused by: java.lang.NullPointerException
> at org.infinispan.persistence.file.SingleFileStore.allocate(SingleFileStore.java:232) ~[classes/:?]
> at org.infinispan.persistence.file.SingleFileStore.write(SingleFileStore.java:332) ~[classes/:?]
> ... 46 more
> {noformat}
> {noformat}
> 23:38:21,145 ERROR (transport-thread-OptimisticTxFailureAfterLockingTest-NodeD-p33826-t2:[Topology-___defaultcache]) [LocalTopologyManagerImpl] ISPN000452: Failed to update topology for cache ___defaultcache
> java.lang.NullPointerException: null
> at org.infinispan.topology.PersistentUUIDManagerImpl.mapAddresses(PersistentUUIDManagerImpl.java:70) ~[classes/:?]
> at org.infinispan.topology.LocalTopologyManagerImpl.doHandleTopologyUpdate(LocalTopologyManagerImpl.java:348) ~[classes/:?]
> at org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleTopologyUpdate$1(LocalTopologyManagerImpl.java:266) ~[classes/:?]
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144) [classes/:?]
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33) [classes/:?]
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174) [classes/:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> {noformat}
> {noformat}
> 23:39:19,342 ERROR (transport-thread-ConcurrentStartChanelLookupTest-NodeA-p40485-t3:[Merge-1]) [ClusterCacheStatus] ISPN000228: Failed to recover cache repl state after the current node became the coordinator
> java.lang.NullPointerException: null
> at org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy.lambda$static$0(PreferAvailabilityStrategy.java:34) ~[classes/:?]
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) ~[?:1.8.0_121]
> at java.util.TimSort.sort(TimSort.java:220) ~[?:1.8.0_121]
> at java.util.Arrays.sort(Arrays.java:1512) ~[?:1.8.0_121]
> at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:348) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_121]
> at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_121]
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_121]
> at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_121]
> at org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy.onPartitionMerge(PreferAvailabilityStrategy.java:122) ~[classes/:?]
> at org.infinispan.topology.ClusterCacheStatus.doMergePartitions(ClusterCacheStatus.java:570) ~[classes/:?]
> at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$recoverClusterStatus$4(ClusterTopologyManagerImpl.java:499) ~[classes/:?]
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144) [classes/:?]
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33) [classes/:?]
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174) [classes/:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8782) Occasional NPE on PersistentUUIDManagerImpl.mapAddresses
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-8782?page=com.atlassian.jira.plugin.... ]
Ryan Emerson resolved ISPN-8782.
--------------------------------
Fix Version/s: 9.3.0.CR1
(was: 9.3.0.Final)
Resolution: Done
> Occasional NPE on PersistentUUIDManagerImpl.mapAddresses
> --------------------------------------------------------
>
> Key: ISPN-8782
> URL: https://issues.jboss.org/browse/ISPN-8782
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Reporter: Sanne Grinovero
> Assignee: Dan Berindei
> Fix For: 9.3.0.CR1
>
>
> When running the testsuite I occasionally see stacktraces such as:
> 23:47:50,649 FATAL (transport-thread-RetryMechanismTest[null, tx=false]-NodeT-p64820-t5) [PersistentUUIDManagerImpl] Cannot find mapping for address RetryMechanismTest[null, tx=false]-NodeT-51212 java.lang.NullPointerException
> at org.infinispan.topology.PersistentUUIDManagerImpl.mapAddresses(PersistentUUIDManagerImpl.java:70)
> at org.infinispan.topology.ClusterCacheStatus.startQueuedRebalance(ClusterCacheStatus.java:824)
> at org.infinispan.topology.ClusterCacheStatus.queueRebalance(ClusterCacheStatus.java:125)
> at org.infinispan.partitionhandling.impl.PreferAvailabilityStrategy.onPartitionMerge(PreferAvailabilityStrategy.java:232)
> at org.infinispan.topology.ClusterCacheStatus.doMergePartitions(ClusterCacheStatus.java:597)
> at org.infinispan.topology.ClusterTopologyManagerImpl.lambda$recoverClusterStatus$6(ClusterTopologyManagerImpl.java:519)
> at org.infinispan.executors.LimitedExecutor.runTasks(LimitedExecutor.java:144)
> at org.infinispan.executors.LimitedExecutor.access$100(LimitedExecutor.java:33)
> at org.infinispan.executors.LimitedExecutor$Runner.run(LimitedExecutor.java:174)
> They seem harmless as it seems like the test results aren't affected, but it's still looking quite bad. Would be best if someone had a look.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-9095) NPE during server shutdown when using scattered cache
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-9095?page=com.atlassian.jira.plugin.... ]
Radim Vansa commented on ISPN-9095:
-----------------------------------
We've noticed this in ISPN-9128, too and will be fixed within that scope.
> NPE during server shutdown when using scattered cache
> -----------------------------------------------------
>
> Key: ISPN-9095
> URL: https://issues.jboss.org/browse/ISPN-9095
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.1.Final
> Reporter: Paul Ferraro
> Assignee: Radim Vansa
>
> We hit NPE when running tests for RFE EAP7-867.
> EAP distribution was built from https://github.com/pferraro/wildfly/tree/scattered .
> Test description: Positive stress test (no failover), 4-node EAP cluster, clients: starting with 400 clients in the beginning, raising the number of clients to 6000 in the end of the test.
> During clean server shutdown in the end of the test, server logged NPE and got stuck:
> {code}
> [JBossINF] [0m[31m07:55:57,643 ERROR [org.infinispan.scattered.impl.ScatteredStateConsumerImpl] (thread-200,ejb,dev214) ISPN000471: Failed processing values received from remote node during rebalance.: java.lang.NullPointerException
> [JBossINF] at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.applyValues(ScatteredStateConsumerImpl.java:505)
> [JBossINF] at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.lambda$getValuesAndApply$8(ScatteredStateConsumerImpl.java:475)
> [JBossINF] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> [JBossINF] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> [JBossINF] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> [JBossINF] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> [JBossINF] at org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:66)
> [JBossINF] at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:56)
> [JBossINF] at org.infinispan.remoting.transport.impl.SingleTargetRequest.onResponse(SingleTargetRequest.java:35)
> [JBossINF] at org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:53)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1304)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1207)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$200(JGroupsTransport.java:123)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.receive(JGroupsTransport.java:1342)
> [JBossINF] at org.jgroups.JChannel.up(JChannel.java:819)
> [JBossINF] at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:134)
> [JBossINF] at org.jgroups.stack.Protocol.up(Protocol.java:340)
> [JBossINF] at org.jgroups.protocols.FORK.up(FORK.java:134)
> [JBossINF] at org.jgroups.protocols.FRAG3.up(FRAG3.java:166)
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
> [JBossINF] at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
> [JBossINF] at org.jgroups.protocols.pbcast.GMS.up(GMS.java:864)
> [JBossINF] at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:240)
> [JBossINF] at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1002)
> [JBossINF] at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:728)
> [JBossINF] at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:383)
> [JBossINF] at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
> [JBossINF] at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:119)
> [JBossINF] at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:199)
> [JBossINF] at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:252)
> [JBossINF] at org.jgroups.protocols.MERGE3.up(MERGE3.java:276)
> [JBossINF] at org.jgroups.protocols.Discovery.up(Discovery.java:267)
> [JBossINF] at org.jgroups.protocols.TP.passMessageUp(TP.java:1248)
> [JBossINF] at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52)
> [JBossINF] at java.lang.Thread.run(Thread.java:748)
> [JBossINF]
> {code}
> Scattered cache was configured with bias-lifespan="0".
> Server configuration:
> http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-ses...
> Server link:
> http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-ses...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8836) Custom region definition causes ISPN000453
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8836?page=com.atlassian.jira.plugin.... ]
Radim Vansa commented on ISPN-8836:
-----------------------------------
No, this is unrelated. ISPN-9136 is more about redeployments; this happens for the first time.
The cause is pretty obvious to me - while [~anton.zalesky] did not provide his persistence.xml/whatever list of properties Spring uses, he's probably trying to have certain configuration for given region. InfinispanRegionFactory does not know about this link and tries to create a new configuration based on the 'entity' cache and name it according to the region name.
> Custom region definition causes ISPN000453
> ------------------------------------------
>
> Key: ISPN-8836
> URL: https://issues.jboss.org/browse/ISPN-8836
> Project: Infinispan
> Issue Type: Bug
> Components: Hibernate Cache
> Affects Versions: 9.2.0.CR2
> Environment: Hibernate 5.2.13 SpringFramework 5.0.2.RELEASE
> Reporter: Anton Zalesky
> Assignee: Radim Vansa
> Priority: Critical
>
> I have entity with defined region
> @Entity
> @Table(name = "Dashboard")
> @Cache(usage = CacheConcurrencyStrategy.READ_WRITE, region = "dashboard")
> public class Dashboard {..}
> And I have infinispan.xml with custom configuration for region
> <local-cache name="dashboard" simple-cache="true" statistics="true" statistics-available="true">
> <transaction mode="NONE" />
> <expiration max-idle="14400000" />
> <memory>
> <object size="10000"/>
> </memory>
> </local-cache>
> On start I get the following exception
> org.infinispan.commons.CacheConfigurationException: ISPN000453: Attempt to define configuration for cache dashboard which already exists
> at org.infinispan.manager.DefaultCacheManager.doDefineConfiguration(DefaultCacheManager.java:370)
> at org.infinispan.manager.DefaultCacheManager.defineConfiguration(DefaultCacheManager.java:342)
> at org.infinispan.hibernate.cache.commons.InfinispanRegionFactory.getCache(InfinispanRegionFactory.java:692)
> at org.infinispan.hibernate.cache.commons.InfinispanRegionFactory.buildEntityRegion(InfinispanRegionFactory.java:332)
> at org.hibernate.cache.spi.RegionFactory.buildEntityRegion(RegionFactory.java:132)
> at org.hibernate.internal.CacheImpl.determineEntityRegionAccessStrategy(CacheImpl.java:439)
> at org.hibernate.metamodel.internal.MetamodelImpl.initialize(MetamodelImpl.java:120)
> at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:300)
> at org.hibernate.boot.internal.SessionFactoryBuilderImpl.build(SessionFactoryBuilderImpl.java:460)
> at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:710)
> at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:726)
> at org.springframework.orm.hibernate5.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:535)
> at org.springframework.orm.hibernate5.LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:519)
> at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1769)
> at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1706)
> ... 97 more
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months