[JBoss JIRA] (ISPN-8715) Local counters
by Pedro Ruivo (JIRA)
Pedro Ruivo created ISPN-8715:
---------------------------------
Summary: Local counters
Key: ISPN-8715
URL: https://issues.jboss.org/browse/ISPN-8715
Project: Infinispan
Issue Type: Feature Request
Reporter: Pedro Ruivo
Assignee: Pedro Ruivo
If the cache manager doesn't have a transport configured, we don't allow the creation of counters.
We can add non-local counters for that scenario.
The counters created will be based on the {{GlobalConfiguration}} used.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8616) DistAsyncFuncTest.testMergeFromNonOwner random failures
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8616?page=com.atlassian.jira.plugin.... ]
Radim Vansa commented on ISPN-8616:
-----------------------------------
I believe this is because after ISPN-7919 the {{invokeCommandAsync}} was replaced with {{invokeCommand}} et all, and the DistAsyncFuncTest overrides directly {{invokeCommandAsync}} to check who is sending invalidations.
> DistAsyncFuncTest.testMergeFromNonOwner random failures
> -------------------------------------------------------
>
> Key: ISPN-8616
> URL: https://issues.jboss.org/browse/ISPN-8616
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.2.0.Beta1
> Reporter: Dan Berindei
> Assignee: Radim Vansa
> Labels: testsuite_stability
>
> {noformat}
> java.lang.AssertionError: Fail on owner cache DistAsyncFuncTest[DIST_ASYNC, tx=false]-NodeA-42474: dc.get(k1) returned null
> at org.infinispan.distribution.BaseDistFunctionalTest.assertOwnershipAndNonOwnership(BaseDistFunctionalTest.java:191)
> at org.infinispan.distribution.BaseDistFunctionalTest.assertOnAllCachesAndOwnership(BaseDistFunctionalTest.java:162)
> at org.infinispan.distribution.BaseDistFunctionalTest.initAndTest(BaseDistFunctionalTest.java:142)
> at org.infinispan.distribution.DistSyncFuncTest.testMergeFromNonOwner(DistSyncFuncTest.java:387)
> {noformat}
> The test method is new and the interceptors have changed, but it's probably the same problem with {{asyncWait}} signaled in ISPN-3741. I commented out the entire implementation of {{asyncWait}} in {{DistAsyncFuncTest}} and it still passed when run by itself, so we need a way to delay the commands and check that the test is not sensitive to such delays.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8616) DistAsyncFuncTest.testMergeFromNonOwner random failures
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8616?page=com.atlassian.jira.plugin.... ]
Radim Vansa reassigned ISPN-8616:
---------------------------------
Assignee: Radim Vansa (was: Ryan Emerson)
> DistAsyncFuncTest.testMergeFromNonOwner random failures
> -------------------------------------------------------
>
> Key: ISPN-8616
> URL: https://issues.jboss.org/browse/ISPN-8616
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.2.0.Beta1
> Reporter: Dan Berindei
> Assignee: Radim Vansa
> Labels: testsuite_stability
>
> {noformat}
> java.lang.AssertionError: Fail on owner cache DistAsyncFuncTest[DIST_ASYNC, tx=false]-NodeA-42474: dc.get(k1) returned null
> at org.infinispan.distribution.BaseDistFunctionalTest.assertOwnershipAndNonOwnership(BaseDistFunctionalTest.java:191)
> at org.infinispan.distribution.BaseDistFunctionalTest.assertOnAllCachesAndOwnership(BaseDistFunctionalTest.java:162)
> at org.infinispan.distribution.BaseDistFunctionalTest.initAndTest(BaseDistFunctionalTest.java:142)
> at org.infinispan.distribution.DistSyncFuncTest.testMergeFromNonOwner(DistSyncFuncTest.java:387)
> {noformat}
> The test method is new and the interceptors have changed, but it's probably the same problem with {{asyncWait}} signaled in ISPN-3741. I commented out the entire implementation of {{asyncWait}} in {{DistAsyncFuncTest}} and it still passed when run by itself, so we need a way to delay the commands and check that the test is not sensitive to such delays.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8714) Don't send acks for async-triangle
by Pedro Ruivo (JIRA)
Pedro Ruivo created ISPN-8714:
---------------------------------
Summary: Don't send acks for async-triangle
Key: ISPN-8714
URL: https://issues.jboss.org/browse/ISPN-8714
Project: Infinispan
Issue Type: Enhancement
Components: Core
Reporter: Pedro Ruivo
Assignee: Pedro Ruivo
With asynchronous caches, the originator doesn't expect a reply from the backups. We don't need to send the ack.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8712) BiasRevocationTest.testFailedRevocationDuringPutAllOnNonOwnerThrowBefore random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8712?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8712:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/5692
> BiasRevocationTest.testFailedRevocationDuringPutAllOnNonOwnerThrowBefore random failures
> ----------------------------------------------------------------------------------------
>
> Key: ISPN-8712
> URL: https://issues.jboss.org/browse/ISPN-8712
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.2.0.CR1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Labels: testsuite_stability
> Fix For: 9.2.0.CR2
>
> Attachments: BiasRevocationTest_ISPN-8644_RemoteGetDuringStateTransferTest_20180116.log.gz
>
>
> The test doesn't wait for the cluster to form before issuing a write operation. After a topology change, the retried command fails on the primary owner with a {{NullPointerException}}:
> {noformat}
> 00:44:52,166 TRACE (remote-thread-Test-NodeA-p63633-t5:[]) [TrianglePerCacheInboundInvocationHandler] Topology changed, retrying: org.infinispan.statetransfer.OutdatedTopologyException
> 00:44:52,166 TRACE (remote-thread-Test-NodeA-p63633-t5:[]) [TrianglePerCacheInboundInvocationHandler] Sending exception ack for command CommandInvocation:Test-NodeB-50920:99611. Originator=Test-NodeB-50920.
> 00:44:52,166 TRACE (remote-thread-Test-NodeA-p63633-t5:[]) [JGroupsTransport] Test-NodeA-62783 sending command to Test-NodeB-50920: ExceptionAckCommand{id=99611, throwable=org.infinispan.remoting.RemoteException: Exception on Test-NodeA-62783, topologyId=4}
> 00:44:52,167 TRACE (jgroups-6,Test-NodeB-50920:[]) [JGroupsTransport] Test-NodeB-50920 received command from Test-NodeA-62783: ExceptionAckCommand{id=99611, throwable=org.infinispan.remoting.RemoteException: Exception on Test-NodeA-62783, topologyId=4}
> 00:44:52,177 TRACE (jgroups-6,Test-NodeB-50920:[]) [EntryWrappingInterceptor] This is a retry - resetting previous values for [MagicKey{2379/BEF29B28/0@Test-NodeA-62783}]
> 00:44:52,177 ERROR (jgroups-6,Test-NodeB-50920:[]) [InvocationContextInterceptor] ISPN000136: Error executing command PutMapCommand, writing keys [MagicKey{2379/BEF29B28/0@Test-NodeA-62783}]
> java.lang.NullPointerException: null
> at org.infinispan.interceptors.impl.EntryWrappingInterceptor.removeFromContextOnRetry(EntryWrappingInterceptor.java:357) ~[classes/:?]
> at org.infinispan.interceptors.impl.EntryWrappingInterceptor.visitPutMapCommand(EntryWrappingInterceptor.java:393) ~[classes/:?]
> at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:80) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59) ~[classes/:?]
> at org.infinispan.interceptors.impl.PrefetchInterceptor.handleWriteManyCommand(PrefetchInterceptor.java:394) ~[classes/:?]
> at org.infinispan.interceptors.impl.PrefetchInterceptor.visitPutMapCommand(PrefetchInterceptor.java:450) ~[classes/:?]
> at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:80) ~[classes/:?]
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndHandle(BaseAsyncInterceptor.java:188) ~[classes/:?]
> at org.infinispan.interceptors.impl.BaseStateTransferInterceptor.retryWhenDone(BaseStateTransferInterceptor.java:157) ~[classes/:?]
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteReturn(StateTransferInterceptor.java:338) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81) ~[classes/:?]
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[?:1.8.0_152]
> at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[?:1.8.0_152]
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_152]
> at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_152]
> at org.infinispan.interceptors.distribution.ScatteredDistributionInterceptor$SyncMergingCompletableFuture.completeExceptionally(ScatteredDistributionInterceptor.java:1397) ~[classes/:?]
> at org.infinispan.interceptors.distribution.ScatteredDistributionInterceptor.lambda$handleWriteManyOnOrigin$25(ScatteredDistributionInterceptor.java:1014) ~[classes/:?]
> at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [?:1.8.0_152]
> at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [?:1.8.0_152]
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [?:1.8.0_152]
> at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) [?:1.8.0_152]
> at org.infinispan.util.concurrent.CommandAckCollector$CombiningCollector.primaryException(CommandAckCollector.java:553) [classes/:?]
> at org.infinispan.util.concurrent.CommandAckCollector$MultiAckTarget.completeExceptionally(CommandAckCollector.java:641) [classes/:?]
> at org.infinispan.util.concurrent.CommandAckCollector.completeExceptionally(CommandAckCollector.java:206) [classes/:?]
> at org.infinispan.commands.write.ExceptionAckCommand.ack(ExceptionAckCommand.java:43) [classes/:?]
> at org.infinispan.remoting.inboundhandler.TrianglePerCacheInboundInvocationHandler.handleExceptionAck(TrianglePerCacheInboundInvocationHandler.java:212) [classes/:?]
> at org.infinispan.remoting.inboundhandler.TrianglePerCacheInboundInvocationHandler.handle(TrianglePerCacheInboundInvocationHandler.java:112) [classes/:?]
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleCacheRpcCommand(GlobalInboundInvocationHandler.java:120) [classes/:?]
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleFromCluster(GlobalInboundInvocationHandler.java:79) [classes/:?]
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processRequest(JGroupsTransport.java:1295) [classes/:?]
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1219) [classes/:?]
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$200(JGroupsTransport.java:124) [classes/:?]
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.receive(JGroupsTransport.java:1357) [classes/:?]
> 00:44:52,205 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.scattered.BiasRevocationTest.testFailedRevocationDuringPutAllOnNonOwnerThrowBefore
> java.lang.AssertionError: Wrong exception thrown: expected:<org.infinispan.util.concurrent.TimeoutException>, actual:<org.infinispan.commons.CacheException>
> at org.infinispan.test.Exceptions.assertException(Exceptions.java:22) ~[test-classes/:?]
> at org.infinispan.test.Exceptions.expectException(Exceptions.java:99) ~[test-classes/:?]
> at org.infinispan.scattered.BiasRevocationTest.testFailedRevocation(BiasRevocationTest.java:105) ~[test-classes/:?]
> at org.infinispan.scattered.BiasRevocationTest.testFailedRevocationDuringPutAllOnNonOwnerThrowBefore(BiasRevocationTest.java:82) ~[test-classes/:?]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8713) "Initial state transfer timed out" in border case
by Walter Pongratz (JIRA)
Walter Pongratz created ISPN-8713:
-------------------------------------
Summary: "Initial state transfer timed out" in border case
Key: ISPN-8713
URL: https://issues.jboss.org/browse/ISPN-8713
Project: Infinispan
Issue Type: Bug
Components: Core
Affects Versions: 9.1.4.Final
Reporter: Walter Pongratz
Priority: Critical
There is a bug in Infinispan 9.1.4 in a cluster where not every cache ist started on each node. For detailed steps to reproduce see below, short summary: If a cache is not started on the coordinator and the last node is not leaving gracefully that cache can not be started again until the coordinator changes.
Known aspects:
* In Infinispan 8.2.6 the same steps do NOT lead to an issue
Likely cause: On a non-graceful exit of the last node with a certain cache the cache is not removed from the cacheStatusMap in ClusterTopologyManagerImpl on the coordinator, the ClusterCacheStatus is only manipulated - See TODO in ClusterCacheStatus.updateCurrentTopology(). On a graceful exit this is done - see ClusterTopologyManagerImpl.handleLeave().
Now somehow a new Node starting the Cache in question waits for the initial state transfer - which never happends because there is no other node with this cache. In Infinispan 8.2.6. this seemed not to be a problem - but in 9.1.4 it is. The fix then would be to either fix this Todo and remove the ClusterCacheStatus from the map OR fix that the new node is not waiting for initial state transfer in this case.
I set the priority to critical because of the difficulty in fixing this in a production envrionment: Once this problem happens the cache can not be started ON ANY NODE OF THE CLUSTER until the coordinator is changed. If a new Node does not start operations personel would assume a problem with that node and try to restart it.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8644) RemoteGetDuringStateTransferTest.testScenario_2322 random test failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8644?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8644:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/5691
> RemoteGetDuringStateTransferTest.testScenario_2322 random test failures
> -----------------------------------------------------------------------
>
> Key: ISPN-8644
> URL: https://issues.jboss.org/browse/ISPN-8644
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.2.0.Beta2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.2.0.CR2
>
>
> {noformat}
> java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.AssertionError: expected:<RemoteGetDuringStateTransferTest-NodeC-51737> but was:<RemoteGetDuringStateTransferTest-NodeB-41634>
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.infinispan.statetransfer.RemoteGetDuringStateTransferTest.testScenario_2xy2(RemoteGetDuringStateTransferTest.java:526)
> at org.infinispan.statetransfer.RemoteGetDuringStateTransferTest.testScenario_2322(RemoteGetDuringStateTransferTest.java:452)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.infinispan.commons.CacheException: java.lang.AssertionError: expected:<RemoteGetDuringStateTransferTest-NodeC-51737> but was:<RemoteGetDuringStateTransferTest-NodeB-41634>
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:141)
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.lambda$new$0(InvocationContextInterceptor.java:62)
> at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21)
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118)
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81)
> at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30)
> at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:66)
> at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:56)
> at org.infinispan.remoting.transport.impl.SingleTargetRequest.onResponse(SingleTargetRequest.java:35)
> at org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:53)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1329)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1239)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$200(JGroupsTransport.java:121)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.receive(JGroupsTransport.java:1367)
> at org.jgroups.JChannel.up(JChannel.java:819)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:893)
> at org.jgroups.protocols.RSVP.up(RSVP.java:163)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
> at org.jgroups.protocols.tom.TOA.up(TOA.java:112)
> at org.jgroups.protocols.pbcast.GMS.up(GMS.java:864)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:240)
> at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1002)
> at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:728)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:383)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
> at org.jgroups.protocols.Discovery.up(Discovery.java:267)
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1229)
> at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
> ... 3 more
> Caused by: java.lang.AssertionError: expected:<RemoteGetDuringStateTransferTest-NodeC-51737> but was:<RemoteGetDuringStateTransferTest-NodeB-41634>
> at org.infinispan.statetransfer.RemoteGetDuringStateTransferTest.lambda$testScenario_2xy2$6(RemoteGetDuringStateTransferTest.java:500)
> at org.infinispan.tx.dld.ControlledRpcManager.lambda$checkResponses$0(ControlledRpcManager.java:86)
> at org.infinispan.tx.dld.ControlledRpcManager.lambda$afterInvokeRemotely$1(ControlledRpcManager.java:132)
> at java.util.ArrayList.forEach(ArrayList.java:1255)
> at org.infinispan.tx.dld.ControlledRpcManager.afterInvokeRemotely(ControlledRpcManager.java:132)
> at org.infinispan.util.AbstractControlledRpcManager.lambda$invokeRemotelyAsync$0(AbstractControlledRpcManager.java:43)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> ... 27 more
> {noformat}
> This cannot happen anymore after the ISPN-7919 fix landed, because {{ControlledRpcManager.checkResponses()}} doesn't see who sent the response, but we should go back and check that the response is from the node we expected.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months
[JBoss JIRA] (ISPN-8693) Improve ControlledRpcManager
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8693?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8693:
-------------------------------
Status: Open (was: New)
> Improve ControlledRpcManager
> ----------------------------
>
> Key: ISPN-8693
> URL: https://issues.jboss.org/browse/ISPN-8693
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Core
> Affects Versions: 9.2.0.Beta2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.2.0.Final
>
>
> The {{ControlledRpcManager}} behaviour is not always obvious: because it only blocks once, it has a lot of tacked-on features to better filter what command to block on.
> Block all commands would be better, because the test would have to unblock them explicitly and make it clear what commands it expects.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 4 months