[JBoss JIRA] (ISPN-3141) Conditional commands with Flag.IGNORE_RETURN_VALUES cause NPEs
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3141?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-3141:
--------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/1843
> Conditional commands with Flag.IGNORE_RETURN_VALUES cause NPEs
> ---------------------------------------------------------------
>
> Key: ISPN-3141
> URL: https://issues.jboss.org/browse/ISPN-3141
> Project: Infinispan
> Issue Type: Bug
> Components: Core API
> Affects Versions: 5.2.6.Final
> Reporter: Mircea Markus
> Assignee: Mircea Markus
> Fix For: 5.3.0.CR1, 5.3.0.Final
>
>
> The root cause of the exception below is:
> - when Flag.IGNORE_RETURN_VALUES is used the node on which the operation is executed returns NULL
> - the operation being conditional return boolean to the user. The null above is unboxed to a boolean which causes a NPE
> The solution is to ignore the Flag.IGNORE_RETURN_VALUES for conditional operations (remove(k, prevValue) replace(k, prevValue,newValue)) and return the result of the operation: this is a boolean, so the network payload is the same as for null.
> {quote}
> 14:07:57,149 WARN [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (Incoming-2,shared=tcp) ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='default', command=ReplaceCommand{key=name, oldValue=[B0x42656c61..(4), newValue=[B0x42656c61..(4), metadata=MimeMetadata(text/plain; charset=UTF-8,-1,SECONDS,-1,SECONDS,null,null), flags=null, successful=true, ignorePreviousValue=false}}: java.lang.NullPointerException
> at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:880)
> at org.infinispan.CacheImpl.replace(CacheImpl.java:871)
> at org.infinispan.DecoratedCache.replace(DecoratedCache.java:551)
> at org.infinispan.xsite.BackupReceiverImpl$BackupCacheUpdater.visitReplaceCommand(BackupReceiverImpl.java:113)
> at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:76)
> at org.infinispan.xsite.BackupReceiverImpl.handleRemoteCommand(BackupReceiverImpl.java:76)
> at org.infinispan.xsite.BackupReceiverRepositoryImpl.handleRemoteCommand(BackupReceiverRepositoryImpl.java:87)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromRemoteSite(CommandAwareRpcDispatcher.java:255)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:230)
> at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:460) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:665) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.JChannel.up(JChannel.java:719) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1008) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.protocols.relay.RELAY2.deliver(RELAY2.java:607) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.protocols.relay.RELAY2.route(RELAY2.java:507) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.protocols.relay.RELAY2.handleMessage(RELAY2.java:482) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.protocols.relay.RELAY2.handleRelayMessage(RELAY2.java:463) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.protocols.relay.Relayer$Bridge.receive(Relayer.java:302) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.JChannel.up(JChannel.java:749) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1012) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 10 months
[JBoss JIRA] (ISPN-3141) Conditional commands with Flag.IGNORE_RETURN_VALUES cause NPEs
by Mircea Markus (JIRA)
Mircea Markus created ISPN-3141:
-----------------------------------
Summary: Conditional commands with Flag.IGNORE_RETURN_VALUES cause NPEs
Key: ISPN-3141
URL: https://issues.jboss.org/browse/ISPN-3141
Project: Infinispan
Issue Type: Bug
Components: Core API
Affects Versions: 5.2.6.Final
Reporter: Mircea Markus
Assignee: Mircea Markus
Fix For: 5.3.0.CR1, 5.3.0.Final
The root cause of the exception below is:
- when Flag.IGNORE_RETURN_VALUES is used the node on which the operation is executed returns NULL
- the operation being conditional return boolean to the user. The null above is unboxed to a boolean which causes a NPE
The solution is to ignore the Flag.IGNORE_RETURN_VALUES for conditional operations (remove(k, prevValue) replace(k, prevValue,newValue)) and return the result of the operation: this is a boolean, so the network payload is the same as for null.
{quote}
14:07:57,149 WARN [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (Incoming-2,shared=tcp) ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='default', command=ReplaceCommand{key=name, oldValue=[B0x42656c61..(4), newValue=[B0x42656c61..(4), metadata=MimeMetadata(text/plain; charset=UTF-8,-1,SECONDS,-1,SECONDS,null,null), flags=null, successful=true, ignorePreviousValue=false}}: java.lang.NullPointerException
at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:880)
at org.infinispan.CacheImpl.replace(CacheImpl.java:871)
at org.infinispan.DecoratedCache.replace(DecoratedCache.java:551)
at org.infinispan.xsite.BackupReceiverImpl$BackupCacheUpdater.visitReplaceCommand(BackupReceiverImpl.java:113)
at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:76)
at org.infinispan.xsite.BackupReceiverImpl.handleRemoteCommand(BackupReceiverImpl.java:76)
at org.infinispan.xsite.BackupReceiverRepositoryImpl.handleRemoteCommand(BackupReceiverRepositoryImpl.java:87)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromRemoteSite(CommandAwareRpcDispatcher.java:255)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:230)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:460) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:377) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:665) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.JChannel.up(JChannel.java:719) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1008) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.protocols.relay.RELAY2.deliver(RELAY2.java:607) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.protocols.relay.RELAY2.route(RELAY2.java:507) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.protocols.relay.RELAY2.handleMessage(RELAY2.java:482) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.protocols.relay.RELAY2.handleRelayMessage(RELAY2.java:463) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.protocols.relay.Relayer$Bridge.receive(Relayer.java:302) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.JChannel.up(JChannel.java:749) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1012) [jgroups-3.3.0.CR2.jar:3.3.0.CR2]
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 10 months
[JBoss JIRA] (ISPN-3140) JMX operation to suppress state transfer
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3140?page=com.atlassian.jira.plugin.... ]
Mircea Markus commented on ISPN-3140:
-------------------------------------
where would the data in this cluster be persisted during the shutdown? simpler with a shared cache store, each cache persisting locally would complicate things a bit.
> JMX operation to suppress state transfer
> ----------------------------------------
>
> Key: ISPN-3140
> URL: https://issues.jboss.org/browse/ISPN-3140
> Project: Infinispan
> Issue Type: Feature Request
> Components: Distributed Cache, State transfer
> Affects Versions: 5.2.6.Final
> Reporter: Manik Surtani
> Assignee: Mircea Markus
> Fix For: 5.3.0.CR1, 5.3.0.Final
>
>
> This feature request is to expose a JMX operation on each node, to suppress state transfer for a period of time. This flag would be {{false}} by default.
> The use case of this flag would be to ease bringing down (and up) a cluster for maintenance work. A typical workflow would be:
> 1) Shut down application requests to the data grid
> 2) Suppress state transfer on all nodes via JMX
> 3) Bring down all nodes
> 4) Perform maintenance work
> 5) Bring up nodes, one at a time. As each node comes up, disable state transfer for the node via JMX.
> 6) Once all nodes are up, enable state transfer for each node again via JMX
> 7) Allow application requests to reach the grid again.
> The purpose of this is to allow smooth and fast shutdown and startup, remove the risk of OOM errors (when bringing a grid down).
> This is a small but useful subset of full manual state transfer as defined in ISPN-1394.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 10 months
[JBoss JIRA] (ISPN-3140) JMX operation to suppress state transfer
by Manik Surtani (JIRA)
Manik Surtani created ISPN-3140:
-----------------------------------
Summary: JMX operation to suppress state transfer
Key: ISPN-3140
URL: https://issues.jboss.org/browse/ISPN-3140
Project: Infinispan
Issue Type: Feature Request
Components: Distributed Cache, State transfer
Affects Versions: 5.2.6.Final
Reporter: Manik Surtani
Assignee: Mircea Markus
Fix For: 5.3.0.CR1, 5.3.0.Final
This feature request is to expose a JMX operation on each node, to suppress state transfer for a period of time. This flag would be {{false}} by default.
The use case of this flag would be to ease bringing down (and up) a cluster for maintenance work. A typical workflow would be:
1) Shut down application requests to the data grid
2) Suppress state transfer on all nodes via JMX
3) Bring down all nodes
4) Perform maintenance work
5) Bring up nodes, one at a time. As each node comes up, disable state transfer for the node via JMX.
6) Once all nodes are up, enable state transfer for each node again via JMX
7) Allow application requests to reach the grid again.
The purpose of this is to allow smooth and fast shutdown and startup, remove the risk of OOM errors (when bringing a grid down).
This is a small but useful subset of full manual state transfer as defined in ISPN-1394.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 10 months
[JBoss JIRA] (ISPN-3137) Intermittent failure XMLConfigurationOverridingTest
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-3137?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-3137:
-----------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Intermittent failure XMLConfigurationOverridingTest
> ---------------------------------------------------
>
> Key: ISPN-3137
> URL: https://issues.jboss.org/browse/ISPN-3137
> Project: Infinispan
> Issue Type: Enhancement
> Reporter: Mircea Markus
> Assignee: Mircea Markus
> Fix For: 5.2.0.CR1
>
>
> {quote}
> java.lang.AssertionError: expected [true] but found [false]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertTrue(Assert.java:42)
> at org.testng.Assert.assertTrue(Assert.java:52)
> at org.infinispan.test.AbstractInfinispanTest.eventually(AbstractInfinispanTest.java:82)
> at org.infinispan.test.AbstractInfinispanTest.eventually(AbstractInfinispanTest.java:65)
> at org.infinispan.configuration.override.XMLConfigurationOverridingTest.access$100(XMLConfigurationOverridingTest.java:58)
> at org.infinispan.configuration.override.XMLConfigurationOverridingTest$3.call(XMLConfigurationOverridingTest.java:143)
> at org.infinispan.test.TestingUtil.withCacheManager(TestingUtil.java:1234)
> at org.infinispan.configuration.override.XMLConfigurationOverridingTest.testLocalCacheOverrideWithConfiguredExpiration(XMLConfigurationOverridingTest.java:116)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 10 months
[JBoss JIRA] (ISPN-3133) AsyncAPINonTxSyncDistTest and AsyncAPINonTxAsyncDistTest are mistakenly executed in REPL mode
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-3133?page=com.atlassian.jira.plugin.... ]
Adrian Nistor commented on ISPN-3133:
-------------------------------------
The DIST test was failing because it was never meant to run with the non-tx concurrent interceptor, but this became the default when ISPN-3106 was resolved and no failure was noticed because this test was basically muted. The failure shows 3 things are wrong. First thing we need to allow extra serializations in these tests whenever forward to primary owner is done. The second thing is to fix BaseDistributionInterceptor to avoid forwarding unsuccessful conditional commands. Third, always return reliable values even if the call is async and requires forwarding to primary. We need to wait for primary owner's response by making the call to primary synchronously (the call from primary to backups is still done async).
> AsyncAPINonTxSyncDistTest and AsyncAPINonTxAsyncDistTest are mistakenly executed in REPL mode
> ---------------------------------------------------------------------------------------------
>
> Key: ISPN-3133
> URL: https://issues.jboss.org/browse/ISPN-3133
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite
> Affects Versions: 5.2.5.Final
> Reporter: Adrian Nistor
> Assignee: Mircea Markus
> Fix For: 5.3.0.Final
>
>
> The test does not properly override method getConfig() from parent due to different signature and fails to configure DIST mode. As a consequence it runs in REPL mode and appears to work. Trying to fix the signature of getConfig() reveals that the test actually fails in DIST mode:
> {code}
> [testng-AsyncAPINonTxSyncDistTest] Test testAsyncMethods(org.infinispan.distribution.AsyncAPINonTxSyncDistTest) failed.
> Test suite progress: tests succeeded: 0, failed: 1, skipped: 0.
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1,232.723 sec <<< FAILURE!
> testAsyncMethods(org.infinispan.distribution.AsyncAPINonTxSyncDistTest) Time elapsed: 1,230.712 sec <<< FAILURE!
> java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.RuntimeException: Failure to marshal argument(s)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.infinispan.util.concurrent.NotifyingFutureAdaptor.get(NotifyingFutureAdaptor.java:63)
> at org.infinispan.replication.AsyncAPINonTxSyncReplTest.testAsyncMethods(AsyncAPINonTxSyncReplTest.java:110)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.infinispan.CacheException: java.lang.RuntimeException: Failure to marshal argument(s)
> at org.infinispan.util.Util.rewrapAsCacheException(Util.java:588)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:197)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:532)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:302)
> at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.handleNonTxWriteCommand(BaseDistributionInterceptor.java:150)
> at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutKeyValueCommand(NonTxDistributionInterceptor.java:91)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.distribution.L1NonTxInterceptor.handleDataWriteCommand(L1NonTxInterceptor.java:152)
> at org.infinispan.interceptors.distribution.L1NonTxInterceptor.visitPutKeyValueCommand(L1NonTxInterceptor.java:108)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:290)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:157)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:68)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1337)
> at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:922)
> at org.infinispan.CacheImpl.access$500(CacheImpl.java:120)
> at org.infinispan.CacheImpl$5.call(CacheImpl.java:1114)
> ... 5 more
> Caused by: java.lang.RuntimeException: Failure to marshal argument(s)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:352)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:371)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:188)
> ... 44 more
> Caused by: java.lang.RuntimeException: Cannot serialize!!
> at org.infinispan.test.data.Key.writeExternal(Key.java:70)
> at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:873)
> at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
> at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119)
> at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:79)
> at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeObject(ReplicableCommandExternalizer.java:64)
> at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeObject(ReplicableCommandExternalizer.java:52)
> at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:415)
> at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
> at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
> at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119)
> at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:79)
> at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.marshallParameters(CacheRpcCommandExternalizer.java:135)
> at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:119)
> at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:78)
> at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:415)
> at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
> at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
> at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119)
> at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectToObjectStream(AbstractJBossMarshaller.java:96)
> at org.infinispan.marshall.VersionAwareMarshaller.objectToBuffer(VersionAwareMarshaller.java:92)
> at org.infinispan.marshall.AbstractMarshaller.objectToBuffer(AbstractMarshaller.java:64)
> at org.infinispan.marshall.AbstractDelegatingMarshaller.objectToBuffer(AbstractDelegatingMarshaller.java:109)
> at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectToBuffer(MarshallerAdapter.java:45)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:350)
> ... 46 more
> Caused by: an exception which occurred:
> in object org.infinispan.test.data.Key@6b
> in object org.infinispan.commands.write.PutKeyValueCommand@670ad2
> in object org.infinispan.commands.remote.SingleRpcCommand@338842f2
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 10 months
[JBoss JIRA] (ISPN-3133) AsyncAPINonTxSyncDistTest and AsyncAPINonTxAsyncDistTest are mistakenly executed in REPL mode
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-3133?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-3133:
--------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/1833
> AsyncAPINonTxSyncDistTest and AsyncAPINonTxAsyncDistTest are mistakenly executed in REPL mode
> ---------------------------------------------------------------------------------------------
>
> Key: ISPN-3133
> URL: https://issues.jboss.org/browse/ISPN-3133
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite
> Affects Versions: 5.2.5.Final
> Reporter: Adrian Nistor
> Assignee: Mircea Markus
> Fix For: 5.3.0.Final
>
>
> The test does not properly override method getConfig() from parent due to different signature and fails to configure DIST mode. As a consequence it runs in REPL mode and appears to work. Trying to fix the signature of getConfig() reveals that the test actually fails in DIST mode:
> {code}
> [testng-AsyncAPINonTxSyncDistTest] Test testAsyncMethods(org.infinispan.distribution.AsyncAPINonTxSyncDistTest) failed.
> Test suite progress: tests succeeded: 0, failed: 1, skipped: 0.
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1,232.723 sec <<< FAILURE!
> testAsyncMethods(org.infinispan.distribution.AsyncAPINonTxSyncDistTest) Time elapsed: 1,230.712 sec <<< FAILURE!
> java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.RuntimeException: Failure to marshal argument(s)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.infinispan.util.concurrent.NotifyingFutureAdaptor.get(NotifyingFutureAdaptor.java:63)
> at org.infinispan.replication.AsyncAPINonTxSyncReplTest.testAsyncMethods(AsyncAPINonTxSyncReplTest.java:110)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.infinispan.CacheException: java.lang.RuntimeException: Failure to marshal argument(s)
> at org.infinispan.util.Util.rewrapAsCacheException(Util.java:588)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:197)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:532)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:302)
> at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.handleNonTxWriteCommand(BaseDistributionInterceptor.java:150)
> at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutKeyValueCommand(NonTxDistributionInterceptor.java:91)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.distribution.L1NonTxInterceptor.handleDataWriteCommand(L1NonTxInterceptor.java:152)
> at org.infinispan.interceptors.distribution.L1NonTxInterceptor.visitPutKeyValueCommand(L1NonTxInterceptor.java:108)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:290)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:157)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:68)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1337)
> at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:922)
> at org.infinispan.CacheImpl.access$500(CacheImpl.java:120)
> at org.infinispan.CacheImpl$5.call(CacheImpl.java:1114)
> ... 5 more
> Caused by: java.lang.RuntimeException: Failure to marshal argument(s)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:352)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:371)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:188)
> ... 44 more
> Caused by: java.lang.RuntimeException: Cannot serialize!!
> at org.infinispan.test.data.Key.writeExternal(Key.java:70)
> at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:873)
> at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
> at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119)
> at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:79)
> at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeObject(ReplicableCommandExternalizer.java:64)
> at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeObject(ReplicableCommandExternalizer.java:52)
> at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:415)
> at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
> at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
> at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119)
> at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:79)
> at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.marshallParameters(CacheRpcCommandExternalizer.java:135)
> at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:119)
> at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:78)
> at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:415)
> at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
> at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
> at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119)
> at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectToObjectStream(AbstractJBossMarshaller.java:96)
> at org.infinispan.marshall.VersionAwareMarshaller.objectToBuffer(VersionAwareMarshaller.java:92)
> at org.infinispan.marshall.AbstractMarshaller.objectToBuffer(AbstractMarshaller.java:64)
> at org.infinispan.marshall.AbstractDelegatingMarshaller.objectToBuffer(AbstractDelegatingMarshaller.java:109)
> at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectToBuffer(MarshallerAdapter.java:45)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:350)
> ... 46 more
> Caused by: an exception which occurred:
> in object org.infinispan.test.data.Key@6b
> in object org.infinispan.commands.write.PutKeyValueCommand@670ad2
> in object org.infinispan.commands.remote.SingleRpcCommand@338842f2
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 10 months
[JBoss JIRA] (ISPN-3120) StateConsumerImpl can ignore state received during a rebalance
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-3120?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-3120:
------------------------------------
My fix was to only set the waitingForState flag in StateConsumerImpl after all the segments have been requested. But it turns out there's a way around that: if a node E requests some segments from another node H, which then dies, the StateConsumerImpl on node E will first remove the transfers from H, and then request the segments from another node F. During this time, the waitingForState flag will remain true, so it's possible to replicate the bug even with my fix.
1. Starting with a 4-node cluster: [E, F, G, H] (topology 6), with numOwners=3.
2. G leaves, and E sends a REBALANCE_START command with nodes [E, F, H] (topology 8). Some segments are owned by [F, H] in the current CH and by [F, H, E] in the pending CH.
3. E requests some segments from F and some segments from H.
4. H leaves, and E sends a CH_UPDATE command with nodes [E, F] (topology 9).
5. The StateConsumerImpl on E removes the transfers from H.
6. E receives a StateResponseCommand from F. Because it just removed the transfers from H, transfersBySegment/transfersBySource maps will remain empty and E will consider it finished receiving state for topology 9.
7. The StateConsumerImpl on E requests the failed segments from F (via a separate StateRequestCommand).
10. E receives a StateResponseCommand from F with actual data, but it ignores it because StateConsumerImpl.updatedKeys == null.
{noformat}
2013-05-23 16:59:59,589 TRACE (transport-thread-2,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Requesting transactions for segments [51, 50, 35, 33, 54, 39, 53, 52, 37, 43, 41] of cache dist from node NodeH-13469
2013-05-23 16:59:59,619 TRACE (transport-thread-2,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Requesting transactions for segments [49, 55, 57, 56] of cache dist from node NodeF-2452
2013-05-23 16:59:59,623 TRACE (transport-thread-2,NodeE) [org.infinispan.statetransfer.InboundTransferTask] Requesting segments [51, 50, 35, 33, 54, 39, 53, 52, 37, 43, 41] of cache dist from node NodeH-13469
2013-05-23 16:59:59,617 TRACE (transport-thread-4,NodeE) [org.infinispan.topology.ClusterTopologyManagerImpl] Attempting to execute command on self: CacheTopologyControlCommand{cache=dist, type=CH_UPDATE, sender=NodeE-18419, joinInfo=null, topologyId=9, currentCH=DefaultConsistentHash{numSegments=60, numOwners=3, members=[NodeE-18419, NodeF-2452]}, pendingCH=DefaultConsistentHash{numSegments=60, numOwners=3, members=[NodeE-18419, NodeF-2452]}, throwable=null, viewId=3}
2013-05-23 16:59:59,624 TRACE (transport-thread-4,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Removing inbound transfers for segments [51, 50, 35, 33, 54, 39, 53, 52, 37, 43, 41] from source NodeH-13469 for cache dist
2013-05-23 16:59:59,624 TRACE (transport-thread-4,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Requesting transactions for segments [51, 50, 35, 33, 54, 39, 53, 52, 37, 43, 41] of cache dist from node NodeF-2452
2013-05-23 16:59:59,624 WARN (transport-thread-2,NodeE) [org.infinispan.statetransfer.InboundTransferTask] ISPN000210: Failed to request segments [51, 50, 35, 33, 54, 39, 53, 52, 37, 43, 41] of cache dist from node NodeH-13469 (node will not be retried)
2013-05-23 16:59:59,624 TRACE (transport-thread-2,NodeE) [org.infinispan.statetransfer.InboundTransferTask] Requesting segments [49, 55, 57, 56] of cache dist from node NodeF-2452
2013-05-23 16:59:59,632 TRACE (transport-thread-2,NodeE) [org.infinispan.statetransfer.InboundTransferTask] Successfully requested segments [49, 55, 57, 56] of cache dist from node NodeF-2452
2013-05-23 16:59:59,634 TRACE (remote-thread-1,NodeE) [org.infinispan.statetransfer.InboundTransferTask] Finished receiving state for segments [49, 55, 57, 56] of cache dist
2013-05-23 16:59:59,634 TRACE (remote-thread-1,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Completion of inbound transfer task: InboundTransferTask{segments=[49, 55, 57, 56], finishedSegments=[49, 55, 57, 56], unfinishedSegments=[], source=NodeF-2452, isCancelled=false, isStartedSuccessfully=true, isCompletedSuccessfully=true, topologyId=8, timeout=240000, cacheName=dist}
2013-05-23 16:59:59,634 DEBUG (remote-thread-1,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Finished receiving of segments for cache dist for topology 9.
2013-05-23 16:59:59,634 TRACE (remote-thread-1,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Stop keeping track of changed keys for state transfer
2013-05-23 16:59:59,652 DEBUG (transport-thread-4,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Finished adding inbound state transfer for segments [35, 33, 39, 37, 43, 41, 51, 50, 54, 53, 52] of cache dist
2013-05-23 16:59:59,652 TRACE (transport-thread-2,NodeE) [org.infinispan.statetransfer.InboundTransferTask] Requesting segments [51, 50, 35, 33, 54, 39, 53, 52, 37, 43, 41] of cache dist from node NodeF-2452
2013-05-23 16:59:59,652 TRACE (transport-thread-4,NodeE) [org.infinispan.statetransfer.StateTransferLockImpl] Signalling transaction data received for topology 9
2013-05-23 16:59:59,661 TRACE (transport-thread-2,NodeE) [org.infinispan.statetransfer.InboundTransferTask] Successfully requested segments [51, 50, 35, 33, 54, 39, 53, 52, 37, 43, 41] of cache dist from node NodeF-2452
2013-05-23 16:59:59,767 DEBUG (remote-thread-1,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Applying new state for segment 43 of cache dist from node NodeF-2452: received 1 cache entries
2013-05-23 16:59:59,767 TRACE (remote-thread-1,NodeE) [org.infinispan.statetransfer.StateConsumerImpl] Received keys [MagicKey#k2{cd331f73@NodeF-2452}] for segment 43 of cache dist from node NodeF-2452
2013-05-23 16:59:59,768 TRACE (remote-thread-1,NodeE) [org.infinispan.interceptors.EntryWrappingInterceptor] State transfer will not write key/value MagicKey#k2{cd331f73@NodeF-2452}/v2 because it was already updated by somebody else
{noformat}
> StateConsumerImpl can ignore state received during a rebalance
> --------------------------------------------------------------
>
> Key: ISPN-3120
> URL: https://issues.jboss.org/browse/ISPN-3120
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer
> Affects Versions: 5.3.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Labels: 5.2.x
> Fix For: 5.2.7.Final, 5.3.0.CR1, 5.3.0.Final
>
>
> This causes random failures in ConcurrentOverlappingLeaveTest and ConcurrentNonOverlappingLeaveTest.
> 1. Starting with a 4-node cluster: [E, F, G, H] (topology 7).
> 2. F leaves, and E sends a REBALANCE_START command with nodes [E, G, H] (topology 8). Some segments are owned by [H] in the current CH and by [H, G] in the pending CH.
> 3. E reports that it finished receiving state with a REBAlANCE_CONFIRM command.
> 4. H leaves, and E sends a CH_UPDATE command with nodes [E, G] (topology 9).
> The segments that were owned by [H] in the previous currentCH are assigned to [E, G] in the new currentCH (otherwise they wouldn't have any owners).
> 5. The StateConsumerImpl on E requests state for the "lost" segments from G.
> 6. G confirms the end of the rebalance as well, and E sends a CH_UPDATE command to end the rebalance (topology 10).
> 7. E sends a REBALANCE_START command to assign all segments for [E, G] (topology 11).
> 8. While the StateConsumerImpl on E is starting the state transfer, it also receives a StateResponseCommand for the lost segments from G.
> 9. Because the structures keeping track of the received state are not properly initialized, E considers it finished receiving state for topology 11.
> 10. E receives a StateResponseCommand from G with actual data, but it ignores it because {{StateConsumerImpl.updatedKeys == null}}.
> {noformat}
> 11:30:39,807 DEBUG (transport-thread-4,NodeE:dist) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache dist: new topology = CacheTopology{id=7, currentCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeE-51027, NodeG-6339, NodeH-47370]}, pendingCH=null}
> 11:30:39,810 DEBUG (transport-thread-3,NodeE:dist) [LocalTopologyManagerImpl] Starting local rebalance for cache dist, topology = CacheTopology{id=8, currentCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeE-51027, NodeG-6339, NodeH-47370]}, pendingCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeE-51027, NodeG-6339, NodeH-47370]}}
> 11:30:39,817 DEBUG (transport-thread-3,NodeE:dist) [StateConsumerImpl] Finished receiving of segments for cache dist for topology 8.
> 11:30:39,832 DEBUG (transport-thread-4,NodeE:dist) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache dist: new topology = CacheTopology{id=9, currentCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeE-51027, NodeG-6339]}, pendingCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeE-51027, NodeG-6339]}}
> 11:30:39,834 DEBUG (transport-thread-4,NodeE:dist) [StateConsumerImpl] Adding inbound state transfer for segments [38, 36, 47, 44, 45] of cache dist
> 11:30:39,853 DEBUG (transport-thread-3,NodeE:dist) [LocalTopologyManagerImpl] Starting local rebalance for cache dist, topology = CacheTopology{id=11, currentCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeE-51027, NodeG-6339]}, pendingCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeE-51027, NodeG-6339]}}
> 11:30:39,859 TRACE (remote-thread-1,NodeE:) [InboundInvocationHandlerImpl] Calling perform() on StateResponseCommand{cache=dist, origin=NodeG-6339, topologyId=9}
> 11:30:39,864 DEBUG (remote-thread-1,NodeE:dist) [StateConsumerImpl] Finished receiving of segments for cache dist for topology 11.
> 11:30:39,866 TRACE (transport-thread-5,NodeE:dist) [LocalTopologyManagerImpl] Ignoring consistent hash update 10 for cache dist, we have already received a newer topology 11
> 11:30:39,868 TRACE (remote-thread-1,NodeE:) [InboundInvocationHandlerImpl] Calling perform() on StateResponseCommand{cache=dist, origin=NodeG-6339, topologyId=11}
> 11:30:39,872 TRACE (remote-thread-1,NodeE:dist dist) [EntryWrappingInterceptor] State transfer will not write key/value MagicKey#k3{672f69c9@NodeG-6339}/v3 because it was already updated by somebody else
> 11:30:40,582 ERROR (testng-ConcurrentNonOverlappingLeaveTest:) [UnitTestTestNGListener] Test testTransactional(org.infinispan.distribution.rehash.ConcurrentNonOverlappingLeaveTest) failed.
> java.lang.AssertionError: Fail on owner cache NodeE-51027: dc.get(MagicKey#k3{672f69c9@NodeG-6339}) returned null!
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 10 months