[JBoss JIRA] (ISPN-2255) FineGrainedAtomicMap missing key, value pairs in some cluster nodes in distributed mode, embedded infinispan cache
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2255?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2255:
--------------------------------
Fix Version/s: 5.2.0.Final
(was: 5.2.0.CR1)
> FineGrainedAtomicMap missing key,value pairs in some cluster nodes in distributed mode, embedded infinispan cache
> -----------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-2255
> URL: https://issues.jboss.org/browse/ISPN-2255
> Project: Infinispan
> Issue Type: Bug
> Components: Fine-grained API
> Affects Versions: 5.1.5.FINAL
> Environment: Details about cluster:
> 1. Infinispan 5.1.5
> 2. java 7u5 64bit
> 3. linux 64bit (two nodes debian and one node ubuntu)
> 4. tomcat 7.0.28
> 5. jbossTM 4.16.0 JTA only
> Tomcat integration code is here:
> https://github.com/zvrablik/tomcatInfinispanSessionManager/tree/master/to...
> 6. all machines are on real hw ( no virtual machines), date and time is synchronized and shouldn't vary more than 1 second
> Reporter: Zdenek Henek
> Assignee: Vladimir Blagojevic
> Fix For: 5.2.0.Final
>
> Attachments: fgamissueTest2.zip, jgroupsConfig.xml, sessionInfinispanConfigtestLB.xml
>
>
> I am using FineGrainedAtomicMap to store data into clustering distributed cache. When clustering set to replicated doesn't cause any issues or at least I haven't run
> into any. When I switch to distributed mode I don't see some key,value pairs in some nodes. This happens randomly. The numOwners is set to two and I have cluster of three nodes.
> more details:
> https://community.jboss.org/thread/203420?tstart=60
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 12 months
[JBoss JIRA] (ISPN-2578) Two PrepareCommands in parallel cause ConcurrentModificationException
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2578?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2578:
--------------------------------
Fix Version/s: 5.2.0.Final
(was: 5.2.0.CR1)
> Two PrepareCommands in parallel cause ConcurrentModificationException
> ---------------------------------------------------------------------
>
> Key: ISPN-2578
> URL: https://issues.jboss.org/browse/ISPN-2578
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.0.Beta5
> Reporter: Radim Vansa
> Assignee: Adrian Nistor
> Priority: Critical
> Fix For: 5.2.0.Final
>
>
> Situation:
> 1) Node A broadcasts PrepareCommand to nodes B, C
> 2) Node A leaves cluster, causing new topology to be installed
> 3) The command arrives to B and C, with lower topology than the current one
> 4) Both B and C forward the command to node D
> 5) D executes the two commands in parallel and finds out that A has left, therefore executing RollbackCommand
> In {{AbstractTxLockingInterceptor.visitRollbackCommand}} we call {{LockManagerImpl.unlockAll}} which iterates over the keys and unlocks them. As these two prepares aren't synchronized over the {{lockedKeys}} set, one may unlock and remove these keys while the other is iterating through them, causing {{ConcurrentModificationException}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 12 months
[JBoss JIRA] (ISPN-2575) KeyTransformer registration is required on all nodes of the cluster, in case of custom keys
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2575?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2575:
--------------------------------
Fix Version/s: 5.2.0.Final
(was: 5.2.0.CR1)
> KeyTransformer registration is required on all nodes of the cluster, in case of custom keys
> -------------------------------------------------------------------------------------------
>
> Key: ISPN-2575
> URL: https://issues.jboss.org/browse/ISPN-2575
> Project: Infinispan
> Issue Type: Feature Request
> Components: Querying
> Affects Versions: 5.2.0.Beta5
> Reporter: Anna Manukyan
> Assignee: Sanne Grinovero
> Fix For: 5.2.0.Final
>
> Attachments: ClusteredCacheTest.java
>
>
> The case is the following:
> I have a clustered cache on which I want to perform a search. I'm doing the following:
> I'm initializing SearchManager on node1, I'm registering a custom key transformer for my key using the created searchmanager, but then when I'm trying to put data into the cache on node1 (which is in REPL_SYNC mode with cache on node2), I'm getting the exception:
> java.lang.IllegalArgumentException: Indexing only works with entries keyed on Strings, primitives and classes that have the @Transformable annotation - you passed in a class org.infinispan.query.test.CustomKey3. Alternatively, see org.infinispan.query.SearchManager#registerKeyTransformer
> When I'm initializing the SearchManager using node2 cache and register the keyTransformer on it as well, then everything works perfectly, even though I am not using the second created SearchManager.
> The test which reproduces the issue is attached to the jira. Please see the test case: testSearchKeyTransformer()
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 12 months
[JBoss JIRA] (ISPN-2483) State transfer issue with the transactions for which the originator has crashed
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2483?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2483:
--------------------------------
Fix Version/s: (was: 5.2.0.CR1)
> State transfer issue with the transactions for which the originator has crashed
> -------------------------------------------------------------------------------
>
> Key: ISPN-2483
> URL: https://issues.jboss.org/browse/ISPN-2483
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer, Transactions
> Affects Versions: 5.1.8.Final, 5.2.0.Beta3
> Reporter: Mircea Markus
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.0.Final
>
>
> State transfer migrates and prepares the transactions for which the originator has left. On the receiving node, this results in the transaction being prepared and acquiring backup locks which are never released (unless manual intervention).
> This should behave as follows:
> - if there's no recovery enabled, the state producer should not send such transactions but drop them
> - if recovery is enabled these transactions should be sent across. They shouldn't be prepared/acquire backup locks, but be placed in the recovery cache (see RecoveryManagerImpl.inDoubtTransactions)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 12 months
[JBoss JIRA] (ISPN-2439) Deadlock in Map/Reduce tasks
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2439?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2439:
--------------------------------
Fix Version/s: 5.2.0.Final
(was: 5.2.0.CR1)
> Deadlock in Map/Reduce tasks
> ----------------------------
>
> Key: ISPN-2439
> URL: https://issues.jboss.org/browse/ISPN-2439
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 5.2.0.Beta2
> Reporter: Dan Berindei
> Assignee: Mircea Markus
> Priority: Critical
> Fix For: 5.2.0.Final
>
> Attachments: dfnmrt.log.gz
>
>
> It looks like the Map/Reduce intermediate caches use pessimistic transactions, but the transactions are not guaranteed to write to the keys in the same order. So it's possible for two tasks to get into a deadlock, ending with a TimeoutException:
> {noformat}
> 16:18:40,649 ERROR (testng-DistributedFourNodesMapReduceTest:) [UnitTestTestNGListener] Test testCombinerDoesNotChangeResult(org.infinispan.distexec.mapreduce.DistributedFourNodesMapReduceTest) failed.
> org.infinispan.CacheException: Could not invoke map phase of MapReduce task on remote nodes
> at org.infinispan.distexec.mapreduce.MapReduceTask.invokeEverywhere(MapReduceTask.java:562)
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhase(MapReduceTask.java:374)
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:315)
> at org.infinispan.distexec.mapreduce.BaseWordCountMapReduceTest.testCombinerDoesNotChangeResult(BaseWordCountMapReduceTest.java:188)
> ...
> Caused by: org.infinispan.CacheException: org.infinispan.CacheException: Could not move intermediate keys/values for M/R task 04244b4b-08b1-4fc4-9755-ed02f3f35a3a
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:97)
> at org.infinispan.commands.read.MapCombineCommand.perform(MapCombineCommand.java:89)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:95)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:110)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:82)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:244)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:217)
> at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:483)
> ...
> Caused by: org.infinispan.CacheException: Could not move intermediate keys/values for M/R task 04244b4b-08b1-4fc4-9755-ed02f3f35a3a
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:281)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:95)
> ... 26 more
> Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [JBoss] for requestor [GlobalTransaction:<NodeD-56763>:10429:remote]! Lock held by [GlobalTransaction:<NodeB-55590>:10432:remote]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:217)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:190)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:190)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:125)
> at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:248)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
> at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:177)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:125)
> at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:174)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:212)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:187)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:131)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:129)
> at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:98)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:347)
> at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:150)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:95)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleWithWaitForBlocks(InboundInvocationHandlerImpl.java:110)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:82)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:244)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:217)
> at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:483)
> ...
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 12 months