[JBoss JIRA] (ISPN-2549) Invalidation mode caches with pessimistic transactions may result in stale remote locks
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2549?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2549:
--------------------------------
Fix Version/s: 5.2.1
5.3.0.Final
> Invalidation mode caches with pessimistic transactions may result in stale remote locks
> ---------------------------------------------------------------------------------------
>
> Key: ISPN-2549
> URL: https://issues.jboss.org/browse/ISPN-2549
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 5.2.0.Beta4
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Fix For: 5.2.1, 5.3.0.Final
>
> Attachments: InvalidationModePessimisticLockingIssueTest.java
>
>
> An invalidation mode cluster with two nodes A and B. Node A is coordinator.
> Putting a key on node B results in a remote lock being acquired on A. But the lock on A is not released when the TX commits. Subsequent attempts to lock the key on A will fail with TimeoutException.
> The key aspects to reproduce this are:
> 1. pessimistic locking
> 2. the transaction starts on a node different than the coordinator
> This scenario does not involve any state transfer, so it should not be related to the already known issues with stale locks caused by state-transfer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2103) Concurrent access after removal of an AtomicMap should NOT result in an IllegalStateException when accessed by other threads
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2103?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2103:
--------------------------------
Fix Version/s: 5.2.1
5.3.0.Final
> Concurrent access after removal of an AtomicMap should NOT result in an IllegalStateException when accessed by other threads
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-2103
> URL: https://issues.jboss.org/browse/ISPN-2103
> Project: Infinispan
> Issue Type: Bug
> Components: Core API
> Affects Versions: 5.1.5.FINAL
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Fix For: 5.2.1, 5.3.0.Final
>
>
> ISPN-1121 introduces an IllegalStateException that is being thrown from AtomicMap methods once the map handle has become stale (ie. removed from cache). In case of concurrent access, the exception is thrown to all threads not just to the thread that performed the removal. This was fine-ish in older versions of Infinispan before introduction of optimistic and pessimistic locking but it should be reconsidered now because:
> 1. It interferes/overlaps with transaction isolation. We should rely on the selected locking scheme (OL/PL) to detect conflicts between transactions and report the problem accordingly. Especially if the stale map is used just for reading - this should be allowed to work rather than stop it.
> 2. This exception is pretty disruptive and awkward to handle. All methods of an AtomicMap can result in this exception and the current thread has to be prepared for handling it if other threads remove the map. Not much transaction isolation.
> 3. Since the TreeCache is backed by AtomicMap nearly all Tree API can throw this.
> The proposed fix consists of:
> 1. removing AtomicHashMap.removed flag and AtomicHashMap.markRemoved() method.
> 2. revising AtomicHashMapProxy.assertValid() method to check only if the map is null (ie. removed) but no longer use the removed flag.
> 3. revising ReadCommittedEntry.commit() method to no longer call markRemoved() method.
> The consequences of these changes are:
> 1. Any further access to a stale map results in IllegalStateException ONLY in the thread that performed the removal. This thread 'knows' the map is stale so it is fine to punish it. Other threads remain unaffected until lock acquisition or commit is performed (depending on locking model).
> 2. Other threads can continue to use the previously obtained map handle for reads without danger of getting an exception.
> 3. If a write operation is done on the map, the results depend on the locking model:
> 3.1 optimistic locking + write skew check: a WriteSkewException will stop the commit during prepare
> 3.2 optimistic locking, no write skew check: the write is committed and the work of the transaction that removed the map is overwritten. The map is effectively revived.
> 3.3 pessimistic locking: same as 3.2
> Please note 3.2 and 3.3 work the same as for normal values (not atomic maps).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-976) TopologyAwareConsistentHash does not work when numOwners=1 and rehashing
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-976?page=com.atlassian.jira.plugin.s... ]
Mircea Markus updated ISPN-976:
-------------------------------
Assignee: Dan Berindei (was: Mircea Markus)
> TopologyAwareConsistentHash does not work when numOwners=1 and rehashing
> -------------------------------------------------------------------------
>
> Key: ISPN-976
> URL: https://issues.jboss.org/browse/ISPN-976
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 4.2.1.FINAL
> Reporter: Mircea Markus
> Assignee: Dan Berindei
> Priority: Critical
>
> Specifically, the problem is with getStateProvidersOnLeave.
>
> Assume you have nodes 1, 2, and 4.
> Now node 3 joins.
> This method will iterate over each address and will skip the joiner (node3).
> For the remaining nodes it will call getOwners
> e.g. - getOwners(node1), getOwners(node2), getOwners(node3)
> This method will always begin by adding the first node in the tailMap which is the address that is being iterated over. Since numOwners is set to 1 it skips the while loop and returns the result.
> So...
> getOwners(1) returns {node1}
> getOwners(2) returns {node2}
> getOwners(4) returns {node4}
> Now we return to getStateProvidersOnLeave
> It now checks to see if the result contains the joiner (node3). However, it can never contain the joiner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-976) TopologyAwareConsistentHash does not work when numOwners=1 and rehashing
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-976?page=com.atlassian.jira.plugin.s... ]
Mircea Markus closed ISPN-976.
------------------------------
Resolution: Rejected
as per the last comment this is oly valid in the scope of 4.2
> TopologyAwareConsistentHash does not work when numOwners=1 and rehashing
> -------------------------------------------------------------------------
>
> Key: ISPN-976
> URL: https://issues.jboss.org/browse/ISPN-976
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 4.2.1.FINAL
> Reporter: Mircea Markus
> Assignee: Dan Berindei
> Priority: Critical
>
> Specifically, the problem is with getStateProvidersOnLeave.
>
> Assume you have nodes 1, 2, and 4.
> Now node 3 joins.
> This method will iterate over each address and will skip the joiner (node3).
> For the remaining nodes it will call getOwners
> e.g. - getOwners(node1), getOwners(node2), getOwners(node3)
> This method will always begin by adding the first node in the tailMap which is the address that is being iterated over. Since numOwners is set to 1 it skips the while loop and returns the result.
> So...
> getOwners(1) returns {node1}
> getOwners(2) returns {node2}
> getOwners(4) returns {node4}
> Now we return to getStateProvidersOnLeave
> It now checks to see if the result contains the joiner (node3). However, it can never contain the joiner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2772) Implement REPLICATED mode as a degenerated DISTRIBUTED mode (nowOwners>=clusterSize)
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2772?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2772:
--------------------------------
Priority: Critical (was: Major)
> Implement REPLICATED mode as a degenerated DISTRIBUTED mode (nowOwners>=clusterSize)
> ------------------------------------------------------------------------------------
>
> Key: ISPN-2772
> URL: https://issues.jboss.org/browse/ISPN-2772
> Project: Infinispan
> Issue Type: Feature Request
> Affects Versions: 5.2.0.Final
> Reporter: Mircea Markus
> Assignee: Adrian Nistor
> Priority: Critical
> Fix For: 5.3.0.Final
>
>
> This has already been done in the case of state transfer, where the distribution state transfer code is reused for replicated caches as well.
> The main reason behind this improvement is to simplify/reduce the code. Also there will be some additional benefits:
> - ATM in replicated mode, the JGroups coordinator always plays the role of main lock owner. The coordinator might get overwhelmed as it has to process the additional TxCompletionNotificationCommand on every transaction (direct consequence of being main lock owner). OTOH in distributed mode, the lock owner is spread between the cluster members.
> As an optimisation, on REPL mode, we can use multicasting (when on UDP) for message sending.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-777) Race conditions in cleaning up stale locks held by dead members in a cluster
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-777?page=com.atlassian.jira.plugin.s... ]
Mircea Markus updated ISPN-777:
-------------------------------
Assignee: Mircea Markus (was: Manik Surtani)
> Race conditions in cleaning up stale locks held by dead members in a cluster
> ----------------------------------------------------------------------------
>
> Key: ISPN-777
> URL: https://issues.jboss.org/browse/ISPN-777
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 4.2.0.BETA1
> Reporter: Vladimir Blagojevic
> Assignee: Mircea Markus
> Priority: Critical
> Fix For: 4.2.0.Final
>
> Attachments: CacheScheduledCounter.java, ISPN-777_output.txt
>
>
> It seems that rollback sometimes does not release acquired eager locks. See attached test program and run two JVM instances on the same machine. Program schedules a task to run every 5 seconds. Tasks simply locks a key, gets the value, increases the value and puts it back surrounded by begin/commit/rollback tx boundary.
> Steps to reproduce (keep repeating steps until problem is encountered):
> 1) Kill one running instance.
> 2) Restart it
> See attached example output of a run.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-777) Race conditions in cleaning up stale locks held by dead members in a cluster
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-777?page=com.atlassian.jira.plugin.s... ]
Mircea Markus updated ISPN-777:
-------------------------------
Fix Version/s: 5.2.1
5.3.0.Final
(was: 4.2.0.Final)
> Race conditions in cleaning up stale locks held by dead members in a cluster
> ----------------------------------------------------------------------------
>
> Key: ISPN-777
> URL: https://issues.jboss.org/browse/ISPN-777
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 4.2.0.BETA1
> Reporter: Vladimir Blagojevic
> Assignee: Mircea Markus
> Priority: Critical
> Fix For: 5.2.1, 5.3.0.Final
>
> Attachments: CacheScheduledCounter.java, ISPN-777_output.txt
>
>
> It seems that rollback sometimes does not release acquired eager locks. See attached test program and run two JVM instances on the same machine. Program schedules a task to run every 5 seconds. Tasks simply locks a key, gets the value, increases the value and puts it back surrounded by begin/commit/rollback tx boundary.
> Steps to reproduce (keep repeating steps until problem is encountered):
> 1) Kill one running instance.
> 2) Restart it
> See attached example output of a run.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2798) Test name differs from test class name
by Adrian Nistor (JIRA)
Adrian Nistor created ISPN-2798:
-----------------------------------
Summary: Test name differs from test class name
Key: ISPN-2798
URL: https://issues.jboss.org/browse/ISPN-2798
Project: Infinispan
Issue Type: Bug
Components: Test Suite
Affects Versions: 5.2.0.Final
Reporter: Adrian Nistor
Assignee: Adrian Nistor
Fix For: 5.2.1, 5.3.0.Alpha1
The log from a recent test run on core contains these warnings.
Some of them are actually superfluous because they refer to abstract classes. UnitTestTestNGListener should not print such a warning if the test class is abstract.
{noformat}
2013-02-05 17:18:14,372 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncAPIAsyncReplTest for class AsyncAPITxAsyncReplTest
2013-02-05 17:18:14,373 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncAPISyncReplTest for class AsyncAPITxSyncReplTest
2013-02-05 17:18:14,376 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name SyncReplImplicitLockingTest for class SyncReplPessimisticLockingTest
2013-02-05 17:18:15,351 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name DldEagerLockingReplicationTest for class DldPessimisticLockingReplicationTest
2013-02-05 17:18:15,354 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name Command line test for class BaseDldPessimisticLockingTest
2013-02-05 17:18:15,355 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name DldEagerLockingDistributedTest for class DldPessimisticLockingDistributedTest
2013-02-05 17:18:19,641 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name TransactionReleaseWithAsync2ndPhaseRecoveryTest for class TransactionCleanupWithAsync2ndPhaseRecoveryTest
2013-02-05 17:18:19,930 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name ReplLockReorderingTest for class ReplicatedLockReorderingTest
2013-02-05 17:18:23,524 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name TransactionReleaseWithAsync2ndPhaseTest for class TransactionCleanupWithAsync2ndPhaseTest
2013-02-05 17:18:39,416 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncTest for class AsyncStoreEvictionTest
2013-02-05 17:18:39,472 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name RemovedEntryFoundAsyncStoreTest for class AsyncStoreFunctionalTest
2013-02-05 17:18:52,937 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name InitiatorCrasherBeforePrepareOptimisticTest for class InitiatorCrashOptimisticTest
2013-02-05 17:19:02,368 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name BasicSingleLockRepOptTest for class BasicSingleLockReplOptTest
2013-02-05 17:19:24,012 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name TopologyAwareConsistentHashFactoryTest for class TopologyAwareSyncConsistentHashFactoryTest
2013-02-05 17:19:34,681 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncAPISyncDistTest for class AsyncAPITxSyncDistTest
2013-02-05 17:19:35,098 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncAPIAsyncDistTest for class AsyncAPITxAsyncDistTest
2013-02-05 17:20:44,563 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name TxAndAsyncOpsTest for class TxCacheAndAsyncOpsTest
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2798) Test name differs from test class name
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-2798?page=com.atlassian.jira.plugin.... ]
Work on ISPN-2798 started by Adrian Nistor.
> Test name differs from test class name
> --------------------------------------
>
> Key: ISPN-2798
> URL: https://issues.jboss.org/browse/ISPN-2798
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite
> Affects Versions: 5.2.0.Final
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Fix For: 5.2.1, 5.3.0.Alpha1
>
>
> The log from a recent test run on core contains these warnings.
> Some of them are actually superfluous because they refer to abstract classes. UnitTestTestNGListener should not print such a warning if the test class is abstract.
> {noformat}
> 2013-02-05 17:18:14,372 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncAPIAsyncReplTest for class AsyncAPITxAsyncReplTest
> 2013-02-05 17:18:14,373 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncAPISyncReplTest for class AsyncAPITxSyncReplTest
> 2013-02-05 17:18:14,376 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name SyncReplImplicitLockingTest for class SyncReplPessimisticLockingTest
> 2013-02-05 17:18:15,351 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name DldEagerLockingReplicationTest for class DldPessimisticLockingReplicationTest
> 2013-02-05 17:18:15,354 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name Command line test for class BaseDldPessimisticLockingTest
> 2013-02-05 17:18:15,355 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name DldEagerLockingDistributedTest for class DldPessimisticLockingDistributedTest
> 2013-02-05 17:18:19,641 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name TransactionReleaseWithAsync2ndPhaseRecoveryTest for class TransactionCleanupWithAsync2ndPhaseRecoveryTest
> 2013-02-05 17:18:19,930 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name ReplLockReorderingTest for class ReplicatedLockReorderingTest
> 2013-02-05 17:18:23,524 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name TransactionReleaseWithAsync2ndPhaseTest for class TransactionCleanupWithAsync2ndPhaseTest
> 2013-02-05 17:18:39,416 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncTest for class AsyncStoreEvictionTest
> 2013-02-05 17:18:39,472 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name RemovedEntryFoundAsyncStoreTest for class AsyncStoreFunctionalTest
> 2013-02-05 17:18:52,937 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name InitiatorCrasherBeforePrepareOptimisticTest for class InitiatorCrashOptimisticTest
> 2013-02-05 17:19:02,368 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name BasicSingleLockRepOptTest for class BasicSingleLockReplOptTest
> 2013-02-05 17:19:24,012 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name TopologyAwareConsistentHashFactoryTest for class TopologyAwareSyncConsistentHashFactoryTest
> 2013-02-05 17:19:34,681 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncAPISyncDistTest for class AsyncAPITxSyncDistTest
> 2013-02-05 17:19:35,098 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name AsyncAPIAsyncDistTest for class AsyncAPITxAsyncDistTest
> 2013-02-05 17:20:44,563 WARN [org.infinispan.test.fwk.UnitTestTestNGListener] (TestNG) Wrong test name TxAndAsyncOpsTest for class TxCacheAndAsyncOpsTest
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months
[JBoss JIRA] (ISPN-2475) When L1.onRehash is enabled, L1 invalidations should be sent to the previous owners
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2475?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2475:
--------------------------------
Fix Version/s: 5.3.0.Final
(was: 6.0.0.Final)
> When L1.onRehash is enabled, L1 invalidations should be sent to the previous owners
> -----------------------------------------------------------------------------------
>
> Key: ISPN-2475
> URL: https://issues.jboss.org/browse/ISPN-2475
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer
> Affects Versions: 5.1.0.FINAL
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.3.0.Final
>
>
> Copied from the parent:
> {quote}
> [...] we can keep track of the consistent hashes in the last 10 minutes (or whatever the L1 lifespan is) and the time of the last invalidation sent for each key. When we need to send a new invalidation, we add the owners in all the consistent hashes since the last invalidation to the invalidation command recipients.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years, 2 months