[JBoss JIRA] (ISPN-825) Consider staggering remote get requests when using DIST
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-825?page=com.atlassian.jira.plugin.s... ]
Dan Berindei updated ISPN-825:
------------------------------
Fix Version/s: 6.0.0.Alpha1
(was: 5.3.0.Final)
> Consider staggering remote get requests when using DIST
> -------------------------------------------------------
>
> Key: ISPN-825
> URL: https://issues.jboss.org/browse/ISPN-825
> Project: Infinispan
> Issue Type: Feature Request
> Components: RPC
> Affects Versions: 4.1.0.Final
> Reporter: Manik Surtani
> Assignee: Mircea Markus
> Priority: Critical
> Labels: optimization, performance
> Fix For: 6.0.0.Alpha1
>
>
> In DIST mode, when a request is made on a key that is not mapped locally, a remote get is sent to all data owners of that key and the first response is used. This can add unnecessary load on the network as all nodes still eventually respond, and if values are large this can cause a lot of unnecessary network traffic.
> The purpose of broadcasting to all data owners is so that (1) if one is down, another could still respond (2) if one is overloaded, others may respond faster.
> A solution around this could be based on either (or both) of:
> * Provide a configurable stagger timeout, e.g. 100ms. E.g., RPC to (random) Owner1. Wait for timeout t. If no response, RPC to Owner2. etc.
> * Always broadcast to a (configurable) subset of owners, e.g., always 2 even if numOwners is 5.
> Needs careful thought and design.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
[JBoss JIRA] (ISPN-962) Entries not committed w/ DistLockingInterceptor and L1 caching disabled.
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-962?page=com.atlassian.jira.plugin.s... ]
Dan Berindei updated ISPN-962:
------------------------------
Fix Version/s: 6.0.0.Alpha1
(was: 5.3.0.Final)
> Entries not committed w/ DistLockingInterceptor and L1 caching disabled.
> ------------------------------------------------------------------------
>
> Key: ISPN-962
> URL: https://issues.jboss.org/browse/ISPN-962
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache, Locking and Concurrency
> Affects Versions: 4.2.0.Final
> Reporter: Shane Johnson
> Assignee: Dan Berindei
> Labels: Invalidation, Rehash
> Fix For: 6.0.0.Alpha1
>
>
> If you choose to disable the L1 cache (enabled=false AND onRehash=false) in distributed mode, the DistLockingInterceptor will NOT commit any invalidations due to a rehash back to the data container.
> The problem is in the commitEntry method.
> boolean doCommit = true;
> if (!dm.isLocal(entry.getKey())) {
> if (configuration.isL1CacheEnabled()) {
> dm.transformForL1(entry);
> } else {
> doCommit = false;
> }
> }
> if (doCommit)
> entry.commit(dataContainer);
> else
> entry.rollback();
> For most commands, dm.isLocal returns TRUE and so the execution proceeds to commit. However, invalidation commands are unique in that they are executed on a remote node even though that node is NOT the owner of the entry. For that reason, the dm.isLocal returns FALSE and the execution proceeds to the L1 cache enabled check. If the L1 cache is disabled, the execution proceeds to set doCommit to false and rollback the invalidation.
> We have temporarily fixed this by updating the else block to check and see if the entry has been removed. If it has not, we set doCommit to false like it does now. Otherwise, we set it to true.
> To be honest, that was a safeguard in case we are missing something. I'm still not sure why we would ever want to set doCommit to false just because the L1 cache has been disabled. However, this change has fixed our problem with entries not being deleted from the original owners on a rehash.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
[JBoss JIRA] (ISPN-1822) Cache entry not evicted from memory on IBM JDK when another entry was loaded from a cache loader and maxEntries had been reached
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-1822?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-1822:
-------------------------------
Fix Version/s: 6.0.0.Alpha1
(was: 5.3.0.Final)
> Cache entry not evicted from memory on IBM JDK when another entry was loaded from a cache loader and maxEntries had been reached
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-1822
> URL: https://issues.jboss.org/browse/ISPN-1822
> Project: Infinispan
> Issue Type: Bug
> Components: Eviction
> Affects Versions: 5.1.0.FINAL
> Environment: java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr9fp1-20110208_03(SR9 FP1))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr9-20110203_74623 (JIT enabled, AOT enabled) ;
> java version "1.7.0"
> Java(TM) SE Runtime Environment (build pxi3270-20110827_01)
> IBM J9 VM (build 2.6, JRE 1.7.0 Linux x86-32 20110810_88604 (JIT enabled, AOT enabled)
> Reporter: Martin Gencur
> Fix For: 6.0.0.Alpha1
>
>
> This behavior is specific to IBM JDK (I tried JDK6 and 7), it works fine with Java HotSpot.
> Steps to reproduce the problem:
> 1) set maxEntries for eviction to 2 and algorithm e.g. to LRU
> 2) store 3 entries key1, key2, key3 to the cache (after that you can see that the cache contains only 2 entries - key2 and key3, the first one was evicted from memory)
> 3) call cache.get("key1")
> 4) PROBLEM - cache contains all key1, key2, key3 even though it should contain only 2 entries - only happens with IBM JDK (6 or 7 ..no matter)
> I'll shortly issue a pull request with a test to ispn-core
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
[JBoss JIRA] (ISPN-1841) Write skew checks are performed on all entries in a transaction context
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-1841?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-1841:
-------------------------------
Fix Version/s: 6.0.0.Alpha1
(was: 5.3.0.Final)
> Write skew checks are performed on all entries in a transaction context
> -----------------------------------------------------------------------
>
> Key: ISPN-1841
> URL: https://issues.jboss.org/browse/ISPN-1841
> Project: Infinispan
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 5.1.6.FINAL
> Reporter: Manik Surtani
> Assignee: Mircea Markus
> Fix For: 6.0.0.Alpha1
>
>
> They should only be performed on entries that are read first and then updated. The current implementation doesn't cause any problems, however it is unnecessary processing and certain transactions may unnecessarily abort if, for example, an entry is read, and not written to, but the entry changes before the transaction commits.
> From Pedro Ruivo's email to infinispan-dev, where this was reported:
> {quote}
> I've noticed that in the last version (5.1.x) the write skew check is
> performed on all keys written. However, from your documentation [1] I
> understood that the write skew was meant to be performed only on the
> written keys that were previously read.
> Is this change intentional?
> Cheers,
> Pedro Ruivo
> [1] https://docs.jboss.org/author/display/ISPN51/Data+Versioning
> "Write skew checks are performed at prepare-time to ensure a concurrent
> transaction hasn't modified an entry while it was read and potentially
> updated based on the value read."
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
[JBoss JIRA] (ISPN-1146) Improve documentation around transaction and locking
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-1146?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-1146:
-------------------------------
Fix Version/s: 6.0.0.Alpha1
(was: 5.3.0.Final)
> Improve documentation around transaction and locking
> ----------------------------------------------------
>
> Key: ISPN-1146
> URL: https://issues.jboss.org/browse/ISPN-1146
> Project: Infinispan
> Issue Type: Task
> Components: Transactions
> Reporter: Mircea Markus
> Assignee: Mircea Markus
> Fix For: 6.0.0.Alpha1
>
>
> As suggested by Jonathan Halliday:
> - preserving some subset of the existing transactional guarantees is all well and good BUT if the user is relying on additional 'guarantees' that are not preserved in all cases then they'll be in trouble. Therefore, it's essential to document not just what the minimum expected guarantees for a given config are, but also that no additional properties that may coincidently be observed are actually guaranteed. Some vendors go further and explicitly document the bad things that may happen with given settings, since in some cases these are not easy to reproduce in testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
[JBoss JIRA] (ISPN-2013) Using explicit "unlock" causes TimeoutException for other Threads
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-2013?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-2013:
-------------------------------
Fix Version/s: 6.0.0.Alpha1
(was: 5.3.0.Final)
> Using explicit "unlock" causes TimeoutException for other Threads
> -----------------------------------------------------------------
>
> Key: ISPN-2013
> URL: https://issues.jboss.org/browse/ISPN-2013
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 5.1.2.FINAL
> Environment: Windows 7 64-bit
> Reporter: Dmitry Udalov
> Assignee: Mircea Markus
> Fix For: 6.0.0.Alpha1
>
> Attachments: LockTest.java
>
>
> In a test I have several tasks that run on a single cache node configured as transactional pessimistic replicated cache. If I explicitly call "unlock" then I consistently see TimeoutException reported by some tasks. Without explicit "unlock" the test works fine. Does it mean that I should never call "unlock" and rely on transaction.commit/rollback ? It's seen with infinispan-5.1.2.FINAL
>
> Here's what in a nutshell each task does:
>
> final lockKey = ...
>
> executor.submit(new Callable<Boolean>() {
> public Boolean call() throws Exception
>
> TransactionManager tx = cache.getAdvancedCache().getTransactionManager();
> tx.begin()
> try {
> if ( lockManager.lock(lockKey) ) {
>
> // ...
>
> // removing next line makes test happy. Otherwise some tasks report TimeoutException (pls. see the stack traces)
> cache.getLockManager().unlock(lockKey);
> }
> } catch(Throwable t) {
> tx.setRollbackOnly();
> } finally {
> if (ut.getStatus() == Status.STATUS_ACTIVE)
> ut.commit();
> else
> ut.rollback();
> }
>
>
> By the time I received that exception all other tasks were completed and they released the lock on the key in question.
> I also see that LockManagerImpl.lock recognized that the lock was not owned by any thread - see "Lock held by [null]", which seems to be right. But yet the lock failed to be acquired.
> Is it a matter to trying it one more time?
>
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [foo] for requestor [GlobalTransaction:<Sound-15075>:8:local]! Lock held by [null]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:206)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:180)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:170)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:209)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:136)
> at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:228)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:129)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
> at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:159)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:129)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:144)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:129)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> at org.infinispan.interceptors.StateTransferLockInterceptor.handleWithRetries(StateTransferLockInterceptor.java:207)
> at org.infinispan.interceptors.StateTransferLockInterceptor.visitLockControlCommand(StateTransferLockInterceptor.java:138)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:129)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
> at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:159)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:129)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:130)
> at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:94)
> at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:129)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345)
> at org.infinispan.CacheImpl.lock(CacheImpl.java:484)
> at org.infinispan.CacheImpl.lock(CacheImpl.java:468)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months
[JBoss JIRA] (ISPN-2103) Concurrent access after removal of an AtomicMap should NOT result in an IllegalStateException when accessed by other threads
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-2103?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-2103:
-------------------------------
Fix Version/s: 6.0.0.Alpha1
(was: 5.3.0.Final)
> Concurrent access after removal of an AtomicMap should NOT result in an IllegalStateException when accessed by other threads
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-2103
> URL: https://issues.jboss.org/browse/ISPN-2103
> Project: Infinispan
> Issue Type: Bug
> Components: Core API
> Affects Versions: 5.1.5.FINAL
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Fix For: 6.0.0.Alpha1
>
>
> ISPN-1121 introduces an IllegalStateException that is being thrown from AtomicMap methods once the map handle has become stale (ie. removed from cache). In case of concurrent access, the exception is thrown to all threads not just to the thread that performed the removal. This was fine-ish in older versions of Infinispan before introduction of optimistic and pessimistic locking but it should be reconsidered now because:
> 1. It interferes/overlaps with transaction isolation. We should rely on the selected locking scheme (OL/PL) to detect conflicts between transactions and report the problem accordingly. Especially if the stale map is used just for reading - this should be allowed to work rather than stop it.
> 2. This exception is pretty disruptive and awkward to handle. All methods of an AtomicMap can result in this exception and the current thread has to be prepared for handling it if other threads remove the map. Not much transaction isolation.
> 3. Since the TreeCache is backed by AtomicMap nearly all Tree API can throw this.
> The proposed fix consists of:
> 1. removing AtomicHashMap.removed flag and AtomicHashMap.markRemoved() method.
> 2. revising AtomicHashMapProxy.assertValid() method to check only if the map is null (ie. removed) but no longer use the removed flag.
> 3. revising ReadCommittedEntry.commit() method to no longer call markRemoved() method.
> The consequences of these changes are:
> 1. Any further access to a stale map results in IllegalStateException ONLY in the thread that performed the removal. This thread 'knows' the map is stale so it is fine to punish it. Other threads remain unaffected until lock acquisition or commit is performed (depending on locking model).
> 2. Other threads can continue to use the previously obtained map handle for reads without danger of getting an exception.
> 3. If a write operation is done on the map, the results depend on the locking model:
> 3.1 optimistic locking + write skew check: a WriteSkewException will stop the commit during prepare
> 3.2 optimistic locking, no write skew check: the write is committed and the work of the transaction that removed the map is overwritten. The map is effectively revived.
> 3.3 pessimistic locking: same as 3.2
> Please note 3.2 and 3.3 work the same as for normal values (not atomic maps).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 6 months