[JBoss JIRA] (ISPN-2974) DeltaAware based fine-grained replication corrupts cache data, if eviction is enabled
by Horia Chiorean (JIRA)
[ https://issues.jboss.org/browse/ISPN-2974?page=com.atlassian.jira.plugin.... ]
Horia Chiorean commented on ISPN-2974:
--------------------------------------
Marking as critical, because ModeShape relies on partial delta updates.
> DeltaAware based fine-grained replication corrupts cache data, if eviction is enabled
> -------------------------------------------------------------------------------------
>
> Key: ISPN-2974
> URL: https://issues.jboss.org/browse/ISPN-2974
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.1.Final, 5.2.5.Final
> Reporter: Horia Chiorean
> Assignee: Mircea Markus
> Priority: Critical
>
> When using a custom {{DeltaAware}} implementation in a cluster with 2 replicated nodes with eviction enabled, data transferred from one node (the writer) to the another (the reader) causes data stored on this node but evicted at the time of the change, to be rewritten with whatever the partial latest delta was.
> In more detail:
> * configure 2 nodes in replicated mode, with eviction enabled
> * consider NodeA the writer and NodeB the reader
> * NodeA inserts some data (custom entries) into the cache
> * NodeB correctly receives via state transfer the initial data
> * NodeA loads & partially updates some information about an entry which was not in the cache - was evicted previously
> * NodeB receives the partial delta with the changes from NodeA, but *instead of merging* with whatever is stored in the persistent store, *replaces the entire entry in the cache*, leaving it in effect with "partial/corrupt information"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years
[JBoss JIRA] (ISPN-2974) DeltaAware based fine-grained replication corrupts cache data, if eviction is enabled
by Horia Chiorean (JIRA)
[ https://issues.jboss.org/browse/ISPN-2974?page=com.atlassian.jira.plugin.... ]
Horia Chiorean updated ISPN-2974:
---------------------------------
Description:
When using a custom {{DeltaAware}} implementation in a cluster with 2 replicated nodes with eviction enabled, data transferred from one node (the writer) to the another (the reader) causes data stored on this node but evicted at the time of the change, to be rewritten with whatever the partial latest delta was.
In more detail:
* configure 2 nodes in replicated mode, with eviction enabled
* consider NodeA the writer and NodeB the reader
* NodeA inserts some data (custom entries) into the cache
* NodeB correctly receives via state transfer the initial data
* NodeA loads & partially updates some information about an entry which was not in the cache - was evicted previously
* NodeB receives the partial delta with the changes from NodeA, but *instead of merging* with whatever is stored in the persistent store, *replaces the entire entry in the cache*, leaving it in effect with "partial/corrupt information"
If eviction is not enabled, everything works as expected.
was:
When using a custom {{DeltaAware}} implementation in a cluster with 2 replicated nodes with eviction enabled, data transferred from one node (the writer) to the another (the reader) causes data stored on this node but evicted at the time of the change, to be rewritten with whatever the partial latest delta was.
In more detail:
* configure 2 nodes in replicated mode, with eviction enabled
* consider NodeA the writer and NodeB the reader
* NodeA inserts some data (custom entries) into the cache
* NodeB correctly receives via state transfer the initial data
* NodeA loads & partially updates some information about an entry which was not in the cache - was evicted previously
* NodeB receives the partial delta with the changes from NodeA, but *instead of merging* with whatever is stored in the persistent store, *replaces the entire entry in the cache*, leaving it in effect with "partial/corrupt information"
> DeltaAware based fine-grained replication corrupts cache data, if eviction is enabled
> -------------------------------------------------------------------------------------
>
> Key: ISPN-2974
> URL: https://issues.jboss.org/browse/ISPN-2974
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.1.Final, 5.2.5.Final
> Reporter: Horia Chiorean
> Assignee: Mircea Markus
> Priority: Critical
>
> When using a custom {{DeltaAware}} implementation in a cluster with 2 replicated nodes with eviction enabled, data transferred from one node (the writer) to the another (the reader) causes data stored on this node but evicted at the time of the change, to be rewritten with whatever the partial latest delta was.
> In more detail:
> * configure 2 nodes in replicated mode, with eviction enabled
> * consider NodeA the writer and NodeB the reader
> * NodeA inserts some data (custom entries) into the cache
> * NodeB correctly receives via state transfer the initial data
> * NodeA loads & partially updates some information about an entry which was not in the cache - was evicted previously
> * NodeB receives the partial delta with the changes from NodeA, but *instead of merging* with whatever is stored in the persistent store, *replaces the entire entry in the cache*, leaving it in effect with "partial/corrupt information"
> If eviction is not enabled, everything works as expected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years
[JBoss JIRA] (ISPN-2974) DeltaAware based fine-grained replication corrupts cache data, if eviction is enabled
by Horia Chiorean (JIRA)
Horia Chiorean created ISPN-2974:
------------------------------------
Summary: DeltaAware based fine-grained replication corrupts cache data, if eviction is enabled
Key: ISPN-2974
URL: https://issues.jboss.org/browse/ISPN-2974
Project: Infinispan
Issue Type: Bug
Affects Versions: 5.2.5.Final, 5.2.1.Final
Reporter: Horia Chiorean
Assignee: Mircea Markus
Priority: Critical
When using a custom {{DeltaAware}} implementation in a cluster with 2 replicated nodes with eviction enabled, data transferred from one node (the writer) to the another (the reader) causes data stored on this node but evicted at the time of the change, to be rewritten with whatever the partial latest delta was.
In more detail:
* configure 2 nodes in replicated mode, with eviction enabled
* consider NodeA the writer and NodeB the reader
* NodeA inserts some data (custom entries) into the cache
* NodeB correctly receives via state transfer the initial data
* NodeA loads & partially updates some information about an entry which was not in the cache - was evicted previously
* NodeB receives the partial delta with the changes from NodeA, but *instead of merging* with whatever is stored in the persistent store, *replaces the entire entry in the cache*, leaving it in effect with "partial/corrupt information"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years
[JBoss JIRA] (ISPN-2913) putForExternalRead leaves locks
by Sebastian Tusk (JIRA)
[ https://issues.jboss.org/browse/ISPN-2913?page=com.atlassian.jira.plugin.... ]
Sebastian Tusk commented on ISPN-2913:
--------------------------------------
My understanding is that putForExternalRead is not transactional. See ISPN-1615. At least I can not observe anything that looks like a transaction when using putForExternalRead. My fix <https://github.com/infinispan/infinispan/pull/1737/files> is to unlock if not in a transaction. And the condition (!ctx.isInTxScope()) can be true. So I guess there are ways to have PutKeyValueCommand outside of transactions.
I do not see anything in 5.2.5.Final that looks like it could affect this problem. But I will test this version next week.
> putForExternalRead leaves locks
> -------------------------------
>
> Key: ISPN-2913
> URL: https://issues.jboss.org/browse/ISPN-2913
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 5.2.1.Final
> Reporter: Sebastian Tusk
> Assignee: Mircea Markus
> Priority: Critical
>
> In TxDistributionInterceptor.remoteGetAndStoreInL1 locks are acquired. Without a transaction these locks are never released. The cache setup is Dist, Async, L1, 2 Nodes, 1 Owner, Optimistic Locking.
> In AbstractTxLockingInterceptor.visitGetKeyValueCommand locks are released explicitly if outside of transactions. I fixed this problem by doing the same in OptimisticLockingInterceptor.visitPutKeyValueCommand. It is very likely that this doesn't fix all problems. For instance OptimisticLockingInterceptor.visitPutMapCommand or PessimisticLockingInterceptor.
> Cache Config:
> <namedCache name="entity">
> <jmxStatistics enabled="true" />
>
> <clustering mode="dist">
> <stateTransfer fetchInMemoryState="false" timeout="20000" />
> <async />
> <l1 enabled="true" />
> <hash numOwners="1"/>
> </clustering>
> <locking isolationLevel="READ_COMMITTED"
> lockAcquisitionTimeout="15000" useLockStriping="false" />
>
> <eviction maxEntries="10000" strategy="LRU" />
> <expiration maxIdle="100000" wakeUpInterval="5000"/>
> <storeAsBinary storeKeysAsBinary="true" storeValuesAsBinary="false" enabled="false" />
>
> <transaction transactionMode="TRANSACTIONAL" autoCommit="false" lockingMode="OPTIMISTIC"/>
> </namedCache>
> Fixed OptimisticLockingInterceptor.visitPutKeyValueCommand:
> @Override
> public Object visitPutKeyValueCommand(InvocationContext ctx, PutKeyValueCommand command) throws Throwable {
> try {
> if (command.isConditional()) markKeyAsRead(ctx, command);
> return invokeNextInterceptor(ctx, command);
> } catch (Throwable te) {
> throw cleanLocksAndRethrow(ctx, te);
> } finally {
> //with putForExternalRead the value might be put into L1 without a transaction
> //we need to release any locks for these cases
> if (!ctx.isInTxScope()) lockManager.unlockAll(ctx);
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years
[JBoss JIRA] (ISPN-2737) Thread naming anomaly when reporting lock timeout
by Takayoshi Kimura (JIRA)
[ https://issues.jboss.org/browse/ISPN-2737?page=com.atlassian.jira.plugin.... ]
Takayoshi Kimura updated ISPN-2737:
-----------------------------------
Fix Version/s: 5.2.2.Final
> Thread naming anomaly when reporting lock timeout
> -------------------------------------------------
>
> Key: ISPN-2737
> URL: https://issues.jboss.org/browse/ISPN-2737
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency, RPC
> Affects Versions: 5.2.0.CR1
> Reporter: Michal Linhard
> Assignee: Galder Zamarreño
> Priority: Minor
> Fix For: 5.2.2.Final, 5.3.0.Final
>
>
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EDG6/view/EDG-REPOR...
> {code}
> 11:47:30,859 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (MemcachedServerWorker-277) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [memcachedCache#key763328] for requestor [Thread[OOB-127,null,5,Thread Pools]]! Lock held by [Thread[OOB-150,null,5,Thread Pools]]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:217) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLockNoCheck(LockManagerImpl.java:200) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:114) [infinispan-core-5.2.0.CR1-redhat-1.jar:5.2.0.CR1-redhat-1]
> ....
> at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.Discovery.up(Discovery.java:359) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2640) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP.passMessageUp(TP.java:1287) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1850) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1823) [jgroups-3.2.5.Final-redhat-1.jar:3.2.5.Final-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_38]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_38]
> at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_38]
> {code}
> note the thread name "MemcachedServerWorker" in an operation coming from the JGroups stack...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years
[JBoss JIRA] (ISPN-2923) provides an optimal way to search and retrieve the first non null value
by Mathieu Lachance (JIRA)
[ https://issues.jboss.org/browse/ISPN-2923?page=com.atlassian.jira.plugin.... ]
Mathieu Lachance commented on ISPN-2923:
----------------------------------------
I did not ended up with null values in the cache. Sorry for the confusion.
When I was saying searching, I was referring more to this scenario :
for each (Map.Entry<K, V> entry : cache.values()){
if (entry.getValue().getACertainField().equals(aCertainValue)){
return entry;
}
}
but in a distributed fashion.
By providing a stop in the map reduce operation we could achieve this.
> provides an optimal way to search and retrieve the first non null value
> -----------------------------------------------------------------------
>
> Key: ISPN-2923
> URL: https://issues.jboss.org/browse/ISPN-2923
> Project: Infinispan
> Issue Type: Feature Request
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 5.2.5.Final
> Reporter: Mathieu Lachance
> Assignee: Vladimir Blagojevic
>
> It would be nice if infinispan could provide a way to the common problem of searching and retrieve the first non null value.
> My attempt was to use the map reduce framework, but it would still scan all the keys and value even if the result as been found. Maybe Infinispan could provide a way to "stop" the map-reduce operation on all nodes when asked, ex, when the first non null value has been found ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years