[JBoss JIRA] (ISPN-3766) "containsKey" and "get" methods are not working properly with "putForExternalRead" during active transaction. WriteSkewException appears.
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3766?page=com.atlassian.jira.plugin.... ]
William Burns edited comment on ISPN-3766 at 11/27/13 4:12 PM:
---------------------------------------------------------------
It is still the expected behavior
* If I don't use "get" method everything is working ok.
It sounds like whatever method you are using doesn't detect write skews then (like #3 below)
* If cache already contains element before "get" invocation - everything is working ok
That is because the PFER doesn't actually modify the value since it is a putIfAbsent call (thus no write skew occurred)
* when I use my workaround for "containsKey" functional - everything is working ok (e.g cache.keySet().contains(key))
That is because you are bypassing all write skew checks and checking the data container directly. Which lets you do this write skew
{quote}
The problem is that I don't need to do read operation at 3, I need to check if element exists in the cache. I need a "containsKey" method, but this method has the same implementation as "get".
{quote}
That is because it also pays attention to write skews. In your case you are trying to later modify your value in the same transaction, but another transaction has changed the value, which is a write skew (PFER has it's own transaction).
was (Author: william.burns):
It is still the expected behavior
* If I don't use "get" method everything is working ok.
It sounds like whatever method you are using doesn't detect write skews then (like #3 below)
* If cache already contains element before "get" invocation - everything is working ok
That is because the PFER doesn't actually modify the value since it is a putIfAbsent call (thus no write skew occurred)
* when I use my workaround for "containsKey" functional - everything is working ok (e.g cache.keySet().contains(key))
That is because you are bypassing all write skew checks and checking the data container directly. Which lets you do this write skew
{quote}
The problem is that I don't need to do read operation at 3, I need to check if element exists in the cache. I need a "containsKey" method, but this method has the same implementation as "get".
{quote}
That is because it also pays attention to write skews. In your case you are trying to later modify your value in the same transaction, but another transaction has changed the value (write skew) in another transaction (PFER has it's own transaction).
> "containsKey" and "get" methods are not working properly with "putForExternalRead" during active transaction. WriteSkewException appears.
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-3766
> URL: https://issues.jboss.org/browse/ISPN-3766
> Project: Infinispan
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 5.3.0.Final, 6.0.0.CR1, 6.0.0.Final
> Reporter: V L
> Assignee: Mircea Markus
> Attachments: InfinispanTest.java
>
>
> I'm not new with Infinispan, we use it from 5.3 version. Now I'm working on adding WriteSkew functional to our project, but unfortunately it's not working as we expected.
> We use infinispan like a cache for mongoDb data source, but this situation doesn't depends on data source vendor.
> The problem is that WriteSkewException is available in specific situation, here are steps for reproducing:
> 1) begin tx -> create an new element -> commit tx.
> 2) wait until cache expires. cache is empty now
> 3) being tx -> try to get element from cache, but it's not found -> load element from data source -> put it to the cache via putForExternalRead for read access outside of current transaction.
> 4) make some modification with element -> put element to cache -> commit tx -> WriteSkewException
> here is my log:
> TX BEGIN tx_status=6 thread=1
> PUT id=key1, tx_status=0, thread=1 cache_size=1, cache_ contains=key:key1, value:value
> TX COMMIT tx_status=0 thread=1
> TX BEGIN. tx_status=6, thread=1
> SIZE id=key1 tx_status=0, thread=1 cache_size=1 contains=key:key1, value:null
> PUT_FOR_EXTERNAL_READ id=key1, tx_status=0 thread=1 cache_size=1 cache_contains=key:key1, value:null
> PUT id=key1 tx_status=0 thread=1 cache_size=1 contains=key:key1, value:value
> TX COMMIT status=0 thread=1
> ISPN000005: Detected write skew on key [1]. Another process has changed the entry since we last read it! Unable to copy entry for update.
> Exception executing call org.infinispan.transaction.WriteSkewException: Detected write skew.
> I think I found the problem is in "get" method. When you invoke "get" method with key that not available in cache, infinispan create an "null" element in cache. This element won't be updated by putForExternalRead method (look at log above). If you skip "get" method invocation everithing will work fine. By the way, everithing is working fine if you skip "step 2" of the scenario.
> I use workaround for this situation. I check if element exist in the cache not by "containsKey" or "get" method but via cache.keySet().contains(key)
> This works ok
> I'm looking forward for you comments, maybe I'm doing something wrong.
> Thanks
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3737) L1 requestor registered after value read
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3737?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-3737:
-------------------------------------
Also this only affects RepeatableRead since Read Committed would return the correct value and thus would have cached the proper value. I will have to add in some Repeatable Read unit tests.
> L1 requestor registered after value read
> ----------------------------------------
>
> Key: ISPN-3737
> URL: https://issues.jboss.org/browse/ISPN-3737
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 6.0.0.Final
> Reporter: Radim Vansa
> Assignee: William Burns
> Priority: Critical
> Labels: 620
>
> As the L1 requestor is registered only after the value is retrieved from data container, the (transactional) update of the value may not invalide the entry after write and the cache gets inconsistent.
> Consider this interleaving of operations (G=get request from other node, C=commit)
> R: read value -> old value
> C: update old -> new
> C: notify requestors for key
> R: add requestor for key
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3767) MassIndexer breaks search feature with one node cluster
by Alan Field (JIRA)
[ https://issues.jboss.org/browse/ISPN-3767?page=com.atlassian.jira.plugin.... ]
Alan Field updated ISPN-3767:
-----------------------------
Assignee: Mircea Markus (was: Alan Field)
> MassIndexer breaks search feature with one node cluster
> -------------------------------------------------------
>
> Key: ISPN-3767
> URL: https://issues.jboss.org/browse/ISPN-3767
> Project: Infinispan
> Issue Type: Bug
> Reporter: Romain Pelisse
> Assignee: Mircea Markus
> Priority: Minor
>
> Hi,
> Trying to cope with the extreme slowliness of put() operation with indexing [1], I've tried to use the MassIndexer, to create the index AFTER adding all the data in the grid. This actually works pretty well, but, when running in a "single node" grid, it "breaks" the search, which always returns 0 result to any kind of query.
> I've modified one of the test suite of InfiniSpan to reproduce the issue:
> https://github.com/rpelisse/infinispan/tree/mass-indexer-breaks-search-wi...
> Once this branch is checked out, just run :
> $ cd ./query
> $ mvn clean -Dtest=org.infinispan.query.distributed.DistributedMassIndexingTest test
> Note: MassIndexer being implemented using the Map/Reduce algorithm, it requires to run within a cluster (ie several instances of ISPN).
> [1] http://stackoverflow.com/questions/10090361/infinispan-very-slow-for-load...
> If run within a single node cluster, the MassIndexer
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3048) Eviction needs to be transactional
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3048?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-3048:
--------------------------------
Labels: 620 (was: )
> Eviction needs to be transactional
> ----------------------------------
>
> Key: ISPN-3048
> URL: https://issues.jboss.org/browse/ISPN-3048
> Project: Infinispan
> Issue Type: Bug
> Components: Eviction
> Affects Versions: 5.3.0.Alpha1
> Reporter: Paul Ferraro
> Assignee: Pedro Ruivo
> Priority: Critical
> Labels: 620
> Fix For: 6.1.0.Final
>
>
> Currently, Infinispan eviction is non-transactional. This makes Infinispan's eviction manager virtually unusable, since non-transactional eviction can cause phantom reads and data loss because it violates the isolation of concurrent transactions. This is especially problematic when using a passivation-enabled cache store. In this case, a cache eviction/passivation can cause a concurrently executed cache retrieval to return null - even though the act of passivation does not change the data - it only changes where it is stored.
> We work around this in the AS by performing eviction manually, using pessimistic locking in combination with eager lock acquisition prior to eviction. This is unfortunate, since it prevents me from leveraging Infinispan's build-in eviction strategies.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3048) Eviction needs to be transactional
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3048?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-3048:
--------------------------------
Labels: (was: 620)
> Eviction needs to be transactional
> ----------------------------------
>
> Key: ISPN-3048
> URL: https://issues.jboss.org/browse/ISPN-3048
> Project: Infinispan
> Issue Type: Bug
> Components: Eviction
> Affects Versions: 5.3.0.Alpha1
> Reporter: Paul Ferraro
> Assignee: Pedro Ruivo
> Priority: Critical
> Fix For: 6.1.0.Final
>
>
> Currently, Infinispan eviction is non-transactional. This makes Infinispan's eviction manager virtually unusable, since non-transactional eviction can cause phantom reads and data loss because it violates the isolation of concurrent transactions. This is especially problematic when using a passivation-enabled cache store. In this case, a cache eviction/passivation can cause a concurrently executed cache retrieval to return null - even though the act of passivation does not change the data - it only changes where it is stored.
> We work around this in the AS by performing eviction manually, using pessimistic locking in combination with eager lock acquisition prior to eviction. This is unfortunate, since it prevents me from leveraging Infinispan's build-in eviction strategies.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3767) MassIndexer breaks search feature with one node cluster
by Romain Pelisse (JIRA)
[ https://issues.jboss.org/browse/ISPN-3767?page=com.atlassian.jira.plugin.... ]
Romain Pelisse updated ISPN-3767:
---------------------------------
I think I misunderstood how the test is working, so my test modification is not really acurate. i'll work on this and update asap.
> MassIndexer breaks search feature with one node cluster
> -------------------------------------------------------
>
> Key: ISPN-3767
> URL: https://issues.jboss.org/browse/ISPN-3767
> Project: Infinispan
> Issue Type: Bug
> Reporter: Romain Pelisse
> Assignee: Alan Field
> Priority: Minor
>
> Hi,
> Trying to cope with the extreme slowliness of put() operation with indexing [1], I've tried to use the MassIndexer, to create the index AFTER adding all the data in the grid. This actually works pretty well, but, when running in a "single node" grid, it "breaks" the search, which always returns 0 result to any kind of query.
> I've modified one of the test suite of InfiniSpan to reproduce the issue:
> https://github.com/rpelisse/infinispan/tree/mass-indexer-breaks-search-wi...
> Once this branch is checked out, just run :
> $ cd ./query
> $ mvn clean -Dtest=org.infinispan.query.distributed.DistributedMassIndexingTest test
> Note: MassIndexer being implemented using the Map/Reduce algorithm, it requires to run within a cluster (ie several instances of ISPN).
> [1] http://stackoverflow.com/questions/10090361/infinispan-very-slow-for-load...
> If run within a single node cluster, the MassIndexer
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3737) L1 requestor registered after value read
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3737?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-3737:
-------------------------------------
Good catch! Normally just putting the registration of the requestor before the get completes would have an inverse effect (we could invalidate something that isn't present), but because of the L1 rewrite to not store L1 values if a concurrent invalidation occurs while waiting on a remote get, that should be a more than feasible fix.
> L1 requestor registered after value read
> ----------------------------------------
>
> Key: ISPN-3737
> URL: https://issues.jboss.org/browse/ISPN-3737
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 6.0.0.Final
> Reporter: Radim Vansa
> Assignee: William Burns
> Priority: Critical
> Labels: 620
>
> As the L1 requestor is registered only after the value is retrieved from data container, the (transactional) update of the value may not invalide the entry after write and the cache gets inconsistent.
> Consider this interleaving of operations (G=get request from other node, C=commit)
> R: read value -> old value
> C: update old -> new
> C: notify requestors for key
> R: add requestor for key
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3737) L1 requestor registered after value read
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3737?page=com.atlassian.jira.plugin.... ]
Work on ISPN-3737 started by William Burns.
> L1 requestor registered after value read
> ----------------------------------------
>
> Key: ISPN-3737
> URL: https://issues.jboss.org/browse/ISPN-3737
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 6.0.0.Final
> Reporter: Radim Vansa
> Assignee: William Burns
> Priority: Critical
> Labels: 620
>
> As the L1 requestor is registered only after the value is retrieved from data container, the (transactional) update of the value may not invalide the entry after write and the cache gets inconsistent.
> Consider this interleaving of operations (G=get request from other node, C=commit)
> R: read value -> old value
> C: update old -> new
> C: notify requestors for key
> R: add requestor for key
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3738) Entry version gets lost during topology change -> NPE
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3738?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-3738:
--------------------------------
Labels: 620 (was: )
> Entry version gets lost during topology change -> NPE
> -----------------------------------------------------
>
> Key: ISPN-3738
> URL: https://issues.jboss.org/browse/ISPN-3738
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 6.0.0.Final
> Reporter: Radim Vansa
> Assignee: Pedro Ruivo
> Priority: Critical
> Labels: 620
>
> Replicated TX cache with WSC, A, B are in cluster, C is joining
> 0. The current CH already contains A and B as owners, C is joining (is not primary owner of anything yet). B is primary owner of K=V.
> 1. A sends PrepareCommand to B and C with put(K, V) (V is null on all nodes)
> 2. C receives PrepareCommand and responds with no versions (it is not primary owner)
> 3. topology changes on B - primary ownership of K is transfered to C
> 4. B receives PrepareCommand, responds without K's version (it is not primary)
> 5. B forwards the Prepare to C as it sees that the command has lower topology ID
> 6. C responds to B's prepare with version of K
> 7. K version is *not* added to B's response, B responds to A
> 8. A finds out that topology has changed, forwards prepare to C
> 9. C responds to C's prepare with version of K
> 10. A receives C's response, but the versions are not added to transaction
> 11. A sends out CommitCommand missing version of K
> 12. all nodes record K=V without version as usual ImmortalCacheEntry
> 13. the next time we try to increase version of K=V, we fail with NPE in SimpleClusteredVersionGenerator (actually when it tries to throw IllegalArgumentException because the null version is unexpected version class)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months
[JBoss JIRA] (ISPN-3745) Forwarded Prepare/Commit executed after transaction finished
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-3745?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-3745:
--------------------------------
Labels: 620 (was: )
> Forwarded Prepare/Commit executed after transaction finished
> ------------------------------------------------------------
>
> Key: ISPN-3745
> URL: https://issues.jboss.org/browse/ISPN-3745
> Project: Infinispan
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 6.0.0.Final
> Reporter: Radim Vansa
> Assignee: Mircea Markus
> Priority: Critical
> Labels: 620
>
> Replicated TX cache, nodes A, B, C
> 0. A and B have topology 2, C already got topology 3
> 1. A sends prepare with topology 2 to B and C, both apply the prepare and respond
> 2. C forwards prepare to B with topology 3
> 3. A sends commit with topology 2 to B and C, both commit and respond
> 4. again, C forwards prepare to B with topology 3
> 5. A and B get updated topology id
> 6. A executes another transaction on the same entry
> 7. prepare and commit from first transaction with topology 3 arrive at B - B overwrites (or removes) the entry again
> Result: on B we have inconsistent state
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 5 months