[JBoss JIRA] (ISPN-5210) Support writing multiple modifications to cache stores in batch when using Transactions
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-5210?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-5210:
--------------------------------
Fix Version/s: 8.2.0.Alpha1
(was: 8.1.0.Final)
> Support writing multiple modifications to cache stores in batch when using Transactions
> ---------------------------------------------------------------------------------------
>
> Key: ISPN-5210
> URL: https://issues.jboss.org/browse/ISPN-5210
> Project: Infinispan
> Issue Type: Feature Request
> Components: Loaders and Stores
> Affects Versions: 6.0.2.Final, 7.1.0.Final
> Reporter: Richard Lucas
> Fix For: 8.2.0.Alpha1
>
>
> Currently writes to a cache store are performed individually for each modification to the cache.
> While this makes sense when using a non-tx cache it would be beneficial to support writing multiple modifications to a cache store in a single call when using a Tx cache.
> Currently all writes are performed in the commit phase and are done by looping through the modifications in the Tx and writing each one in turn to the the store.
> Instead of doing this it would useful to pass all modifications to the store in a single call, this would allow:
> a) Taking advantage of support for batch updates in underlying stores (JDBC, MongoDB, DynamoDB) for a more efficient write through.
> b) Allow store implementations the chance to clean up if one or more updates in the batch fail (this is especially useful if the store does not support Tx and rollbacks as it means the store implementation can at least try to clean up any partial updates it has performed, something which is not currently possible).
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 10 months
[JBoss JIRA] (ISPN-5179) Add distributed execution and map/reduce job statistics
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-5179?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-5179:
--------------------------------
Fix Version/s: 8.2.0.Alpha1
(was: 8.1.0.Final)
> Add distributed execution and map/reduce job statistics
> --------------------------------------------------------
>
> Key: ISPN-5179
> URL: https://issues.jboss.org/browse/ISPN-5179
> Project: Infinispan
> Issue Type: Feature Request
> Components: JMX, reporting and management
> Reporter: Vladimir Blagojevic
> Assignee: Vladimir Blagojevic
> Fix For: 8.2.0.Alpha1
>
>
> We should add DMR/JMX statistics for the running distributed execution jobs as well as map/reduce jobs. The statistics will also include overview/total system statistics of previously executed jobs; we might store statistics of individual executed jobs in some internal cache. However, the primary objective is to calculate and maintain dist.exec and map/reduce job statistics for Infinispan admin console.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 10 months
[JBoss JIRA] (ISPN-5163) A write operation with the SKIP_LOCKING flag can roll back the transaction
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-5163?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-5163:
--------------------------------
Fix Version/s: 8.2.0.Alpha1
(was: 8.1.0.Final)
> A write operation with the SKIP_LOCKING flag can roll back the transaction
> --------------------------------------------------------------------------
>
> Key: ISPN-5163
> URL: https://issues.jboss.org/browse/ISPN-5163
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.0.3.Final, 7.1.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 8.2.0.Alpha1
>
>
> When a write operation has the SKIP_LOCKING flag, it does not send a {{LockControlCommand}} to the primary owner, but it can send a {{ClusteredGetCommand}} with {{acquireRemoteLocks=true}} instead. The {{ClusteredGetCommmand}} will then execute a {{LockControlCommand}} with the origin not set properly, and {{TxInterceptor}} will roll back the transaction because the originator ({{null}}) appears to have left the cluster.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 10 months
[JBoss JIRA] (ISPN-5151) DistributedSharedCacheTwoNodesMapReduceTest.testInvokeMapReduceOnAllKeys random failures
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-5151?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-5151:
--------------------------------
Fix Version/s: 8.2.0.Alpha1
(was: 8.1.0.Final)
> DistributedSharedCacheTwoNodesMapReduceTest.testInvokeMapReduceOnAllKeys random failures
> ----------------------------------------------------------------------------------------
>
> Key: ISPN-5151
> URL: https://issues.jboss.org/browse/ISPN-5151
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.0.3.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.2.0.Alpha1
>
>
> The method {{invokeMapReduce()}} doesn't really invoke the M/R task, it only creates it, and the execution only starts when the test method calls {{task.execute()}} explicitly. It shouldn't try to check the contents of the shared intermediary cache, because the intermediary cache may not exist yet - and it may accidentally create it with the wrong configuration. I get this error when I run only the {{testInvokeMapReduceOnAllKeys}} method:
> {noformat}
> 09:55:37,632 TRACE (testng-DistributedSharedCacheTwoNodesMapReduceTest:) [DefaultCacheManager] About to wire and start cache __tmpMapReduce
> 09:55:37,646 DEBUG (testng-DistributedSharedCacheTwoNodesMapReduceTest:) [MapReduceTask] Invoking CreateCacheCommand{cacheManager=null, cacheNameToCreate='__tmpMapReduce', cacheConfigurationName='__tmpMapReduce', start=true', size=2} across members [DistributedSharedCacheTwoNodesMapReduceTest-NodeA-19271, DistributedSharedCacheTwoNodesMapReduceTest-NodeB-10341]
> 10:32:56,324 ERROR (testng-DistributedSharedCacheTwoNodesMapReduceTest:) [UnitTestTestNGListener] Test testInvokeMapReduceOnAllKeys(org.infinispan.distexec.mapreduce.DistributedSharedCacheTwoNodesMapReduceTest) failed.
> org.infinispan.distexec.mapreduce.MapReduceException: Map phase failed
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhase(MapReduceTask.java:607)
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeHelper(MapReduceTask.java:473)
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:414)
> at org.infinispan.distexec.mapreduce.BaseWordCountMapReduceTest.testInvokeMapReduceOnAllKeys(BaseWordCountMapReduceTest.java:162)
> Caused by: org.infinispan.commons.CacheException: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:105)
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.invokeMapCombineLocally(MapReduceTask.java:1174)
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.access$300(MapReduceTask.java:1101)
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:1123)
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:1119)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapKeysToNodes(MapReduceManagerImpl.java:363)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.migrateIntermediateKeysAndValues(MapReduceManagerImpl.java:327)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombine(MapReduceManagerImpl.java:260)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:103)
> ... 10 more
> {noformat}
> Even if the check is moved after the M/R task is finished, it still wouldn't be correct, because the task only cleans up the shared intermediary cache asynchronously. So it needs to use {{eventually()}} to avoid errors like this:
> {noformat}
> 04:06:32,260 ERROR (testng-DistributedSharedCacheTwoNodesMapReduceTest:) [UnitTestTestNGListener] Test testInvokeMapReduceOnAllKeys(org.infinispan.distexec.mapreduce.DistributedSharedCacheTwoNodesMapReduceTest) failed.
> java.lang.AssertionError: Shared cache __tmpMapReduce is not empty. It has 5 keys/values: [ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=is], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@21ae10d3}, ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=JUDCon], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@108d6b51}, ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=cool], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@77949e8f}, ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=Infinispan], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@712a6071}, ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=community], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@291bdf76}] expected:<0> but was:<5>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.infinispan.distexec.mapreduce.DistributedSharedCacheTwoNodesMapReduceTest.invokeMapReduce(DistributedSharedCacheTwoNodesMapReduceTest.java:44)
> at org.infinispan.distexec.mapreduce.BaseWordCountMapReduceTest.testInvokeMapReduceOnAllKeys(BaseWordCountMapReduceTest.java:161)
> 04:06:32,579 TRACE (transport-thread-NodeA-p29577-t6:) [InvocationContextInterceptor] Invoked with command RemoveCommand{key=IntermediateCompositeKey [taskId=eb7da48a-5922-4671-9037-4077e209744c, key=RedHat], value=null, flags=null, valueMatcher=MATCH_ALWAYS} and InvocationContext [org.infinispan.context.SingleKeyNonTxInvocationContext@c0bbc61]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 10 months
[JBoss JIRA] (ISPN-5083) Hot Rod decoder should use async Cache operations
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-5083?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-5083:
--------------------------------
Fix Version/s: 8.2.0.Alpha1
(was: 8.1.0.Final)
> Hot Rod decoder should use async Cache operations
> -------------------------------------------------
>
> Key: ISPN-5083
> URL: https://issues.jboss.org/browse/ISPN-5083
> Project: Infinispan
> Issue Type: Enhancement
> Components: Remote Protocols
> Reporter: Galder Zamarreño
> Assignee: Gustavo Fernandes
> Fix For: 8.2.0.Alpha1
>
>
> Hot Rod decoder is currently tying up Netty threads as a result of calling up to Infinispan sync operations. Instead, Hot Rod decoder should call up async operations, convert the Notifying Futures to Scala Futures, and write up the reply when it's received. This should increase performance specially under heavy load.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 10 months