[JBoss JIRA] (ISPN-5151) DistributedSharedCacheTwoNodesMapReduceTest.testInvokeMapReduceOnAllKeys random failures
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5151?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5151:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> DistributedSharedCacheTwoNodesMapReduceTest.testInvokeMapReduceOnAllKeys random failures
> ----------------------------------------------------------------------------------------
>
> Key: ISPN-5151
> URL: https://issues.jboss.org/browse/ISPN-5151
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.0.3.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.2.0.CR1
>
>
> The method {{invokeMapReduce()}} doesn't really invoke the M/R task, it only creates it, and the execution only starts when the test method calls {{task.execute()}} explicitly. It shouldn't try to check the contents of the shared intermediary cache, because the intermediary cache may not exist yet - and it may accidentally create it with the wrong configuration. I get this error when I run only the {{testInvokeMapReduceOnAllKeys}} method:
> {noformat}
> 09:55:37,632 TRACE (testng-DistributedSharedCacheTwoNodesMapReduceTest:) [DefaultCacheManager] About to wire and start cache __tmpMapReduce
> 09:55:37,646 DEBUG (testng-DistributedSharedCacheTwoNodesMapReduceTest:) [MapReduceTask] Invoking CreateCacheCommand{cacheManager=null, cacheNameToCreate='__tmpMapReduce', cacheConfigurationName='__tmpMapReduce', start=true', size=2} across members [DistributedSharedCacheTwoNodesMapReduceTest-NodeA-19271, DistributedSharedCacheTwoNodesMapReduceTest-NodeB-10341]
> 10:32:56,324 ERROR (testng-DistributedSharedCacheTwoNodesMapReduceTest:) [UnitTestTestNGListener] Test testInvokeMapReduceOnAllKeys(org.infinispan.distexec.mapreduce.DistributedSharedCacheTwoNodesMapReduceTest) failed.
> org.infinispan.distexec.mapreduce.MapReduceException: Map phase failed
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhase(MapReduceTask.java:607)
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeHelper(MapReduceTask.java:473)
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:414)
> at org.infinispan.distexec.mapreduce.BaseWordCountMapReduceTest.testInvokeMapReduceOnAllKeys(BaseWordCountMapReduceTest.java:162)
> Caused by: org.infinispan.commons.CacheException: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:105)
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.invokeMapCombineLocally(MapReduceTask.java:1174)
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.access$300(MapReduceTask.java:1101)
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:1123)
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:1119)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapKeysToNodes(MapReduceManagerImpl.java:363)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.migrateIntermediateKeysAndValues(MapReduceManagerImpl.java:327)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombine(MapReduceManagerImpl.java:260)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:103)
> ... 10 more
> {noformat}
> Even if the check is moved after the M/R task is finished, it still wouldn't be correct, because the task only cleans up the shared intermediary cache asynchronously. So it needs to use {{eventually()}} to avoid errors like this:
> {noformat}
> 04:06:32,260 ERROR (testng-DistributedSharedCacheTwoNodesMapReduceTest:) [UnitTestTestNGListener] Test testInvokeMapReduceOnAllKeys(org.infinispan.distexec.mapreduce.DistributedSharedCacheTwoNodesMapReduceTest) failed.
> java.lang.AssertionError: Shared cache __tmpMapReduce is not empty. It has 5 keys/values: [ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=is], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@21ae10d3}, ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=JUDCon], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@108d6b51}, ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=cool], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@77949e8f}, ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=Infinispan], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@712a6071}, ImmortalCacheEntry{key=IntermediateCompositeKey [taskId=88948a8b-2a8a-4c13-bc45-4dc3a9f6b0fb, key=community], value=org.infinispan.distexec.mapreduce.MapReduceManagerImpl$DeltaAwareList@291bdf76}] expected:<0> but was:<5>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.infinispan.distexec.mapreduce.DistributedSharedCacheTwoNodesMapReduceTest.invokeMapReduce(DistributedSharedCacheTwoNodesMapReduceTest.java:44)
> at org.infinispan.distexec.mapreduce.BaseWordCountMapReduceTest.testInvokeMapReduceOnAllKeys(BaseWordCountMapReduceTest.java:161)
> 04:06:32,579 TRACE (transport-thread-NodeA-p29577-t6:) [InvocationContextInterceptor] Invoked with command RemoveCommand{key=IntermediateCompositeKey [taskId=eb7da48a-5922-4671-9037-4077e209744c, key=RedHat], value=null, flags=null, valueMatcher=MATCH_ALWAYS} and InvocationContext [org.infinispan.context.SingleKeyNonTxInvocationContext@c0bbc61]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5083) Hot Rod decoder should use async Cache operations
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5083?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5083:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> Hot Rod decoder should use async Cache operations
> -------------------------------------------------
>
> Key: ISPN-5083
> URL: https://issues.jboss.org/browse/ISPN-5083
> Project: Infinispan
> Issue Type: Enhancement
> Components: Remote Protocols
> Reporter: Galder Zamarreño
> Assignee: Gustavo Fernandes
> Fix For: 8.2.0.CR1
>
>
> Hot Rod decoder is currently tying up Netty threads as a result of calling up to Infinispan sync operations. Instead, Hot Rod decoder should call up async operations, convert the Notifying Futures to Scala Futures, and write up the reply when it's received. This should increase performance specially under heavy load.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5075) org.infinispan.IllegalLifecycleStateException in query tests
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5075?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5075:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> org.infinispan.IllegalLifecycleStateException in query tests
> ------------------------------------------------------------
>
> Key: ISPN-5075
> URL: https://issues.jboss.org/browse/ISPN-5075
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Query
> Affects Versions: 7.1.0.Alpha1
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Fix For: 8.2.0.CR1
>
>
> Many query tests log an error after running:
> {code}
> Exception in thread "Hibernate Search: async deletion of index segments-1" org.infinispan.IllegalLifecycleStateException: ISPN000323: Cache 'LuceneIndexesMetadata' is in 'TERMINATED' state and so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:89)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:77)
> at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:44)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:422)
> at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:427)
> at org.infinispan.lucene.impl.FileListOperations.getFileList(FileListOperations.java:160)
> at org.infinispan.lucene.impl.FileListOperations.deleteFileName(FileListOperations.java:129)
> at org.infinispan.lucene.impl.DirectoryImplementor.deleteFile(DirectoryImplementor.java:64)
> at org.infinispan.lucene.impl.DirectoryLuceneV4$DeleteTask.run(DirectoryLuceneV4.java:195)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> That error does not cause the tests to fail, and are due to the async deletes of segments recently implemented. Need to check if the executors are being flushed correctly when the cache manager stops
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5210) Support writing multiple modifications to cache stores in batch when using Transactions
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5210?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5210:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> Support writing multiple modifications to cache stores in batch when using Transactions
> ---------------------------------------------------------------------------------------
>
> Key: ISPN-5210
> URL: https://issues.jboss.org/browse/ISPN-5210
> Project: Infinispan
> Issue Type: Feature Request
> Components: Loaders and Stores
> Affects Versions: 6.0.2.Final, 7.1.0.Final
> Reporter: Richard Lucas
> Fix For: 8.2.0.CR1
>
>
> Currently writes to a cache store are performed individually for each modification to the cache.
> While this makes sense when using a non-tx cache it would be beneficial to support writing multiple modifications to a cache store in a single call when using a Tx cache.
> Currently all writes are performed in the commit phase and are done by looping through the modifications in the Tx and writing each one in turn to the the store.
> Instead of doing this it would useful to pass all modifications to the store in a single call, this would allow:
> a) Taking advantage of support for batch updates in underlying stores (JDBC, MongoDB, DynamoDB) for a more efficient write through.
> b) Allow store implementations the chance to clean up if one or more updates in the batch fail (this is especially useful if the store does not support Tx and rollbacks as it means the store implementation can at least try to clean up any partial updates it has performed, something which is not currently possible).
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5076) Pessimistic transactions can lose their locks when the primary owner changes
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5076?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5076:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> Pessimistic transactions can lose their locks when the primary owner changes
> ----------------------------------------------------------------------------
>
> Key: ISPN-5076
> URL: https://issues.jboss.org/browse/ISPN-5076
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.0.2.Final, 7.1.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Labels: 7.0
> Fix For: 8.2.0.CR1
>
>
> In a pessimistic cache, if a transaction {{T1}} has a {{put(k, v)}} operation and the primary owner of the key is the originator, the lock is acquired on the originator but it is not replicated to on the backup(s).
> If one of the backup owners becomes the primary owner, it will allow another transaction {{T2}} to lock (and update) key {{k}} before it receives the one-phase prepare command from the originator of {{T1}}.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5482) JSR-107 - Provide mechanism to handle expiration events
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5482?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5482:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> JSR-107 - Provide mechanism to handle expiration events
> --------------------------------------------------------
>
> Key: ISPN-5482
> URL: https://issues.jboss.org/browse/ISPN-5482
> Project: Infinispan
> Issue Type: Feature Request
> Components: JCache
> Reporter: Matej Čimbora
> Assignee: Gustavo Fernandes
> Fix For: 8.2.0.CR1, 8.0.3.Final
>
>
> Currently, expiration events are supported only by embedded mode & additionally require entry to be accessed in order to be created. This can lead to CacheEntryExpiredListener not being notified when an entry expires.
> Note: Not covered by TCK tests.
> Example:
> {code}
> @Test
> public void testExpiration(Method m) {
> Cache<String, String> cache1 = getCache1(m);
> Cache<String, String> cache2 = getCache2(m);
> TestExpiredListener listener = new TestExpiredListener();
> MutableCacheEntryListenerConfiguration conf1 = new MutableCacheEntryListenerConfiguration(FactoryBuilder.factoryOf(listener), null, false, false);
> cache1.registerCacheEntryListener(conf1);
> cache2.put("key1", "val1");
> sleep(5000);
> // Required by embedded JCache implementation to work
> assert cache1.get("key1") == null;
> // Failing for remote JCache implementation
> assertEquals(1, listener.invocationCount);
> }
> private static class TestExpiredListener implements CacheEntryExpiredListener, Serializable {
> private int invocationCount;
> @Override
> public void onExpired(Iterable iterable) throws CacheEntryListenerException {
> Iterator iterator = iterable.iterator();
> while (iterator.hasNext()) {
> iterator.next();
> invocationCount++;
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5476) Cross-site tests should run in parallel
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5476?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5476:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> Cross-site tests should run in parallel
> ---------------------------------------
>
> Key: ISPN-5476
> URL: https://issues.jboss.org/browse/ISPN-5476
> Project: Infinispan
> Issue Type: Task
> Components: Core, Cross-Site Replication, Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Fix For: 8.2.0.CR1
>
>
> Currently the cross-site tests have to run in a single thread, and that means they're much slower than the regular core tests.
> It also means they need to run with a separate Maven profile, and that (combined with their duration) makes it very unlikely for devs to run the xsite tests on their machines.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5475) Narayana should be configured to use a volatile store by default
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5475?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5475:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> Narayana should be configured to use a volatile store by default
> ----------------------------------------------------------------
>
> Key: ISPN-5475
> URL: https://issues.jboss.org/browse/ISPN-5475
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 8.2.0.CR1
>
>
> The {{jbossts-properties.xml}} configuration file in the core module configures a file store by default, and tests have to call {{TestCacheManagerFactory.markAsTransactional()}} (or one of the methods that calls it) to configure a volatile store instead.
> Furthermore, the {{jbossts-properties.xml}} file is explicitly filtered out of the core tests jar, so other modules can't use it.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month
[JBoss JIRA] (ISPN-5470) Remote-executor threads should not block to acquire locks
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5470?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5470:
----------------------------------
Fix Version/s: 8.2.0.CR1
(was: 8.2.0.Beta2)
> Remote-executor threads should not block to acquire locks
> ---------------------------------------------------------
>
> Key: ISPN-5470
> URL: https://issues.jboss.org/browse/ISPN-5470
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 7.2.1.Final
> Reporter: Dan Berindei
> Fix For: 8.2.0.CR1
>
>
> Currently, enabling the queue on the remote-executor thread pool can cause deadlocks, because a CommitCommand/1PCPrepareCommand could end up in the queue while a remote-executor thread is busy waiting to acquire the same lock that this commit would release.
> If trying to acquire a lock would free the thread until the key has been acquired, we could enable the queue on the remote-executor/OOB thread pools, and we would need a lot less threads for the same load.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 1 month