July 2014 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-4217) Use ArrayList instead of LinkedList for DeltaAware and Delta in M/R

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4217?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration updated ISPN-4217: ------------------------------------------ Bugzilla Update: Perform Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979 > Use ArrayList instead of LinkedList for DeltaAware and Delta in M/R > ------------------------------------------------------------------- > > Key: ISPN-4217 > URL: https://issues.jboss.org/browse/ISPN-4217 > Project: Infinispan > Issue Type: Enhancement > Security Level: Public(Everyone can see) > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha5 > > > Investigate switching to ArrayList as implementation of DeltaAware and Delta as ArrayList uses less memory than LinkedList. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4453) MapReduceTask#executeAsynchronously() isn't asynchronous

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4453?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration updated ISPN-4453: ------------------------------------------ Bugzilla Update: Perform Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979 > MapReduceTask#executeAsynchronously() isn't asynchronous > -------------------------------------------------------- > > Key: ISPN-4453 > URL: https://issues.jboss.org/browse/ISPN-4453 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Affects Versions: 6.0.2.Final > Reporter: Rich DiCroce > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha5 > > > Quote from the linked forum thread: > {quote} > MapReduceTask#executeAsynchronously() doesn't actually do anything asynchronously. It just returns a MapReduceTaskFuture containing a Callable that calls execute(). The only place I see that Callable being called is in MapReduceTaskFuture#get(). > > In other words, the task isn't actually started until you call Future#get(), which isn't asynchronous at all! On top of that, MapReduceTaskFuture extends AbstractInProcessFuture, which "implements" the get(long, TimeUnit) method by just calling get(). Which means that MapReduceTaskFuture doesn't respect the timeout parameters and will just run until it completes. > {quote} -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4440) Remove setMaxCollectorSize API from MapReduceTask

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4440?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration updated ISPN-4440: ------------------------------------------ Bugzilla Update: Perform Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979 > Remove setMaxCollectorSize API from MapReduceTask > ------------------------------------------------- > > Key: ISPN-4440 > URL: https://issues.jboss.org/browse/ISPN-4440 > Project: Infinispan > Issue Type: Task > Security Level: Public(Everyone can see) > Components: Distributed Execution and Map/Reduce > Affects Versions: 7.0.0.Alpha4 > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Priority: Minor > Fix For: 7.0.0.Alpha5 > > > During the refinement of parallel execution of M/R algorithm we introduced an abstraction maxCollectorSize on the level of MapReduceTask. The ideas was that during execution of map/combine phase, number of intermediate keys/values collected in a Collector could potentially become very large. By limiting size of collector, intermediate key/values are transferred to intermediate cache in batches before reduce phase is executed and OutOfMemoryError issues are avoided as well. > However, during the extensive performance phase Alan Field, Dan Berindei and I have concluded that maxCollectorSize set to 10000 entries gives the best trade off between performance and memory use. Therefore there is no need to expose this value to MapReduceTask users. > Having said that there might be some uses cases where holding 10000 intermediate large memory footprint objects might lead to OOM, and in such cases users should allocate more heap to MapReduceTasks. We might consider introducing again this API should such a need arise. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4472) Upgrade TestNG from 6.8 to 6.8.8

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4472?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration updated ISPN-4472: ------------------------------------------ Bugzilla Update: Perform Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979 > Upgrade TestNG from 6.8 to 6.8.8 > -------------------------------- > > Key: ISPN-4472 > URL: https://issues.jboss.org/browse/ISPN-4472 > Project: Infinispan > Issue Type: Task > Security Level: Public(Everyone can see) > Affects Versions: 7.0.0.Alpha4 > Reporter: Ion Savin > Assignee: Ion Savin > Fix For: 7.0.0.Alpha5 > > > Versions starting from 6.8.1 should fix OOME related to TestNG report generation when there's a bit more test output. > Related issue: > https://github.com/cbeust/testng/issues/291 -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4486) Map/Reduce DefaultCollector needs to be synchronized in certain use cases

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4486?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4486: ----------------------------------------------- Dan Berindei <dberinde(a)redhat.com> changed the Status of [bug 1116979|https://bugzilla.redhat.com/show_bug.cgi?id=1116979] from NEW to POST > Map/Reduce DefaultCollector needs to be synchronized in certain use cases > ------------------------------------------------------------------------- > > Key: ISPN-4486 > URL: https://issues.jboss.org/browse/ISPN-4486 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha5 > > > We have found a random test failure Dan traced to a particular use of DefaultCollector in MapReduceManagerImpl. We did not need synchronization in all other cases but in this one we did. See github PR for a proposed solution -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4471) MapReduceTask: memory leak with useIntermediateSharedCache = true

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4471?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration updated ISPN-4471: ------------------------------------------ Bugzilla Update: Perform Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979 > MapReduceTask: memory leak with useIntermediateSharedCache = true > ----------------------------------------------------------------- > > Key: ISPN-4471 > URL: https://issues.jboss.org/browse/ISPN-4471 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Distributed Execution and Map/Reduce > Affects Versions: 6.0.2.Final > Reporter: Rich DiCroce > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha5 > > > When using an intermediate shared cache for the reduce phase, MapReduceTask puts the entries into the cache with no expiration and apparently never removes them. This eventually results in OutOfMemoryErrors. > One workaround is to disable use of the intermediate shared cache, which causes a new cache to be created and destroyed for every task, which "fixes" the problem of not removing intermediate values. However, it causes a ton of log spam: > {noformat} > 2014-07-02 11:55:10,014 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-21) ISPN000031: MBeans were successfully registered to the platform MBean server. > 2014-07-02 11:55:10,016 INFO [org.jboss.as.clustering.infinispan] (transport-thread-21) JBAS010281: Started e71dddc0-60ce-4cb9-ac8c-615d60866393 cache from GamingPortal container > 2014-07-02 11:55:10,023 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-5) ISPN000031: MBeans were successfully registered to the platform MBean server. > 2014-07-02 11:55:10,024 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-4) ISPN000031: MBeans were successfully registered to the platform MBean server. > 2014-07-02 11:55:10,025 INFO [org.jboss.as.clustering.infinispan] (transport-thread-5) JBAS010281: Started 22d387d6-69c6-48b2-9701-ea64c08d66ad cache from GamingPortal container > 2014-07-02 11:55:10,026 INFO [org.jboss.as.clustering.infinispan] (transport-thread-4) JBAS010281: Started bfaf92a0-a030-4624-93a7-0fee097415d7 cache from NMS container > 2014-07-02 11:55:10,037 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 22d387d6-69c6-48b2-9701-ea64c08d66ad cache from GamingPortal container > 2014-07-02 11:55:10,040 INFO [org.jboss.as.clustering.infinispan] (EJB default - 1) JBAS010282: Stopped bfaf92a0-a030-4624-93a7-0fee097415d7 cache from NMS container > 2014-07-02 11:55:10,047 INFO [org.jboss.as.clustering.infinispan] (EJB default - 6) JBAS010282: Stopped e71dddc0-60ce-4cb9-ac8c-615d60866393 cache from GamingPortal container > 2014-07-02 11:55:10,047 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-0) ISPN000031: MBeans were successfully registered to the platform MBean server. > 2014-07-02 11:55:10,048 INFO [org.jboss.as.clustering.infinispan] (transport-thread-0) JBAS010281: Started bed74bd3-a227-43e0-b262-62c19dd444a7 cache from GamingPortal container > 2014-07-02 11:55:10,052 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped bed74bd3-a227-43e0-b262-62c19dd444a7 cache from GamingPortal container > 2014-07-02 11:55:10,063 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-7) ISPN000031: MBeans were successfully registered to the platform MBean server. > 2014-07-02 11:55:10,064 INFO [org.jboss.as.clustering.infinispan] (transport-thread-7) JBAS010281: Started 63cce570-0169-40c2-bc9f-e045c2864702 cache from GamingPortal container > 2014-07-02 11:55:10,068 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 63cce570-0169-40c2-bc9f-e045c2864702 cache from GamingPortal container > 2014-07-02 11:55:10,072 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-19) ISPN000031: MBeans were successfully registered to the platform MBean server. > 2014-07-02 11:55:10,073 INFO [org.jboss.as.clustering.infinispan] (transport-thread-19) JBAS010281: Started 83f7b355-d4c6-4a0a-aade-ce2509293d77 cache from GamingPortal container > 2014-07-02 11:55:10,077 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 83f7b355-d4c6-4a0a-aade-ce2509293d77 cache from GamingPortal container > {noformat} > I also observed one NullPointerException with distributeReducePhase = true and useIntermediateSharedCache = false. This could be related to ISPN-4460, but I'm not sure. > {noformat} > Caused by: org.infinispan.commons.CacheException: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.NullPointerException > at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:348) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:634) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask$3.call(MapReduceTask.java:652) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask$MapReduceTaskFuture.get(MapReduceTask.java:760) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > ... 63 more > Caused by: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.NullPointerException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_45] > at org.infinispan.distexec.mapreduce.MapReduceTask$TaskPart.get(MapReduceTask.java:845) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhase(MapReduceTask.java:439) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:342) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > ... 66 more > Caused by: org.infinispan.commons.CacheException: java.lang.NullPointerException > at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:100) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.invokeMapCombineLocally(MapReduceTask.java:967) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.access$200(MapReduceTask.java:894) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:916) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:912) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45] > Caused by: java.lang.NullPointerException > at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapKeysToNodes(MapReduceManagerImpl.java:355) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.migrateIntermediateKeys(MapReduceManagerImpl.java:264) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:258) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final] > ... 10 more > {noformat} -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4486) Map/Reduce DefaultCollector needs to be synchronized in certain use cases

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4486?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration updated ISPN-4486: ------------------------------------------ Bugzilla Update: Perform Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979 > Map/Reduce DefaultCollector needs to be synchronized in certain use cases > ------------------------------------------------------------------------- > > Key: ISPN-4486 > URL: https://issues.jboss.org/browse/ISPN-4486 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha5 > > > We have found a random test failure Dan traced to a particular use of DefaultCollector in MapReduceManagerImpl. We did not need synchronization in all other cases but in this one we did. See github PR for a proposed solution -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4486) Map/Reduce DefaultCollector needs to be synchronized in certain use cases

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-4486?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-4486: ------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done > Map/Reduce DefaultCollector needs to be synchronized in certain use cases > ------------------------------------------------------------------------- > > Key: ISPN-4486 > URL: https://issues.jboss.org/browse/ISPN-4486 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha5 > > > We have found a random test failure Dan traced to a particular use of DefaultCollector in MapReduceManagerImpl. We did not need synchronization in all other cases but in this one we did. See github PR for a proposed solution -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4486) Map/Reduce DefaultCollector needs to be synchronized in certain use cases

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-4486?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-4486: ------------------------------- Status: Pull Request Sent (was: Open) > Map/Reduce DefaultCollector needs to be synchronized in certain use cases > ------------------------------------------------------------------------- > > Key: ISPN-4486 > URL: https://issues.jboss.org/browse/ISPN-4486 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha5 > > > We have found a random test failure Dan traced to a particular use of DefaultCollector in MapReduceManagerImpl. We did not need synchronization in all other cases but in this one we did. See github PR for a proposed solution -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4487) Force OSGi test execution order

by Ion Savin (JIRA)

Ion Savin created ISPN-4487: ------------------------------- Summary: Force OSGi test execution order Key: ISPN-4487 URL: https://issues.jboss.org/browse/ISPN-4487 Project: Infinispan Issue Type: Task Security Level: Public (Everyone can see) Reporter: Ion Savin Assignee: Ion Savin Related to this issue: https://ops4j1.jira.com/browse/PAXEXAM-639 Temporarily force the tests to execute in alphabetical order (guarantees the PerClass test doesn't execute first). -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 6 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues July 2014