[infinispan-issues] [JBoss JIRA] (ISPN-4471) MapReduceTask: memory leak with useIntermediateSharedCache = true
Pedro Ruivo (JIRA)
issues at jboss.org
Mon Jul 7 07:26:24 EDT 2014
[ https://issues.jboss.org/browse/ISPN-4471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pedro Ruivo resolved ISPN-4471.
-------------------------------
Fix Version/s: 7.0.0.Alpha5
Resolution: Done
> MapReduceTask: memory leak with useIntermediateSharedCache = true
> -----------------------------------------------------------------
>
> Key: ISPN-4471
> URL: https://issues.jboss.org/browse/ISPN-4471
> Project: Infinispan
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 6.0.2.Final
> Reporter: Rich DiCroce
> Assignee: Vladimir Blagojevic
> Fix For: 7.0.0.Alpha5
>
>
> When using an intermediate shared cache for the reduce phase, MapReduceTask puts the entries into the cache with no expiration and apparently never removes them. This eventually results in OutOfMemoryErrors.
> One workaround is to disable use of the intermediate shared cache, which causes a new cache to be created and destroyed for every task, which "fixes" the problem of not removing intermediate values. However, it causes a ton of log spam:
> {noformat}
> 2014-07-02 11:55:10,014 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-21) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,016 INFO [org.jboss.as.clustering.infinispan] (transport-thread-21) JBAS010281: Started e71dddc0-60ce-4cb9-ac8c-615d60866393 cache from GamingPortal container
> 2014-07-02 11:55:10,023 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-5) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,024 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-4) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,025 INFO [org.jboss.as.clustering.infinispan] (transport-thread-5) JBAS010281: Started 22d387d6-69c6-48b2-9701-ea64c08d66ad cache from GamingPortal container
> 2014-07-02 11:55:10,026 INFO [org.jboss.as.clustering.infinispan] (transport-thread-4) JBAS010281: Started bfaf92a0-a030-4624-93a7-0fee097415d7 cache from NMS container
> 2014-07-02 11:55:10,037 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 22d387d6-69c6-48b2-9701-ea64c08d66ad cache from GamingPortal container
> 2014-07-02 11:55:10,040 INFO [org.jboss.as.clustering.infinispan] (EJB default - 1) JBAS010282: Stopped bfaf92a0-a030-4624-93a7-0fee097415d7 cache from NMS container
> 2014-07-02 11:55:10,047 INFO [org.jboss.as.clustering.infinispan] (EJB default - 6) JBAS010282: Stopped e71dddc0-60ce-4cb9-ac8c-615d60866393 cache from GamingPortal container
> 2014-07-02 11:55:10,047 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-0) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,048 INFO [org.jboss.as.clustering.infinispan] (transport-thread-0) JBAS010281: Started bed74bd3-a227-43e0-b262-62c19dd444a7 cache from GamingPortal container
> 2014-07-02 11:55:10,052 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped bed74bd3-a227-43e0-b262-62c19dd444a7 cache from GamingPortal container
> 2014-07-02 11:55:10,063 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-7) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,064 INFO [org.jboss.as.clustering.infinispan] (transport-thread-7) JBAS010281: Started 63cce570-0169-40c2-bc9f-e045c2864702 cache from GamingPortal container
> 2014-07-02 11:55:10,068 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 63cce570-0169-40c2-bc9f-e045c2864702 cache from GamingPortal container
> 2014-07-02 11:55:10,072 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-19) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,073 INFO [org.jboss.as.clustering.infinispan] (transport-thread-19) JBAS010281: Started 83f7b355-d4c6-4a0a-aade-ce2509293d77 cache from GamingPortal container
> 2014-07-02 11:55:10,077 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 83f7b355-d4c6-4a0a-aade-ce2509293d77 cache from GamingPortal container
> {noformat}
> I also observed one NullPointerException with distributeReducePhase = true and useIntermediateSharedCache = false. This could be related to ISPN-4460, but I'm not sure.
> {noformat}
> Caused by: org.infinispan.commons.CacheException: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:348) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:634) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$3.call(MapReduceTask.java:652) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapReduceTaskFuture.get(MapReduceTask.java:760) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> ... 63 more
> Caused by: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_45]
> at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_45]
> at org.infinispan.distexec.mapreduce.MapReduceTask$TaskPart.get(MapReduceTask.java:845) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhase(MapReduceTask.java:439) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:342) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> ... 66 more
> Caused by: org.infinispan.commons.CacheException: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:100) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.invokeMapCombineLocally(MapReduceTask.java:967) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.access$200(MapReduceTask.java:894) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:916) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:912) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
> Caused by: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapKeysToNodes(MapReduceManagerImpl.java:355) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.migrateIntermediateKeys(MapReduceManagerImpl.java:264) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:258) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> ... 10 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
More information about the infinispan-issues
mailing list