[JBoss JIRA] (ISPN-4453) MapReduceTask#executeAsynchronously() isn't asynchronous
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4453?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-4453:
------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979
> MapReduceTask#executeAsynchronously() isn't asynchronous
> --------------------------------------------------------
>
> Key: ISPN-4453
> URL: https://issues.jboss.org/browse/ISPN-4453
> Project: Infinispan
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Affects Versions: 6.0.2.Final
> Reporter: Rich DiCroce
> Assignee: Vladimir Blagojevic
> Fix For: 7.0.0.Alpha5
>
>
> Quote from the linked forum thread:
> {quote}
> MapReduceTask#executeAsynchronously() doesn't actually do anything asynchronously. It just returns a MapReduceTaskFuture containing a Callable that calls execute(). The only place I see that Callable being called is in MapReduceTaskFuture#get().
>
> In other words, the task isn't actually started until you call Future#get(), which isn't asynchronous at all! On top of that, MapReduceTaskFuture extends AbstractInProcessFuture, which "implements" the get(long, TimeUnit) method by just calling get(). Which means that MapReduceTaskFuture doesn't respect the timeout parameters and will just run until it completes.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 12 months
[JBoss JIRA] (ISPN-4440) Remove setMaxCollectorSize API from MapReduceTask
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4440?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-4440:
------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979
> Remove setMaxCollectorSize API from MapReduceTask
> -------------------------------------------------
>
> Key: ISPN-4440
> URL: https://issues.jboss.org/browse/ISPN-4440
> Project: Infinispan
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 7.0.0.Alpha4
> Reporter: Vladimir Blagojevic
> Assignee: Vladimir Blagojevic
> Priority: Minor
> Fix For: 7.0.0.Alpha5
>
>
> During the refinement of parallel execution of M/R algorithm we introduced an abstraction maxCollectorSize on the level of MapReduceTask. The ideas was that during execution of map/combine phase, number of intermediate keys/values collected in a Collector could potentially become very large. By limiting size of collector, intermediate key/values are transferred to intermediate cache in batches before reduce phase is executed and OutOfMemoryError issues are avoided as well.
> However, during the extensive performance phase Alan Field, Dan Berindei and I have concluded that maxCollectorSize set to 10000 entries gives the best trade off between performance and memory use. Therefore there is no need to expose this value to MapReduceTask users.
> Having said that there might be some uses cases where holding 10000 intermediate large memory footprint objects might lead to OOM, and in such cases users should allocate more heap to MapReduceTasks. We might consider introducing again this API should such a need arise.
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 12 months
[JBoss JIRA] (ISPN-4471) MapReduceTask: memory leak with useIntermediateSharedCache = true
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4471?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-4471:
------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1116979
> MapReduceTask: memory leak with useIntermediateSharedCache = true
> -----------------------------------------------------------------
>
> Key: ISPN-4471
> URL: https://issues.jboss.org/browse/ISPN-4471
> Project: Infinispan
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 6.0.2.Final
> Reporter: Rich DiCroce
> Assignee: Vladimir Blagojevic
> Fix For: 7.0.0.Alpha5
>
>
> When using an intermediate shared cache for the reduce phase, MapReduceTask puts the entries into the cache with no expiration and apparently never removes them. This eventually results in OutOfMemoryErrors.
> One workaround is to disable use of the intermediate shared cache, which causes a new cache to be created and destroyed for every task, which "fixes" the problem of not removing intermediate values. However, it causes a ton of log spam:
> {noformat}
> 2014-07-02 11:55:10,014 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-21) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,016 INFO [org.jboss.as.clustering.infinispan] (transport-thread-21) JBAS010281: Started e71dddc0-60ce-4cb9-ac8c-615d60866393 cache from GamingPortal container
> 2014-07-02 11:55:10,023 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-5) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,024 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-4) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,025 INFO [org.jboss.as.clustering.infinispan] (transport-thread-5) JBAS010281: Started 22d387d6-69c6-48b2-9701-ea64c08d66ad cache from GamingPortal container
> 2014-07-02 11:55:10,026 INFO [org.jboss.as.clustering.infinispan] (transport-thread-4) JBAS010281: Started bfaf92a0-a030-4624-93a7-0fee097415d7 cache from NMS container
> 2014-07-02 11:55:10,037 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 22d387d6-69c6-48b2-9701-ea64c08d66ad cache from GamingPortal container
> 2014-07-02 11:55:10,040 INFO [org.jboss.as.clustering.infinispan] (EJB default - 1) JBAS010282: Stopped bfaf92a0-a030-4624-93a7-0fee097415d7 cache from NMS container
> 2014-07-02 11:55:10,047 INFO [org.jboss.as.clustering.infinispan] (EJB default - 6) JBAS010282: Stopped e71dddc0-60ce-4cb9-ac8c-615d60866393 cache from GamingPortal container
> 2014-07-02 11:55:10,047 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-0) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,048 INFO [org.jboss.as.clustering.infinispan] (transport-thread-0) JBAS010281: Started bed74bd3-a227-43e0-b262-62c19dd444a7 cache from GamingPortal container
> 2014-07-02 11:55:10,052 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped bed74bd3-a227-43e0-b262-62c19dd444a7 cache from GamingPortal container
> 2014-07-02 11:55:10,063 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-7) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,064 INFO [org.jboss.as.clustering.infinispan] (transport-thread-7) JBAS010281: Started 63cce570-0169-40c2-bc9f-e045c2864702 cache from GamingPortal container
> 2014-07-02 11:55:10,068 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 63cce570-0169-40c2-bc9f-e045c2864702 cache from GamingPortal container
> 2014-07-02 11:55:10,072 INFO [org.infinispan.jmx.CacheJmxRegistration] (transport-thread-19) ISPN000031: MBeans were successfully registered to the platform MBean server.
> 2014-07-02 11:55:10,073 INFO [org.jboss.as.clustering.infinispan] (transport-thread-19) JBAS010281: Started 83f7b355-d4c6-4a0a-aade-ce2509293d77 cache from GamingPortal container
> 2014-07-02 11:55:10,077 INFO [org.jboss.as.clustering.infinispan] (EJB default - 2) JBAS010282: Stopped 83f7b355-d4c6-4a0a-aade-ce2509293d77 cache from GamingPortal container
> {noformat}
> I also observed one NullPointerException with distributeReducePhase = true and useIntermediateSharedCache = false. This could be related to ISPN-4460, but I'm not sure.
> {noformat}
> Caused by: org.infinispan.commons.CacheException: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:348) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:634) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$3.call(MapReduceTask.java:652) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapReduceTaskFuture.get(MapReduceTask.java:760) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> ... 63 more
> Caused by: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_45]
> at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_45]
> at org.infinispan.distexec.mapreduce.MapReduceTask$TaskPart.get(MapReduceTask.java:845) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhase(MapReduceTask.java:439) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:342) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> ... 66 more
> Caused by: org.infinispan.commons.CacheException: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:100) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.invokeMapCombineLocally(MapReduceTask.java:967) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart.access$200(MapReduceTask.java:894) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:916) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceTask$MapTaskPart$1.call(MapReduceTask.java:912) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
> Caused by: java.lang.NullPointerException
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapKeysToNodes(MapReduceManagerImpl.java:355) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.migrateIntermediateKeys(MapReduceManagerImpl.java:264) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:258) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]
> ... 10 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 12 months
[JBoss JIRA] (ISPN-4311) Security tests, Windows, access denied (java.security.SecurityPermission setPolicy)
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4311?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4311:
-----------------------------------------------
Jiri Holusa <jholusa(a)redhat.com> changed the Status of [bug 1100259|https://bugzilla.redhat.com/show_bug.cgi?id=1100259] from ON_QA to VERIFIED
> Security tests, Windows, access denied (java.security.SecurityPermission setPolicy)
> -----------------------------------------------------------------------------------
>
> Key: ISPN-4311
> URL: https://issues.jboss.org/browse/ISPN-4311
> Project: Infinispan
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Security
> Affects Versions: 7.0.0.Alpha4
> Reporter: Tomas Sykora
> Assignee: Tristan Tarrant
> Labels: 63gablocker, testsuite_stability
> Fix For: 7.0.0.Alpha5
>
>
> A pile of Security tests has problems with denied access when running the test suite on Windows environment.
> Stacktrace
> java.security.AccessControlException: access denied (javax.security.auth.AuthPermission doAs)
> at java.security.AccessControlContext.checkPermission(AccessControlContext.java:374)
> at java.security.AccessController.checkPermission(AccessController.java:549)
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
> at javax.security.auth.Subject.doAs(Subject.java:326)
> at org.infinispan.security.QueryAuthorizationTest.createCacheManager(QueryAuthorizationTest.java:44)
> at org.infinispan.test.SingleCacheManagerTest.setup(SingleCacheManagerTest.java:31)
> at org.infinispan.test.SingleCacheManagerTest.createBeforeClass(SingleCacheManagerTest.java:44)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:564)
> at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:213)
> at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:138)
> at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:175)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:107)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> *Affected tests:*
> org.infinispan.security.ExecutionAuthorizationTest.testExecMapReduce
> org.infinispan.security.QueryAuthorizationTest.createBeforeClass
> org.infinispan.security.ExecutionAuthorizationTest.testExecDistExec
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
9 years, 12 months