[infinispan-dev] Intermediate cache in M/R API

Radim Vansa rvansa at redhat.com
Tue Mar 18 03:58:15 EDT 2014


What does exactly the configName refer to? Some cache name as it appears 
in configuration, and then the intermediate cache will copy that 
configuration and create new custom cache? That means, that configured 
cache will never be instantiated?

Why don't you use Configuration directly instead, if you want to specify 
configuration?

Radim

On 03/17/2014 03:58 PM, Vladimir Blagojevic wrote:
> Guys,
>
> We need some input on how to design API regarding use of intermediate
> caches [1]. As you might know one of the requirements for improving our
> M/R is allowing applications to use custom defined intermediate
> key/value cache used to store keys/values of map/combine phase before
> being reduced in reduced phase.
>
>
> Currently we have a constructor where one can specify whether to use
> shared or per-task intermediate cache. And now we wanted to add an
> additional method:
>
> usingIntermediateCache(String cacheName, String cacheConfigurationName);
>
> that will enable use of custom intermediate cache.
>
> Now, Dan, and rightly so, thought this was a bit confusing. Are we
> referring to intermediate shared or per-task intermediate cache when
> using the above mentioned method.
>
> His proposal is touse a per-task intermediate cache with our default
> specified intermediate cache configuration. Remove the constructor
> parameter in MapReduceTask regarding shared or non shared cache and add
> configuration methods for both caches:
>
>
>       usingIntermediateCache(String configName) - use a per-task
> intermediate cache with the given configuration
>       usingSharedIntermediateCache(String cache) - use a shared cache
> with our default configuration
>       usingSharedIntermediateCache(String cache, String configName) - use
> a shared cache with the given configuration
>
>
> Note that we need a name for shared cache because we want to enable
> application to easily remove/inspect that cache after all m/r tasks
> sharing that intermediate cache have been executed.
>
> What are your thoughts here?
>
> Vladimir
>
>
>
> [1] https://issues.jboss.org/browse/ISPN-4021
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss DataGrid QA



More information about the infinispan-dev mailing list