[infinispan-dev] Intermediate cache in M/R API

[Dan Berindei]

On Tue, Mar 18, 2014 at 9:58 AM, Radim Vansa <rvansa at redhat.com> wrote:

> What does exactly the configName refer to? Some cache name as it appears
> in configuration, and then the intermediate cache will copy that
> configuration and create new custom cache? That means, that configured
> cache will never be instantiated?
>

Yes. Only usingSharedIntermediateCache(
String cache) uses the same cache name and config name.


>
> Why don't you use Configuration directly instead, if you want to specify
> configuration?
>

Two reasons:
1. The Configuration classes are not serializable.
2. Some users might prefer to define the configuration via XML.

Dan



>
> Radim
>
> On 03/17/2014 03:58 PM, Vladimir Blagojevic wrote:
> > Guys,
> >
> > We need some input on how to design API regarding use of intermediate
> > caches [1]. As you might know one of the requirements for improving our
> > M/R is allowing applications to use custom defined intermediate
> > key/value cache used to store keys/values of map/combine phase before
> > being reduced in reduced phase.
> >
> >
> > Currently we have a constructor where one can specify whether to use
> > shared or per-task intermediate cache. And now we wanted to add an
> > additional method:
> >
> > usingIntermediateCache(String cacheName, String cacheConfigurationName);
> >
> > that will enable use of custom intermediate cache.
> >
> > Now, Dan, and rightly so, thought this was a bit confusing. Are we
> > referring to intermediate shared or per-task intermediate cache when
> > using the above mentioned method.
> >
> > His proposal is touse a per-task intermediate cache with our default
> > specified intermediate cache configuration. Remove the constructor
> > parameter in MapReduceTask regarding shared or non shared cache and add
> > configuration methods for both caches:
> >
> >
> >       usingIntermediateCache(String configName) - use a per-task
> > intermediate cache with the given configuration
> >       usingSharedIntermediateCache(String cache) - use a shared cache
> > with our default configuration
> >       usingSharedIntermediateCache(String cache, String configName) - use
> > a shared cache with the given configuration
> >
> >
> > Note that we need a name for shared cache because we want to enable
> > application to easily remove/inspect that cache after all m/r tasks
> > sharing that intermediate cache have been executed.
> >
> > What are your thoughts here?
> >
> > Vladimir
> >
> >
> >
> > [1] https://issues.jboss.org/browse/ISPN-4021
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss DataGrid QA
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140318/282660e0/attachment.html