[infinispan-dev] Distributed tasks - specifying task input

[Manik Surtani]

On 17 Dec 2010, at 11:56, Vladimir Blagojevic wrote:

> On 10-12-17 8:42 AM, Manik Surtani wrote:
>> 
>> I was just wondering whether such a use case exists or whether we're 
>> just inventing stuff.  :-)  It would lead to a much more cumbersome 
>> API since you'd need to provide a map of cache names to keys, etc.
> 
> Ok agreed. Unless there is a way to solve this nicely lets go with cache 
> based DT. I'll think more about this over the weekend.

Hmm, just saw Sanne's message.  Ok, so if we were to support multiple caches... how could this look?

interface DistributedTask<K, V, R> {

   Collection<String> cacheNames();

   Collection<K> keys(String cache);

   void map(String cacheName, Map.Entry<K, V> e, DistributedTaskContext <K, V, R> ctx);

   R reduce(Map<Address, DistributedTaskContext<K, V, R>> contexts);

}

interface DistributedTaskContext<K, V, R> {
   
   String getCacheName();
  
   // ... map-like methods on K, V
}

DistributedForkJoinPool p = cacheManager.newDistributedForkJoinPool();

result = p.invoke(new DistributedTask() { ... });

and let the DistributedTask impl provide the names of caches and keys as needed?  Some semantics - a null collection of cache names = all caches, a null collection of keys = all keys?  Or is that a bit shit?  :-)

Cheers
Manik

> 
> Also, why would you call DT#map for each key specified? I am thinking it 
> should be called once when task is migrated to execution node, context 
> should be filled with entries on that cache node and passed to map. So 
> call map once per node with all keys already in context.

Just saves the implementation having to iterate over the collection.  We do the iterating and callback each time.  :-)  Minor point.

> I'd also try to use our own interfaces here rather than pure Map so we 
> can evolve them if we need to - DistributedTaskContext or whatever.....

+1.

--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org