[infinispan-issues] [JBoss JIRA] (ISPN-2192) Allow using Distributed Execution against any cache (not just clustered caches)

[Randall Hauch (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712690#comment-12712690 ] 

Randall Hauch edited comment on ISPN-2192 at 8/21/12 9:04 AM:
--------------------------------------------------------------

Yes, I originally suggested keeping/using the {{Distributed*}} interfaces because they actually work semantically for all modes - the only thing "distributed" about each of them is simply the name. The "submitEverywhere(...)" is generic enough that I think that'd make sense even in a local cache.

However, it may make more sense in the long run to define new interfaces that are named independently of the mode. For example:

{code:java}
Cache<K, V> cache = ...
CacheCallable<K,V,R> callable = ...
CacheExecutorService executor = cache.getExecutor();
List<Future<R>> futures = executor.submitEverywhere(callable);
{code}

Notice how Infinispan should IMO provide a way to obtain a proper {{CacheExecutorService}} implementation based upon the configuration. The example above obtains it from the cache (which IMO would be ideal), though I can see a factory or utility class being useful here. Of course, I could still use my own {{CacheExecutorService}} if I knew which one would work best.

Really, this is not very different than using {{DistributedCallable}} and {{DistributedExecutorService}} - just the names have changed slightly. Whether this is important is up to the Infinispan team.

BTW, I think that the contract should be that the calling code always has to be prepared to handle multiple Future<R> instances and be able to combine multiple {{R}} results. Even though a naive local executor might only return one, an advanced local executor implementation could parallelize the work and return multiple.
                
      was (Author: rhauch):
    Well, ideally I could execute code like this on any cache:

{code:java}
Cache<K, V> cache = ...
CacheCallable<K,V,R> callable = ...
CacheExecutorService executor = cache.getExecutor(); // or perhaps from a utility or factory, 
                                                     // but something that Infinispan provides
List<Future<R>> futures = executor.submitEverywhere(callable);
{code}

Of course, I could still use my own {{CacheExecutorService}} if I knew which one would work best.

I'm not sure this is terribly different than using {{DistributedCallable}} and {{DistributedExecutorService}}, since those interfaces actually don't have anything specific to being "distributed" other than their names (which is why I originally suggested simply reusing them as-is). Perhaps if {{Cache*}} interfaces will be created, the {{Distributed*}} interfaces could extend the newer {{Cache*}} interfaces. WDYT?
                  
> Allow using Distributed Execution against any cache (not just clustered caches)
> -------------------------------------------------------------------------------
>
>                 Key: ISPN-2192
>                 URL: https://issues.jboss.org/browse/ISPN-2192
>             Project: Infinispan
>          Issue Type: Enhancement
>    Affects Versions: 5.1.2.FINAL
>            Reporter: Randall Hauch
>            Assignee: Manik Surtani
>            Priority: Critical
>
> It is often desirable to process all entries in a cache, and there is no reliable and consistent way to do that for any cache. 
> If a cache is clustered, then map-reduce can be used (although IIUC it currently doesn't guarantee to process non-materialized entries for caches that have a cache store) and distributed execution can be used (IIUC this will be called for all keys owned by the cluster node). However, neither of these work on local caches.
> If a cache is local and there is no cache store, then apparently {{keySet()}} does work and allows a client to iterate over the keys in the local cache. However, if the local cache is configured to have a cache store, then the {{keySet()}} method will return only the keys for the materialized entries and does not return all of the keys in the cache. Of course, if a local cache does have a cache store, then its possible to obtain the keys using {{CacheLoader.loadAllKeys(...)}} (obviously this is probably not something a client should do).
> In short, it should be possible to use Distributed Execution against any cache, regardless of its configuration. (Ideally, it would be possible for Map-Reduce to also be used against any cache.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira