[infinispan-dev] Distributed execution framework

[Jeff Ramsdale]

Hi Vladimir,

Rio is really a service deployment/management framework. So, while Rio
doesn't specifically provide a distributed execution framework it
would be an excellent platform on which to build one since, arguably,
a unit of execution is simply a degenerate service--one without a
public interface. In fact, I've done some work around this idea:
http://bitbucket.org/jramsdale/completion-service

My code provides an ExecutorService implementation that accepts work
(i.e. simple Callable and Runnable implementations, though with the
additional requirement that they be Serializable) and uses a JavaSpace
behind the scenes to distribute the work to an arbitrary number of
worker nodes. It even supports remote cancellation. While the unit
test works great for Callables when I actually deploy it my jobs never
seem to escape the JavaSpace (i.e. the workers don't see the work).
Haven't figured out what's going on there yet. But the code should
give you an idea of one way to implement Master/Worker (a distributed
execution pattern that predates Map/Reduce) using a JavaSpace. There's
still more work to do: adding transactions, timeouts, and lease
management, but these are marked with comments as placeholders.

The framework is designed to allow subclassing of the BasicWorker to
add Infinispan-awareness. This would allow an InfinispanWorker to
identify that it should only pick up InfinispanTasks, for instance.
Rio, at deployment time, could ensure that InfinispanWorkers only get
deployed to Cybernodes that contain Infinispan cluster members, if so
desired. These cluster members could be injected into the workers as
service dependencies so that there is no remote call--it's all in-JVM
and handled by Rio's support for service associations. (I haven't
thought through how to do partitioned data affinity, but that seems
less difficult than many of the other problems involved in distributed
execution.)

Even if you don't use my code (which is far from production ready), or
the Master/Worker pattern, Rio could provide service provisioning,
association, monitoring, SLAs, etc. to your solution. Creating a
Worker that has Infinispan injected as a required dependency would be
easy in Rio and keeps the work-doing concern separate from
Infinispan's own data grid concerns. In fact, refactoring Infinispan's
server implementations to be Rio services would also be powerful. They
could come and go, be upgraded, etc., without the data grid itself
being affected since they would simply be collocated services.

I appreciate the questions, glad to answer as many as I can!

-jeff

On Mon, Jul 26, 2010 at 12:53 PM, Vladimir Blagojevic
<vblagoje at redhat.com> wrote:
> Jeff,
>
> Apologies for late response. I had to read up on Rio project a bit:) Correct me if I am wrong but it seems to me that Rio project has different objectives than what we are trying to develop - distributed execution framework.
>
> It seems that, as Manik wrote to you, the work you have done is related to managing Infinispan as a service?
>
> Regards,
> Vladimir
> On 2010-07-23, at 1:11 PM, Jeff Ramsdale wrote:
>
>> Hi Vladimir,
>>
>> It's been a small dream of mine to integrate Infinispan and Rio
>> (http://www.rio-project.org/), which is built on Jini and is a mature
>> platform for building and deploying scalable, dynamic, distributed
>> applications, including service deployment, instrumentation, and SLAs.
>> A first step would be to deploy Infinispan itself to run on Rio's
>> framework (I've had some preliminary success with this). In itself
>> this would be a powerful combination as Rio could potentially use
>> Infinispan's exposed instrumentation to dynamically reshape the
>> cluster to respond to load. Additionally, through its ability to
>> manage the deployment of services it could handle all the distribution
>> of work to Infinispan nodes should there be occasion to "move the code
>> to the data rather than the data to the code."
>>
>> The Infinispan community could write these things from scratch, but
>> Jini and Rio have already solved many of the difficult issues around
>> managing compute grids but they don't provide a data grid. Infinispan
>> as a data grid fits with Rio very well indeed. Rio is Apache 2.0
>> licensed. The Jini code previously owned by Sun has been donated to
>> become Apache River:
>> http://incubator.apache.org/river/RIVER/index.html and, of course, is
>> also Apache 2.0 licensed. It might be of interest to this community
>> that recent work has resulted in some integration between Drools and
>> Rio to provide distributed service policy handler enforcement. See the
>> Rio source for a running example (with UI).
>>
>> Another interesting project (built on the platform suggested above)
>> would be to create an Infinispan-backed JavaSpaces implementation, but
>> that's a post for another time...
>>
>> -jeff
>>
>> On Wed, Jul 21, 2010 at 10:42 AM, Vladimir Blagojevic
>> <vblagoje at redhat.com> wrote:
>>> Hi,
>>>
>>> I wanted to gather feedback from community regarding proposed scope and objectives of Distributed execution framework targeted for Infinispan 5.0. The document is a gentle introduction to the subject of distributed execution and as such is accessible to wider audience. Eager to hear your thoughts.
>>>
>>> Regards,
>>> Vladimir
>>>
>>> https://community.jboss.org/wiki/InfinispanDistributedExecutionFramework
>>>
>>>
>>> --
>>> Vladimir Blagojevic
>>> JBoss Clustering Team
>>> JBoss, by Red Hat
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Vladimir Blagojevic
> JBoss Clustering Team
> JBoss, by Red Hat
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>