[infinispan-dev] Infinispan - Hadoop integration
Mircea Markus
mmarkus at redhat.com
Thu Mar 13 18:19:14 EDT 2014
On Mar 13, 2014, at 22:17, Sanne Grinovero <sanne at infinispan.org> wrote:
> On 13 March 2014 22:05, Mircea Markus <mmarkus at redhat.com> wrote:
>>
>> On Mar 13, 2014, at 20:59, Ales Justin <ales.justin at gmail.com> wrote:
>>
>>>> - also important to notice that we will have both an Hadoop and an Infinispan cluster running in parallel: the user will interact with the former in order to run M/R tasks. Hadoop will use Infinispan (integration achieved through InputFormat and OutputFormat ) in order to get the data to be processed.
>>>
>>> Would this be 2 JVMs, or you can trick Hadoop to start Infinispan as well -- hence 1JVM?
>>
>> good point, ideally it should be a single VM: reduced serialization cost (in vm access) and simpler architecture. That's if you're not using C/S mode, of course.
>
> ?
> Don't try confusing us again on that :-)
> I think we agreed that the job would *always* run in strict locality
> with the datacontainer (i.e. in the same JVM). Sure, an Hadoop client
> would be connecting from somewhere else but that's unrelated.
we did discuss the possibility of running it over hotrod though, do you see a problem with that?
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
More information about the infinispan-dev
mailing list