[infinispan-dev] Partial state transfer in Infinispan

[Manik Surtani]

We use partial state transfer not to generate partial state per cache, but the entire state per cache, but since we have > 1 cache sharing a given JGroups channel, as far as JGroups in concerned this *is* partial state of a node.  I.e., the state of just 1 cache on a channel, not all the caches.

So we actually use cacheName as the state identifier (in JGroups' ExtendedMessageListener).

But yes, there is no reason why we can't replace this with RPC as per Distribution, however I think we do need a streaming solution - not just for replication but distribution as well.  As such I'd only want to re-implement this bit once, rather than a temp RPC based solution first.  So we need a mechanism to either:

(1) open a separate TCP socket for the sake of streaming state, or
(2) reuse the sockets JGroups opens.

They both have their pros and cons.

(1) is more configuration, firewall setup, and a spiderweb of connections in a large grid
(2) would mean multiplexing with JGroups' use of the socket.

Any thoughts and suggestions?

Cheers
Manik

On 1 Jun 2011, at 15:14, Bela Ban wrote:

> We currently use JGroups' partial state transfer to transfer individual 
> caches from one Infinispan instance to another.
> 
> Since I got rid of partial state transfer in JGroups 3.0, and don't like 
> to add it back, I'd like to know whether this is still needed.
> 
> I thought that we currently require the same set of caches to be 
> available in all Infinispan instances, and the reason (IIRC) was that 
> distribution wouldn't work if we have caches 1 and 2 available on 
> instances A and B, but not on C, because consistent hashing distributes 
> the data based on views, and we didn't want to have to keep track of 
> individual caches...
> 
> Why are we actually using JGroups' state transfer with replication, but 
> use our own state transfer with distribution ?
> 
> Opinions ?
> 
> -- 
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org