[infinispan-dev] Partial state transfer in Infinispan

[Paul Ferraro]

On Wed, 2011-06-01 at 16:46 +0200, Bela Ban wrote:
> 
> On 6/1/11 4:21 PM, Sanne Grinovero wrote:
> > Hi Bela,
> >
> > 2011/6/1 Bela Ban<bban at redhat.com>:
> >> We currently use JGroups' partial state transfer to transfer individual
> >> caches from one Infinispan instance to another.
> >>
> >> Since I got rid of partial state transfer in JGroups 3.0, and don't like
> >> to add it back, I'd like to know whether this is still needed.
> >>
> >> I thought that we currently require the same set of caches to be
> >> available in all Infinispan instances, and the reason (IIRC) was that
> >> distribution wouldn't work if we have caches 1 and 2 available on
> >> instances A and B, but not on C, because consistent hashing distributes
> >> the data based on views, and we didn't want to have to keep track of
> >> individual caches...
> >
> > Well I really don't like this limitation in Infinispan and was
> > actually hoping that we could remove it at some point.
> > Imagine the scenario in which you have a running cluster, and at some
> > point the new release of your application needs an additional cache:
> > there's no way to start a new node having this new cache.
> 
> 
> Yes, I fully agree. Another example is HTTP web sessions: currently 1 
> webapp == 1 cache, so we currently require the same webapps to be 
> deployed in all JBoss instances *if* we use replication (distribution is 
> different) !

As of AS7, web sessions for all webapps are stored in a single cache.
https://issues.jboss.org/browse/JBCLUSTER-293

> > Also right now when an application starts, it's possible with the
> > proper timing that it joins the cluster before having defined and
> > started all caches (starting the cachemanager and the caches is not an
> > atomic operation), basically failing to start because of this
> > limitation.
> 
> 
> Yep
> 
> 
> > Maybe it's still possible to build such a thing on top of non-partial
> > state transfer? As it doesn't exist, we didn't design it.
> 
> 
> Yes. Well, Infinispan already uses its own state transfer for 
> distribution, I wonder why this isn't the case for replication.
> 
> 
> >> Why are we actually using JGroups' state transfer with replication, but
> >> use our own state transfer with distribution ?
> >
> > I don't know, but guess it's because each node has a different set of
> > keys so no node has the same state as another ?
> 
> You could still use JGroups state transfer; getState() would list the 
> state provider as target node.
> 
> In general, partial state transfer involves transferring (1) the partial 
> state and (2) the digest, which is a vector of highest seqnos seen for 
> every member. When we get a partial state, we always overwrite our own 
> digest with the one received, and update our state accordingly. However, 
> when this is done a couple of times, for different partial states, I'm 
> not sure that we won't receive a few messages multiple times, due to the 
> digest overwriting...
> 
> I think the cleanest solution would be for you guys to reuse the state 
> transfer you already use for state transfer in distribution mode.
>