Re: [infinispan-dev] Partial state transfer in Infinispan

On 6/1/11 4:21 PM, Sanne Grinovero wrote: > Hi Bela, > > 2011/6/1 Bela Ban<bban(a)redhat.com>: >> We currently use JGroups' partial state transfer to transfer individual >> caches from one Infinispan instance to another. >> >> Since I got rid of partial state transfer in JGroups 3.0, and don't like >> to add it back, I'd like to know whether this is still needed. >> >> I thought that we currently require the same set of caches to be >> available in all Infinispan instances, and the reason (IIRC) was that >> distribution wouldn't work if we have caches 1 and 2 available on >> instances A and B, but not on C, because consistent hashing distributes >> the data based on views, and we didn't want to have to keep track of >> individual caches... > > Well I really don't like this limitation in Infinispan and was > actually hoping that we could remove it at some point. > Imagine the scenario in which you have a running cluster, and at some > point the new release of your application needs an additional cache: > there's no way to start a new node having this new cache. Yes, I fully agree. Another example is HTTP web sessions: currently 1 webapp == 1 cache, so we currently require the same webapps to be deployed in all JBoss instances *if* we use replication (distribution is different) !

> Also right now when an application starts, it's possible with the > proper timing that it joins the cluster before having defined and > started all caches (starting the cachemanager and the caches is not an > atomic operation), basically failing to start because of this > limitation. Yep > Maybe it's still possible to build such a thing on top of non-partial > state transfer? As it doesn't exist, we didn't design it. Yes. Well, Infinispan already uses its own state transfer for distribution, I wonder why this isn't the case for replication. >> Why are we actually using JGroups' state transfer with replication, but >> use our own state transfer with distribution ? > > I don't know, but guess it's because each node has a different set of > keys so no node has the same state as another ? You could still use JGroups state transfer; getState() would list the state provider as target node. In general, partial state transfer involves transferring (1) the partial state and (2) the digest, which is a vector of highest seqnos seen for every member. When we get a partial state, we always overwrite our own digest with the one received, and update our state accordingly. However, when this is done a couple of times, for different partial states, I'm not sure that we won't receive a few messages multiple times, due to the digest overwriting... I think the cleanest solution would be for you guys to reuse the state transfer you already use for state transfer in distribution mode.