[infinispan-dev] DIST and rehashing - again

[Galder Zamarreno]

Manik Surtani wrote:
> Ok, these rehash designs are flawed.  While they work in concept, they 
> could lock up a cluster for periods of time while a rehash is in 
> progress, since every node may provide some state to every other node, 
> and while state is being streamed, a recipient would need to queue 
> incoming transactions to be applied on top of any state applied.  While 
> this will work very well if just one node receives state at at time, it 
> won't work if (theoretically) every node receives some state (for 
> example if a node leaves).

I don't know the DIST code very well yet but here's an idea:

Maybe instead of doing the state transfer for the key, rehash() could 
queue these state transfer requests and do the following: The next time 
a put/get for a key that needs to be rehashed comes along, do the 
reshahing then.

Obviously, this has issues for gets arriving in nodes not containing the 
key looked after. In theory, you'd need to use the hash that was 
originally used to store that key to be able to find it but not 
necessarily. You could try with the current hash result and if that 
doesn't return anything, do a clustered get on all nodes in cluster. As 
a response to this clustered get, the node could initiate an state 
transfer of this key to the new host node if still necessary. It could 
well happen that after N views, the owners of the keys should still be 
the same as it was N views ago.

If a get arrives on a node A already containing the key but the key is 
also present in node B that needs to pass the key node C, not much we 
can do until a request comes in to node C or a node not containing that key.

For puts, updates on nodes that should transfer the key would trigger 
the transfering together with the update.

The major downside I see to this is from a user perspective. The user 
does not have an up to date view of where the keys should reside until a 
put/get is done in all keys that should have been rehashed.

> 
> I have an alternate approach (sketched out in a notebook here) which 
> will mean some redesign to the core dist code, I will publish these on 
> the wiki when I get back from vacation.
> 
> In the meanwhile though, it means no Beta1 today (boo!) but I will push 
> out an Alpha5 (yaayy!) since there are a lot of other changes and 
> improvements which I want to make public.
> 
> Watch this space.
> 
> Cheers
> Manik
> 
> 
> On 8 Jun 2009, at 15:34, Manik Surtani wrote:
> 
>> Ok, so here is where I have jotted down designs for rehashing in DIST.
>>
>>     http://www.jboss.org/community/wiki/DIST-distributedcachemode#Rehashing 
>>
>>
>> Please have a look and comment accordingly.  Some points:
>>
>> 1.  Heterogenous nodes are not supported at this time.  I expect to 
>> add support for this later.
>> 2.  Actual state is transferred using RPC calls.  A more efficient 
>> streaming mechanism will come in later.
>> 3.  Each view change will result in a rehash event.  Combining view 
>> changes to a single rehash is something we would consider for later.
>>
>> Cheers
>> -- 
>> Manik Surtani
>> manik at jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> -- 
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat