[infinispan-dev] Streamed rehashing

[Manik Surtani]

Hey dude - sorry for the late response.  I'm cc'ing infinispan-dev BTW.  Gotta keep the community in the loop!  :)

On 15 Apr 2010, at 16:48, Vladimir Blagojevic wrote:

> 
> On 2010-04-13, at 11:09 AM, Manik Surtani wrote:
> 
>>> 
>>> I would register RehashedSTM (an impl of STM) into container under cacheName.append("_REHASH") or whatever other token. RehashedSTM will capture the logic of serializing/deserializing rehashing state.
>> 
>> Yeah that would work... 
>> 
>>> We can have maybe DistManagerImpl implement STM.  Not sure though if DistManagerImpl can be registered under multiple names in container....
>> 
>> Hmm, that may prove tricky... any particular reason?  DMI could just have a reference to a RehashingSTM...
>> 
> 
> Good progress so far.
> 
> 
> 
> JOIN question:
> 
> During join, as is, joiner simply gets synchronous response with a state by sending RehashControlCommand to state producer. Since we are essentially doing a simple channel#getState now we can no longer rely on RehashControlCommand to carry information needed to do a join transfer - namely newCH and joiner. We have to pick up these values from DistributionManagerImpl. 
> 
> We already have access to both old and new ch on DistributionImpl since we broadcast JOIN_REHASH_START to all nodes. Therefore, we have access to both old and new CH through DistributionImpl.getConsistentHash() which at this moment is UnionConsistentHash. All good! 

Cool.

> However, RehashingSTM does not have access to joiner Address since neither DistributionManagerImpl not DistributionManager expose it. And we need joiner to decide if certain cache entry should be included into a state as producer produces state for joiner. Can we expose joiner in DistributionManager? Or is there another clever way to obtain it?

Can this not be sent as a part of the getState request?  Or as a part of the streaming protocol?  E.g., 

1.  getState opens streams between a joiner and various state producers.
2.  before state producers start streaming state, they need to wait for the joiner to publish it's address on the stream

> LEAVE question:
> 
> I am now looking at push state. I don't think there is another way to accomplish leave but to invert state transfer. Instead of pushing state to receivers, we need to inform all state receiver nodes to initiate state transfer to a certain node. How do you feel about this? A bit of a kludge, no?

Yes, but this is still a logical PUSH since it is initiated by the state sender.  :)

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org