Re: [infinispan-dev] State transfer improvements for 5.2?

On Thu, Jan 26, 2012 at 7:17 PM, Manik Surtani<manik(a)jboss.org> wrote: > On 26 Jan 2012, at 16:05, Sanne Grinovero wrote: > >> On 26 January 2012 10:36, Manik Surtani<manik(a)jboss.org> wrote: >>> Dan, >>> >>> Looking through what we currently have, given the blocking nature of state transfer, I wonder if we can improve on it by pausing the state transfer from time to time to flush waiting transactions? I.e., if state transfer is in progress and a write command or transaction is waiting for it to finish, and since state transfer is chunked anyway, perhaps to let a few transactions through in between transferring chunks? >> Wondering about my proposal about transfering values separately from >> locks not being easier to implement? Or likely same effort, but >> better.. > Well, thats what I want to try and understand. If periodic flushing is easier/quicker/less risky to implement, I think we could do it for 5.2.0. :) > I don't think it would be any easier/less risky - the moment we allow transactions to go through our data container iterator is no longer valid and we could e.g. push data that's been already deleted on the old owner. The only improvement to the current blocking state transfer that I think would be easy enough is https://issues.jboss.org/browse/ISPN-1731 - when we acquire a key lock we'd have to try first with 0 timeout and then when we wait with a non-zero timeout we'd have to register our thread so that state transfer can interrupt it. In fact, since non-blocking state transfer also requires threads to line up at a barrier at some point, this should help non-blocking state transfer as well. Actually there is something else, but I'd consider it more of a bug. With optimistic transactions we retry the prepare after it failed to acquire the state transfer lock, but with pessimistic transactions I'm not sure the retry works properly. > Sanne, have you documented your design on separate transfer of state vs locks on the wiki somewhere? I'd be curious to see what Paolo and the others at INESC and CINI think about it as well. :) > I wrote a paragraph about it here: https://community.jboss.org/wiki/AsymmetricCachesAndManualRehashingDesign I'm not sure if Sanne validated it... Cheers Dan