5:11 p.m.
Hi,
I've started (again) working on ISPN-5021 [1], and I'd like to get some
common agreement on few terms. So below I summarize my understanding (or
misunderstanding) of these, please state you opinion, thinking a bit
more generally.
State transfer: whole process beginning with some ST-related event =
node being detected to crash/sending join or leave request, and ending
when this event is processed. When another event happens, the current ST
can either be finished or canceled and then *another* ST can begin.
State transfer is a cluster-wide process, though it cannot be started
and ended absolutely simultaneously.
Rebalance: one phase of ST, when the data transfer occurs.
Data rehash: this is a bit painful point: we have DataRehashEvent where
the name suggest that it is related rather to rebalance, but currently
it fires when CacheTopology.getPendingCH() == null (that means when ST
is complete), and the event itself also looks more like it should be
fired at the end of state transfer. If we have something more to do
after the rebalance, I am not sure how useful is firing that just
because all data has been transferred (but for example before old data
has been wiped out). Should I add another StateTransferEvent event (and
appropriate listeners)? That would break the compatibility with tightly
related implementations...
WDYT?
Radim
[1] https://issues.jboss.org/browse/ISPN-5021
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
11:14 p.m.
On Thu, Jan 19, 2017 at 11:13 AM Radim Vansa <rvansa(a)redhat.com> wrote:
Hi,
I've started (again) working on ISPN-5021 [1], and I'd like to get some
common agreement on few terms. So below I summarize my understanding (or
misunderstanding) of these, please state you opinion, thinking a bit
more generally.
State transfer: whole process beginning with some ST-related event =
node being detected to crash/sending join or leave request, and ending
when this event is processed. When another event happens, the current ST
can either be finished or canceled and then *another* ST can begin.
State transfer is a cluster-wide process, though it cannot be started
and ended absolutely simultaneously.
Rebalance: one phase of ST, when the data transfer occurs.
Data rehash: this is a bit painful point: we have DataRehashEvent where
the name suggest that it is related rather to rebalance, but currently
it fires when CacheTopology.getPendingCH() == null (that means when ST
is complete), and the event itself also looks more like it should be
fired at the end of state transfer. If we have something more to do
after the rebalance, I am not sure how useful is firing that just
because all data has been transferred (but for example before old data
has been wiped out). Should I add another StateTransferEvent event (and
appropriate listeners)? That would break the compatibility with tightly
related implementations...
The stream retry mechanism currently uses DataRehashedEvent. It is
beneficial but not required for it to be notified immediately after all
entries have been transferred but before any have been removed. This
shortens the window for when a stream operation is retried since it has to
be sure that all entries for a given segment are present, but we don't care
if entries for a non owned segment are present. But the good thing is that
the code shouldn't require much work at all to be changed if we needed to.
I am personally not keen on having another event that is like
DataRehashedEvent, but only signifies entries were removed. It seems a bit
confusing. Is there any usage you were thinking that required the old
entries to be removed that could benefit from the new event/listeners?
WDYT?
Radim
[1] https://issues.jboss.org/browse/ISPN-5021
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
10:13 a.m.
On 01/19/2017 11:14 PM, William Burns wrote:
On Thu, Jan 19, 2017 at 11:13 AM Radim Vansa <rvansa(a)redhat.com
<mailto:rvansa@redhat.com>> wrote:
Hi,
I've started (again) working on ISPN-5021 [1], and I'd like to get
some
common agreement on few terms. So below I summarize my
understanding (or
misunderstanding) of these, please state you opinion, thinking a bit
more generally.
State transfer: whole process beginning with some ST-related event =
node being detected to crash/sending join or leave request, and ending
when this event is processed. When another event happens, the
current ST
can either be finished or canceled and then *another* ST can begin.
State transfer is a cluster-wide process, though it cannot be started
and ended absolutely simultaneously.
Rebalance: one phase of ST, when the data transfer occurs.
Data rehash: this is a bit painful point: we have DataRehashEvent
where
the name suggest that it is related rather to rebalance, but currently
it fires when CacheTopology.getPendingCH() == null (that means when ST
is complete), and the event itself also looks more like it should be
fired at the end of state transfer. If we have something more to do
after the rebalance, I am not sure how useful is firing that just
because all data has been transferred (but for example before old data
has been wiped out). Should I add another StateTransferEvent event
(and
appropriate listeners)? That would break the compatibility with
tightly
related implementations...
The stream retry mechanism currently uses DataRehashedEvent. It is
beneficial but not required for it to be notified immediately after
all entries have been transferred but before any have been removed.
This shortens the window for when a stream operation is retried since
it has to be sure that all entries for a given segment are present,
but we don't care if entries for a non owned segment are present. But
the good thing is that the code shouldn't require much work at all to
be changed if we needed to.
I am personally not keen on having another event that is like
DataRehashedEvent, but only signifies entries were removed. It seems a
bit confusing. Is there any usage you were thinking that required the
old entries to be removed that could benefit from the new event/listeners?
Okay, so from streams POV, DataRehashedEvent should mark rebalance, and
be fired once all incoming segments arrive. The implementation does not
rely on pendingCH being null when the event is fired, right? I like such
definition as "data rehash" sounds like moving data around.
I can see StateTransferEvent being useful when downscaling the cluster,
and you want to remove one node at a time (as the current leave is not
too graceful and calling cacheManager.stop() does not wait until ST
completes).
R.
WDYT?
Radim
[1] https://issues.jboss.org/browse/ISPN-5021
--
Radim Vansa <rvansa(a)redhat.com <mailto:rvansa@redhat.com>>
JBoss Performance Team
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
11:57 a.m.
Oops, I thought I had replied to this...
On Thu, Jan 19, 2017 at 6:11 PM, Radim Vansa <rvansa(a)redhat.com> wrote:
Hi,
I've started (again) working on ISPN-5021 [1], and I'd like to get some
common agreement on few terms. So below I summarize my understanding (or
misunderstanding) of these, please state you opinion, thinking a bit
more generally.
State transfer: whole process beginning with some ST-related event =
node being detected to crash/sending join or leave request, and ending
when this event is processed. When another event happens, the current ST
can either be finished or canceled and then *another* ST can begin.
State transfer is a cluster-wide process, though it cannot be started
and ended absolutely simultaneously.
Rebalance: one phase of ST, when the data transfer occurs.
I see this exactly the opposite way: state transfer is the actual
transfer of the data, and the rebalance is the whole process of adding
a "pending" consistent hash, starting the state transfer, confirming
the transfer of state, and finally replacing the "current" consistent
hash with the pending one.
Data rehash: this is a bit painful point: we have DataRehashEvent
where
the name suggest that it is related rather to rebalance, but currently
it fires when CacheTopology.getPendingCH() == null (that means when ST
is complete), and the event itself also looks more like it should be
fired at the end of state transfer. If we have something more to do
after the rebalance, I am not sure how useful is firing that just
because all data has been transferred (but for example before old data
has been wiped out). Should I add another StateTransferEvent event (and
appropriate listeners)? That would break the compatibility with tightly
related implementations...
The problem I was trying to solve with the "rebalance" name in 5.2 is
that before we had two different concepts:
1. State transfer applied only to replicated caches, and it meant
transferring state from the coordinator to a joiner.
2. Rehashing meant computing a new consistent hash based on the
JGroups view, receiving state from old owners based on the consistent
hash of the old JGroups view, and removing state no longer owned in
the new consistent hash. The coordinator didn't have a special role.
Rebalancing was meant to be an extension of rehashing to cover both
distributed and replicated caches (now with replicated caches also
being able to receive state from multiple members in parallel). Some
of the components (StateTransferManager) and events (DataRehashEvent)
kept their names, however.
WDYT?
Radim
[1] https://issues.jboss.org/browse/ISPN-5021
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
2894
days inactive
2920
days old
infinispan-dev@lists.jboss.org
3 comments
3 participants
participants (3)
-
Dan Berindei -
Radim Vansa -
William Burns