[infinispan-dev] DeltaAware: different local/remote behaviour

[Erik Salter]

Hi Radim,

We may be doing something similar.  I was implementing something along the
lines of a queue of operations that resolve into a single value. This
implementation uses Total Order and CRDTs.  I also want a changelog to
send to the backups.

I already use DeltaAware quite liberally in my production environment.
I've always looked at it as an implementation detail if the originator ==
primary owner.  While this does make for some inefficiencies, like
increased memory utilization (I have a lot of keys for very large
objects), it's worth it to me from a simplicity standpoint.

I always use DeltaAware with SKIP_REMOTE_LOOKUP.

The real fun with DeltaAware are the cases where a backup receives a
DeltaAware instance and the key isn't in its data container.  It will
issue a remote get to pull the complete context before applying the delta.
 During state transfer, this will lead to increased thread utilization on
the joining nodes.  I have a use case where I must restart half my cluster
while there's 100K DeltaAware keys being written at a high data rate.
With numOwners == 2, there are 3 nodes in the union CH.  A new backup will
issue 2 remote GetKeyValueCommands.  I have a hack to stagger the gets to
reduce bandwidth, but if we're rethinking the implementation this should
be an additional consideration.

Regards,

Erik
  

On 12/2/14, 9:33 AM, "Radim Vansa" <rvansa at redhat.com> wrote:

>Hi,
>
>I was trying to implement an effective atomic counters [1] in Infinispan
>using the DeltaAware interface, but trying to use DeltaAware I've
>spotted an unexpected behaviour; I wanted to have a Delta for
>getAndIncrement() method, that would simply increment the value without
>knowing the previous value ahead, and return this previous value.
>Therefore, I was inserting a fake DeltaAware object into the cache that
>generates this relative Delta.
>
>This works as long as the originator != primary owner, as the delta is
>generated during marshalling. However, if I store that object locally,
>the fake object is not used to generate the delta and reapply it on
>current instance in data container, but it is stored directly.
>
>Is such difference in local/remote behaviour bug or feature? (this is
>the main question in this mail)
>
>It seems to me that there are two reasons to use deltas: reducing size
>of RPCs and reduce their total number. So the design should optimize both.
>
>I have another doubts about DeltaAware interface usefulness, tracked in
>ISPN-5035 [2] - while it reduces bandwith from originator to primary
>owner, the response from primary owner to originator carries the full
>value. I also find quite inconvenient that only PutKeyValueCommand
>somehow works with deltas, but ReplaceCommand does not.
>
>I've also noticed that the backup carries the full value [3], not quite
>a good idea when we're trying to reduce bandwith.
>
>Generally, I think that EntryProcessor-like interface would be more
>useful than DeltaAware.
>
>Radim
>
>[1] https://github.com/rvansa/infinispan/tree/t_objects
>[2] https://issues.jboss.org/browse/ISPN-5035
>[3] https://issues.jboss.org/browse/ISPN-5037
>
>-- 
>Radim Vansa <rvansa at redhat.com>
>JBoss DataGrid QA
>
>_______________________________________________
>infinispan-dev mailing list
>infinispan-dev at lists.jboss.org
>https://lists.jboss.org/mailman/listinfo/infinispan-dev