[infinispan-dev] DeltaAware: different local/remote behaviour

[Radim Vansa]

Hi Erik,

it's great to get community (users') feedback on API :)

Comments inline

On 12/02/2014 04:04 PM, Erik Salter wrote:
> Hi Radim,
>
> We may be doing something similar.  I was implementing something along the
> lines of a queue of operations that resolve into a single value. This
> implementation uses Total Order and CRDTs.  I also want a changelog to
> send to the backups.
>
> I already use DeltaAware quite liberally in my production environment.
> I've always looked at it as an implementation detail if the originator ==
> primary owner.  While this does make for some inefficiencies, like
> increased memory utilization (I have a lot of keys for very large
> objects), it's worth it to me from a simplicity standpoint.

Yes, and that's what I'd like to do :) When designing the object, I was 
expecting that all updates will be in the delta-way, and therefore I 
report that it's not this way and I have to adapt the code in a hackish way.

>
> I always use DeltaAware with SKIP_REMOTE_LOOKUP.

I see. But in some use cases you'd want some condensed report what was 
the result of applying delta.

>
> The real fun with DeltaAware are the cases where a backup receives a
> DeltaAware instance and the key isn't in its data container.  It will
> issue a remote get to pull the complete context before applying the delta.
>   During state transfer, this will lead to increased thread utilization on
> the joining nodes.  I have a use case where I must restart half my cluster
> while there's 100K DeltaAware keys being written at a high data rate.
> With numOwners == 2, there are 3 nodes in the union CH.  A new backup will
> issue 2 remote GetKeyValueCommands.

Hmm, does not sound really convenient but I don't see what other could 
be done when the delta-updated entry is not in place yet.

> I have a hack to stagger the gets to
> reduce bandwidth, but if we're rethinking the implementation this should
> be an additional consideration.

Nobody said we're rethinking this - I was just providing the feedback 
from my POV after first starting to play with DeltaAware.

Radim

>
> Regards,
>
> Erik
>    
>
> On 12/2/14, 9:33 AM, "Radim Vansa" <rvansa at redhat.com> wrote:
>
>> Hi,
>>
>> I was trying to implement an effective atomic counters [1] in Infinispan
>> using the DeltaAware interface, but trying to use DeltaAware I've
>> spotted an unexpected behaviour; I wanted to have a Delta for
>> getAndIncrement() method, that would simply increment the value without
>> knowing the previous value ahead, and return this previous value.
>> Therefore, I was inserting a fake DeltaAware object into the cache that
>> generates this relative Delta.
>>
>> This works as long as the originator != primary owner, as the delta is
>> generated during marshalling. However, if I store that object locally,
>> the fake object is not used to generate the delta and reapply it on
>> current instance in data container, but it is stored directly.
>>
>> Is such difference in local/remote behaviour bug or feature? (this is
>> the main question in this mail)
>>
>> It seems to me that there are two reasons to use deltas: reducing size
>> of RPCs and reduce their total number. So the design should optimize both.
>>
>> I have another doubts about DeltaAware interface usefulness, tracked in
>> ISPN-5035 [2] - while it reduces bandwith from originator to primary
>> owner, the response from primary owner to originator carries the full
>> value. I also find quite inconvenient that only PutKeyValueCommand
>> somehow works with deltas, but ReplaceCommand does not.
>>
>> I've also noticed that the backup carries the full value [3], not quite
>> a good idea when we're trying to reduce bandwith.
>>
>> Generally, I think that EntryProcessor-like interface would be more
>> useful than DeltaAware.
>>
>> Radim
>>
>> [1] https://github.com/rvansa/infinispan/tree/t_objects
>> [2] https://issues.jboss.org/browse/ISPN-5035
>> [3] https://issues.jboss.org/browse/ISPN-5037
>>
>> -- 
>> Radim Vansa <rvansa at redhat.com>
>> JBoss DataGrid QA
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss DataGrid QA