[infinispan-dev] Distributed Counter Discussion

[Bela Ban]

On 14/03/16 23:17, Randall Hauch wrote:
> What are the requirements? What are the distributed counters for? Is the
> counter to be monotonically increasing? Can there be any missed values?
> Does the counter need to increment and decrement? What is the *smallest*
> API you need initially?
>
> There are two choices when implementing a distributed counter: use
> central coordination (like JGroups counters), or use independent
> counters on separate machines that will eventually converge to the
> correct value (CRDTs). Coordinated counters are expensive and therefore
> slow, and can suffer from problems during network or cluster problems.

The question is what do you get for this? If your app can't afford 
duplicate counter values during a network partition, then - yes - there 
is some overhead. CRDTs won't be able to guarantee this property. OTOH 
CRDTs are fast when you only care about some sort of eventual 
consistency, and don't need 'hard' consistency.


> For example, what happens during a split brain? OTOH, CRDTs are
> decentralized so therefore are very fast, easily merged, and fault
> tolerant;

Yes, CRDTS are AP whereas jgroups-raft counters are CP. JGroups 
counters, otoh, are CRAP (consistent, reliable, available and 
partition-aware.

Take the last sentence with a grain of salt :-)

> they’re excellent when counting things that are occurring
> independently and therefore may be more suited for
> monitoring/metrics/accumulators/etc.  Both have very different behaviors
> under ideal and failure scenarios, have different performance and
> consistency guarantees, and are useful in different scenarios. Make sure
> you choose accordingly.
>
> For information about CRDTs, make sure you’ve read the CRDT paper by
> Shapiro: http://hal.upmc.fr/inria-00555588/document
>
> Randall
>
>> On Mar 14, 2016, at 2:14 PM, Pedro Ruivo <pedro at infinispan.org
>> <mailto:pedro at infinispan.org>> wrote:
>>
>> Hi everybody,
>>
>> Discussion about distributed counters.
>>
>> == Public API ==
>>
>> interface Counter
>>
>> String getName() //counter name
>> long get()//current value. may return stale value due to concurrent
>> operations to other nodes.
>> void increment() //async or sync increment. default add(1)
>> void decrement() //async or sync decrement. default add(-1)
>> void add(long)   //async or sync add.
>> void reset()     //resets to initial value
>>
>> Note: Tried to make the interface as simple as possible with support for
>> sync and async operations. To avoid any confusion, I consider an async
>> operation as happening somewhat in the future, i.e. eventually
>> increments/decrements.
>> The sync operation happens somewhat during the method execution.
>>
>> interface AtomiCounter extends Counter
>>
>> long addAndGet()       //adds a returns the new value. sync operation
>> long incrementAndGet() //increments and returns the new value. sync
>> operation. default addAndGet(1)
>> long decrementAndGet() //decrements and returns the new value. sync
>> operation. default addAndGet(-1)
>>
>> interface AdvancedCounter extends Counter
>>
>> long getMin/MaxThreshold() //returns the min and max threshold value
>> void add/removeListener()  //adds a listener that is invoked when the
>> value change. Can be extended to notify when it is "reseted" and when
>> the threshold is reached.
>>
>> Note: should this interface be splitted?
>>
>> == Details ==
>>
>> This is what I have in mind. Two counter managers: one based on JGroups
>> counter and another one based on Infinispan cache.
>> The first one creates AtomicCounters and it first perfectly. All
>> counters are created with an initial value (zero by default)
>> The second generates counters with all the options available. It can mix
>> sync/async operation and all counters will be in the same cache. The
>> cache will be configure by us and it would be an internal cache. This
>> will use all the features available in the cache.
>>
>> Configuration-wise, I'm thinking about 2 parameters: number of backups
>> and timeout (for sync operations).
>>
>> So, comment bellow and let me know alternatives, improvement or if I
>> missed something.
>>
>> ps. I also consider implement a counter based on JGroups-raft but I
>> believe it is an overkill.
>> ps2. sorry for the long email :( I tried to be shorter as possible.
>>
>> Cheers,
>> Pedro
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)