[infinispan-dev] Distributed Counter Discussion

[Sanne Grinovero]

Great starting point!
Some comments inline:

On 14 March 2016 at 19:14, Pedro Ruivo <pedro at infinispan.org> wrote:
> Hi everybody,
>
> Discussion about distributed counters.
>
> == Public API ==
>
> interface Counter

As a user, how do I get a Counter instance? From a the CacheContainer interface?

Will they have their own configuration section in the configuration file?

>
> String getName() //counter name
> long get()       //current value. may return stale value due to concurrent
> operations to other nodes.

This is what puzzles me the most. I'm not sure if the feature is
actually useful, unless we can clearly state how far outdated the
value could be.

I think a slightly more formal definition would be in order. For
example I think it would be acceptable to say that this will return a
value from the range of values the primary owner of this counter was
holding in the timeframe between the method is being invoked and the
time the value is returned.

Could it optionally be integrated with Total Order ? Transactions?

> void increment() //async or sync increment. default add(1)
> void decrement() //async or sync decrement. default add(-1)
> void add(long)   //async or sync add.
> void reset()     //resets to initial value
>
> Note: Tried to make the interface as simple as possible with support for
> sync and async operations. To avoid any confusion, I consider an async
> operation as happening somewhat in the future, i.e. eventually
> increments/decrements.
> The sync operation happens somewhat during the method execution.
>
> interface AtomiCounter extends Counter
>
> long addAndGet()       //adds a returns the new value. sync operation
> long incrementAndGet() //increments and returns the new value. sync
> operation. default addAndGet(1)
> long decrementAndGet() //decrements and returns the new value. sync
> operation. default addAndGet(-1)
>
> interface AdvancedCounter extends Counter
>
> long getMin/MaxThreshold() //returns the min and max threshold value

"threshold" ??

> void add/removeListener()  //adds a listener that is invoked when the
> value change. Can be extended to notify when it is "reseted" and when
> the threshold is reached.
>
> Note: should this interface be splitted?

I'd prefer a single interface, with reduced redundancy.
For example, is there really a benefit in having a "void increment()"
and also a "long addAndGet()" ? [Besides The fact that only the first
one can benefit of an async option]

Besides, I am no longer sure that it's good thing that methods in
Infinispan can be async vs sync depending on configuration switches;
I'd rather make it explicit in the signature and simplify the
configuration by removing such a flag.

Making the methods which are async-capable to look "explicitly async"
should also allow us to add completeable futures & similar.

>
> == Details ==
>
> This is what I have in mind. Two counter managers: one based on JGroups
> counter and another one based on Infinispan cache.
> The first one creates AtomicCounters and it first perfectly. All
> counters are created with an initial value (zero by default)
> The second generates counters with all the options available. It can mix
> sync/async operation and all counters will be in the same cache. The
> cache will be configure by us and it would be an internal cache. This
> will use all the features available in the cache.

Rather than a split organized on the backing implementation details,
I'd prefer to see a split based on purpose of the counter.

For example JGroups has different styles of counters: one might need
one to do an atomic increment which is monotonically increasing across
the whole cluster - from the point of view from an omniscent observer
- but in many cases we just need the generation of a guarantee of a
monotonic unique number: that allows for example to have each node
pre-allocate a range of values and pre-fetch a new block while its
running out of values.

For some systems this is not acceptable because a failing server might
result in some of its allocated numbers to not ever be used, so
creating gaps; for others it's not acceptable that the figures
returned by the "monotonic counter" are not monotonic across each
other when confronting results from different cluster nodes, but for
many use cases that's acceptable. My point being that we need to be
able to choose between these possible semantics.

As an Infinispan consumer being able to express such details is
essential, while having to choose between "backed by JGroups counters"
or not is irrelevant and actually makes it harder.

In terms of Infinispan integration, the most important point I'd like
to see is persistence: i.e. make sure you can store them in a
CacheStore.
How we do that efficiently will also affect design; for example I was
expecting to see a Counter as something which is stored in a Cache, so
inheriting details such as configured CacheStore(s) and number of
owners.

Thanks for starting this!

Sanne

>
> Configuration-wise, I'm thinking about 2 parameters: number of backups
> and timeout (for sync operations).
>
> So, comment bellow and let me know alternatives, improvement or if I
> missed something.
>
> ps. I also consider implement a counter based on JGroups-raft but I
> believe it is an overkill.
> ps2. sorry for the long email :( I tried to be shorter as possible.
>
> Cheers,
> Pedro
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev