8:14 p.m.
Hi everybody,
Discussion about distributed counters.
== Public API ==
interface Counter
String getName() //counter name
long get() //current value. may return stale value due to concurrent
operations to other nodes.
void increment() //async or sync increment. default add(1)
void decrement() //async or sync decrement. default add(-1)
void add(long) //async or sync add.
void reset() //resets to initial value
Note: Tried to make the interface as simple as possible with support for
sync and async operations. To avoid any confusion, I consider an async
operation as happening somewhat in the future, i.e. eventually
increments/decrements.
The sync operation happens somewhat during the method execution.
interface AtomiCounter extends Counter
long addAndGet() //adds a returns the new value. sync operation
long incrementAndGet() //increments and returns the new value. sync
operation. default addAndGet(1)
long decrementAndGet() //decrements and returns the new value. sync
operation. default addAndGet(-1)
interface AdvancedCounter extends Counter
long getMin/MaxThreshold() //returns the min and max threshold value
void add/removeListener() //adds a listener that is invoked when the
value change. Can be extended to notify when it is "reseted" and when
the threshold is reached.
Note: should this interface be splitted?
== Details ==
This is what I have in mind. Two counter managers: one based on JGroups
counter and another one based on Infinispan cache.
The first one creates AtomicCounters and it first perfectly. All
counters are created with an initial value (zero by default)
The second generates counters with all the options available. It can mix
sync/async operation and all counters will be in the same cache. The
cache will be configure by us and it would be an internal cache. This
will use all the features available in the cache.
Configuration-wise, I'm thinking about 2 parameters: number of backups
and timeout (for sync operations).
So, comment bellow and let me know alternatives, improvement or if I
missed something.
ps. I also consider implement a counter based on JGroups-raft but I
believe it is an overkill.
ps2. sorry for the long email :( I tried to be shorter as possible.
Cheers,
Pedro
10:27 p.m.
Great starting point!
Some comments inline:
On 14 March 2016 at 19:14, Pedro Ruivo <pedro(a)infinispan.org> wrote:
Hi everybody,
Discussion about distributed counters.
== Public API ==
interface Counter
As a user, how do I get a Counter instance? From a the CacheContainer interface?
Will they have their own configuration section in the configuration file?
String getName() //counter name
long get() //current value. may return stale value due to concurrent
operations to other nodes.
This is what puzzles me the most. I'm not sure if the feature is
actually useful, unless we can clearly state how far outdated the
value could be.
I think a slightly more formal definition would be in order. For
example I think it would be acceptable to say that this will return a
value from the range of values the primary owner of this counter was
holding in the timeframe between the method is being invoked and the
time the value is returned.
Could it optionally be integrated with Total Order ? Transactions?
void increment() //async or sync increment. default add(1)
void decrement() //async or sync decrement. default add(-1)
void add(long) //async or sync add.
void reset() //resets to initial value
Note: Tried to make the interface as simple as possible with support for
sync and async operations. To avoid any confusion, I consider an async
operation as happening somewhat in the future, i.e. eventually
increments/decrements.
The sync operation happens somewhat during the method execution.
interface AtomiCounter extends Counter
long addAndGet() //adds a returns the new value. sync operation
long incrementAndGet() //increments and returns the new value. sync
operation. default addAndGet(1)
long decrementAndGet() //decrements and returns the new value. sync
operation. default addAndGet(-1)
interface AdvancedCounter extends Counter
long getMin/MaxThreshold() //returns the min and max threshold value
"threshold" ??
void add/removeListener() //adds a listener that is invoked when
the
value change. Can be extended to notify when it is "reseted" and when
the threshold is reached.
Note: should this interface be splitted?
I'd prefer a single interface, with reduced redundancy.
For example, is there really a benefit in having a "void increment()"
and also a "long addAndGet()" ? [Besides The fact that only the first
one can benefit of an async option]
Besides, I am no longer sure that it's good thing that methods in
Infinispan can be async vs sync depending on configuration switches;
I'd rather make it explicit in the signature and simplify the
configuration by removing such a flag.
Making the methods which are async-capable to look "explicitly async"
should also allow us to add completeable futures & similar.
== Details ==
This is what I have in mind. Two counter managers: one based on JGroups
counter and another one based on Infinispan cache.
The first one creates AtomicCounters and it first perfectly. All
counters are created with an initial value (zero by default)
The second generates counters with all the options available. It can mix
sync/async operation and all counters will be in the same cache. The
cache will be configure by us and it would be an internal cache. This
will use all the features available in the cache.
Rather than a split organized on the backing implementation details,
I'd prefer to see a split based on purpose of the counter.
For example JGroups has different styles of counters: one might need
one to do an atomic increment which is monotonically increasing across
the whole cluster - from the point of view from an omniscent observer
- but in many cases we just need the generation of a guarantee of a
monotonic unique number: that allows for example to have each node
pre-allocate a range of values and pre-fetch a new block while its
running out of values.
For some systems this is not acceptable because a failing server might
result in some of its allocated numbers to not ever be used, so
creating gaps; for others it's not acceptable that the figures
returned by the "monotonic counter" are not monotonic across each
other when confronting results from different cluster nodes, but for
many use cases that's acceptable. My point being that we need to be
able to choose between these possible semantics.
As an Infinispan consumer being able to express such details is
essential, while having to choose between "backed by JGroups counters"
or not is irrelevant and actually makes it harder.
In terms of Infinispan integration, the most important point I'd like
to see is persistence: i.e. make sure you can store them in a
CacheStore.
How we do that efficiently will also affect design; for example I was
expecting to see a Counter as something which is stored in a Cache, so
inheriting details such as configured CacheStore(s) and number of
owners.
Thanks for starting this!
Sanne
Configuration-wise, I'm thinking about 2 parameters: number of backups
and timeout (for sync operations).
So, comment bellow and let me know alternatives, improvement or if I
missed something.
ps. I also consider implement a counter based on JGroups-raft but I
believe it is an overkill.
ps2. sorry for the long email :( I tried to be shorter as possible.
Cheers,
Pedro
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
9:26 a.m.
I second Sanne's opinion about the sync/async API: make the API express
the synchronicity directly. I would even propose that for synchronous
methods, there's only the CompletableFuture<Void> or
CompletableFuture<Long> variant; we are already reaching the concurrency
limits of application that lets threads block, so let's give user a hint
that he should use such notification API. If anyone prefers the sync
variant, he can always use get().
Let's settle on some nomenclature, too, because these JGroups-counters
and RAFT-counters don't have commonly known properties. It almost seems
that the term *counter* is so overloaded that we shouldn't use it all.
There is a widely known j.u.c.AtomicLong, so if we want to implement
that (strict JMM-like properties), let's call it AtomicLong. Does not
have to follow j.u.c.AtomicLong API, but it's definitely CP.
Something providing unique values should be called Sequence. This will
be probably the one batching ranges (therefore possibly with gaps). By
default non-monotonic, but monotonicity could be a ctor arg (in a
similar way as fairness is set for j.u.concurrent.* classes).
As for CRDTs, I can't imagine how this could be easily built on top of
current 'passive' cache (without any syncing). But as for names *CRDT
counter* is explanatory enough.
If someone needs quotas, let's create Quota according to their needs
(soft and hard threshold, fluent operation below soft, some jitter when
above that). It seems that this will be closest to Sequence by reserving
some ranges. Don't let them shoot themselves into foot with some liger
counter.
And I hope you'll build these on top of the functional API, with at most
one RPC per operation.
Radim
On 03/14/2016 10:27 PM, Sanne Grinovero wrote:
Great starting point!
Some comments inline:
On 14 March 2016 at 19:14, Pedro Ruivo <pedro(a)infinispan.org> wrote:
> Hi everybody,
>
> Discussion about distributed counters.
>
> == Public API ==
>
> interface Counter
As a user, how do I get a Counter instance? From a the CacheContainer interface?
Will they have their own configuration section in the configuration file?
> String getName() //counter name
> long get() //current value. may return stale value due to concurrent
> operations to other nodes.
This is what puzzles me the most. I'm not sure if the feature is
actually useful, unless we can clearly state how far outdated the
value could be.
I think a slightly more formal definition would be in order. For
example I think it would be acceptable to say that this will return a
value from the range of values the primary owner of this counter was
holding in the timeframe between the method is being invoked and the
time the value is returned.
Could it optionally be integrated with Total Order ? Transactions?
> void increment() //async or sync increment. default add(1)
> void decrement() //async or sync decrement. default add(-1)
> void add(long) //async or sync add.
> void reset() //resets to initial value
>
> Note: Tried to make the interface as simple as possible with support for
> sync and async operations. To avoid any confusion, I consider an async
> operation as happening somewhat in the future, i.e. eventually
> increments/decrements.
> The sync operation happens somewhat during the method execution.
>
> interface AtomiCounter extends Counter
>
> long addAndGet() //adds a returns the new value. sync operation
> long incrementAndGet() //increments and returns the new value. sync
> operation. default addAndGet(1)
> long decrementAndGet() //decrements and returns the new value. sync
> operation. default addAndGet(-1)
>
> interface AdvancedCounter extends Counter
>
> long getMin/MaxThreshold() //returns the min and max threshold value
"threshold" ??
> void add/removeListener() //adds a listener that is invoked when the
> value change. Can be extended to notify when it is "reseted" and when
> the threshold is reached.
>
> Note: should this interface be splitted?
I'd prefer a single interface, with reduced redundancy.
For example, is there really a benefit in having a "void increment()"
and also a "long addAndGet()" ? [Besides The fact that only the first
one can benefit of an async option]
Besides, I am no longer sure that it's good thing that methods in
Infinispan can be async vs sync depending on configuration switches;
I'd rather make it explicit in the signature and simplify the
configuration by removing such a flag.
Making the methods which are async-capable to look "explicitly async"
should also allow us to add completeable futures & similar.
> == Details ==
>
> This is what I have in mind. Two counter managers: one based on JGroups
> counter and another one based on Infinispan cache.
> The first one creates AtomicCounters and it first perfectly. All
> counters are created with an initial value (zero by default)
> The second generates counters with all the options available. It can mix
> sync/async operation and all counters will be in the same cache. The
> cache will be configure by us and it would be an internal cache. This
> will use all the features available in the cache.
Rather than a split organized on the backing implementation details,
I'd prefer to see a split based on purpose of the counter.
For example JGroups has different styles of counters: one might need
one to do an atomic increment which is monotonically increasing across
the whole cluster - from the point of view from an omniscent observer
- but in many cases we just need the generation of a guarantee of a
monotonic unique number: that allows for example to have each node
pre-allocate a range of values and pre-fetch a new block while its
running out of values.
For some systems this is not acceptable because a failing server might
result in some of its allocated numbers to not ever be used, so
creating gaps; for others it's not acceptable that the figures
returned by the "monotonic counter" are not monotonic across each
other when confronting results from different cluster nodes, but for
many use cases that's acceptable. My point being that we need to be
able to choose between these possible semantics.
As an Infinispan consumer being able to express such details is
essential, while having to choose between "backed by JGroups counters"
or not is irrelevant and actually makes it harder.
In terms of Infinispan integration, the most important point I'd like
to see is persistence: i.e. make sure you can store them in a
CacheStore.
How we do that efficiently will also affect design; for example I was
expecting to see a Counter as something which is stored in a Cache, so
inheriting details such as configured CacheStore(s) and number of
owners.
Thanks for starting this!
Sanne
> Configuration-wise, I'm thinking about 2 parameters: number of backups
> and timeout (for sync operations).
>
> So, comment bellow and let me know alternatives, improvement or if I
> missed something.
>
> ps. I also consider implement a counter based on JGroups-raft but I
> believe it is an overkill.
> ps2. sorry for the long email :( I tried to be shorter as possible.
>
> Cheers,
> Pedro
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
3:15 p.m.
On Mar 15, 2016, at 3:26 AM, Radim Vansa <rvansa(a)redhat.com>
wrote:
I second Sanne's opinion about the sync/async API: make the API express
the synchronicity directly. I would even propose that for synchronous
methods, there's only the CompletableFuture<Void> or
CompletableFuture<Long> variant; we are already reaching the concurrency
limits of application that lets threads block, so let's give user a hint
that he should use such notification API. If anyone prefers the sync
variant, he can always use get().
Let's settle on some nomenclature, too, because these JGroups-counters
and RAFT-counters don't have commonly known properties. It almost seems
that the term *counter* is so overloaded that we shouldn't use it all.
There is a widely known j.u.c.AtomicLong, so if we want to implement
that (strict JMM-like properties), let's call it AtomicLong. Does not
have to follow j.u.c.AtomicLong API, but it's definitely CP.
Something providing unique values should be called Sequence. This will
be probably the one batching ranges (therefore possibly with gaps). By
default non-monotonic, but monotonicity could be a ctor arg (in a
similar way as fairness is set for j.u.concurrent.* classes).
+1 for differentiating in the API between a counter that a client increments (or resets)
and a sequence generator that a client can use to obtain (non)monotonic values.
As for CRDTs, I can't imagine how this could be easily built on top of
current 'passive' cache (without any syncing). But as for names *CRDT
counter* is explanatory enough.
If someone needs quotas, let's create Quota according to their needs
(soft and hard threshold, fluent operation below soft, some jitter when
above that). It seems that this will be closest to Sequence by reserving
some ranges. Don't let them shoot themselves into foot with some liger
counter.
+1 for creating a specific interface to enable quotas. Perhaps it is implemented with some
more general purpose functionality that might be exposed in the future, but for now why
not keep it narrowly focused.
And I hope you'll build these on top of the functional API, with at most
one RPC per operation.
Radim
On 03/14/2016 10:27 PM, Sanne Grinovero wrote:
> Great starting point!
> Some comments inline:
>
> On 14 March 2016 at 19:14, Pedro Ruivo <pedro(a)infinispan.org> wrote:
>> Hi everybody,
>>
>> Discussion about distributed counters.
>>
>> == Public API ==
>>
>> interface Counter
> As a user, how do I get a Counter instance? From a the CacheContainer interface?
>
> Will they have their own configuration section in the configuration file?
>
>> String getName() //counter name
>> long get() //current value. may return stale value due to concurrent
>> operations to other nodes.
> This is what puzzles me the most. I'm not sure if the feature is
> actually useful, unless we can clearly state how far outdated the
> value could be.
>
> I think a slightly more formal definition would be in order. For
> example I think it would be acceptable to say that this will return a
> value from the range of values the primary owner of this counter was
> holding in the timeframe between the method is being invoked and the
> time the value is returned.
>
> Could it optionally be integrated with Total Order ? Transactions?
>
>> void increment() //async or sync increment. default add(1)
>> void decrement() //async or sync decrement. default add(-1)
>> void add(long) //async or sync add.
>> void reset() //resets to initial value
>>
>> Note: Tried to make the interface as simple as possible with support for
>> sync and async operations. To avoid any confusion, I consider an async
>> operation as happening somewhat in the future, i.e. eventually
>> increments/decrements.
>> The sync operation happens somewhat during the method execution.
>>
>> interface AtomiCounter extends Counter
>>
>> long addAndGet() //adds a returns the new value. sync operation
>> long incrementAndGet() //increments and returns the new value. sync
>> operation. default addAndGet(1)
>> long decrementAndGet() //decrements and returns the new value. sync
>> operation. default addAndGet(-1)
>>
>> interface AdvancedCounter extends Counter
>>
>> long getMin/MaxThreshold() //returns the min and max threshold value
> "threshold" ??
>
>> void add/removeListener() //adds a listener that is invoked when the
>> value change. Can be extended to notify when it is "reseted" and when
>> the threshold is reached.
>>
>> Note: should this interface be splitted?
> I'd prefer a single interface, with reduced redundancy.
> For example, is there really a benefit in having a "void increment()"
> and also a "long addAndGet()" ? [Besides The fact that only the first
> one can benefit of an async option]
>
> Besides, I am no longer sure that it's good thing that methods in
> Infinispan can be async vs sync depending on configuration switches;
> I'd rather make it explicit in the signature and simplify the
> configuration by removing such a flag.
>
> Making the methods which are async-capable to look "explicitly async"
> should also allow us to add completeable futures & similar.
>
>> == Details ==
>>
>> This is what I have in mind. Two counter managers: one based on JGroups
>> counter and another one based on Infinispan cache.
>> The first one creates AtomicCounters and it first perfectly. All
>> counters are created with an initial value (zero by default)
>> The second generates counters with all the options available. It can mix
>> sync/async operation and all counters will be in the same cache. The
>> cache will be configure by us and it would be an internal cache. This
>> will use all the features available in the cache.
> Rather than a split organized on the backing implementation details,
> I'd prefer to see a split based on purpose of the counter.
>
> For example JGroups has different styles of counters: one might need
> one to do an atomic increment which is monotonically increasing across
> the whole cluster - from the point of view from an omniscent observer
> - but in many cases we just need the generation of a guarantee of a
> monotonic unique number: that allows for example to have each node
> pre-allocate a range of values and pre-fetch a new block while its
> running out of values.
>
> For some systems this is not acceptable because a failing server might
> result in some of its allocated numbers to not ever be used, so
> creating gaps; for others it's not acceptable that the figures
> returned by the "monotonic counter" are not monotonic across each
> other when confronting results from different cluster nodes, but for
> many use cases that's acceptable. My point being that we need to be
> able to choose between these possible semantics.
>
> As an Infinispan consumer being able to express such details is
> essential, while having to choose between "backed by JGroups counters"
> or not is irrelevant and actually makes it harder.
>
> In terms of Infinispan integration, the most important point I'd like
> to see is persistence: i.e. make sure you can store them in a
> CacheStore.
> How we do that efficiently will also affect design; for example I was
> expecting to see a Counter as something which is stored in a Cache, so
> inheriting details such as configured CacheStore(s) and number of
> owners.
>
> Thanks for starting this!
>
> Sanne
>
>> Configuration-wise, I'm thinking about 2 parameters: number of backups
>> and timeout (for sync operations).
>>
>> So, comment bellow and let me know alternatives, improvement or if I
>> missed something.
>>
>> ps. I also consider implement a counter based on JGroups-raft but I
>> believe it is an overkill.
>> ps2. sorry for the long email :( I tried to be shorter as possible.
>>
>> Cheers,
>> Pedro
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
12:58 p.m.
Comments inline
On 03/14/2016 09:27 PM, Sanne Grinovero wrote:
On 14 March 2016 at 19:14, Pedro Ruivo <pedro(a)infinispan.org>
wrote:
As a user, how do I get a Counter instance? From a the CacheContainer interface?
You can get a Counter from a CounterManager. The CounterManager is
associated to the CacheManager.
I have two options to get the CounterManager:
CacheManager.as(CouterManager.class), as discussed during the Rome
meeting (2015). This will change the API in the core but it allows to
introduce the other building blocked (locks, sequences, latches, ...)
The other alternative I have in mind is CounterMager.get(CacheManager).
The only advantage I see is no need to change the API in core.
guys... comment what alternative do you prefer...
Will they have their own configuration section in the configuration file?
I have no idea here... probably something like the cache store
configurations would work.
> long get() //current value. may return stale value due to
concurrent
> operations to other nodes.
This is what puzzles me the most. I'm not sure if the feature is
actually useful, unless we can clearly state how far outdated the
value could be.
I think a slightly more formal definition would be in order. For
example I think it would be acceptable to say that this will return a
value from the range of values the primary owner of this counter was
holding in the timeframe between the method is being invoked and the
time the value is returned.
Could it optionally be integrated with Total Order ? Transactions?
That's why we are having this discussion. If someone heard some
requirements from an user, it would be good to write them here.
IMO, at least, the get() must include all the finished operations in the
local node.
>
> long getMin/MaxThreshold() //returns the min and max threshold value
"threshold" ??
See Tristan's reply.
I'd prefer a single interface, with reduced redundancy.
For example, is there really a benefit in having a "void increment()"
and also a "long addAndGet()" ? [Besides The fact that only the first
one can benefit of an async option]
Besides, I am no longer sure that it's good thing that methods in
Infinispan can be async vs sync depending on configuration switches;
I'd rather make it explicit in the signature and simplify the
configuration by removing such a flag.
Making the methods which are async-capable to look "explicitly async"
should also allow us to add completeable futures & similar.
So, lets revamp the API. We can drop the AtomiCounter and create a
simple Counter interface like:
String getName()
CompletableFuture<Long> addAndGet()
long get()
void reset()
It is the simplest we can get and it covers all the semantics we need so
far. If something does not fit, we can start improving it.
Rather than a split organized on the backing implementation details,
I'd prefer to see a split based on purpose of the counter.
So, what about multiple CounterManager where in each implementation we
specified what semantic the Counter created ensures?
For example JGroups has different styles of counters: one might need
one to do an atomic increment which is monotonically increasing across
the whole cluster - from the point of view from an omniscent observer
- but in many cases we just need the generation of a guarantee of a
monotonic unique number: that allows for example to have each node
pre-allocate a range of values and pre-fetch a new block while its
running out of values.
you lost me in the pre-allocate range of values. I think you are talking
about sequences (we also discussed that in Rome) but I don't think it
should be the primary objective of a counter.
For some systems this is not acceptable because a failing server might
result in some of its allocated numbers to not ever be used, so
creating gaps; for others it's not acceptable that the figures
returned by the "monotonic counter" are not monotonic across each
other when confronting results from different cluster nodes, but for
many use cases that's acceptable. My point being that we need to be
able to choose between these possible semantics.
As an Infinispan consumer being able to express such details is
essential, while having to choose between "backed by JGroups counters"
or not is irrelevant and actually makes it harder.
In terms of Infinispan integration, the most important point I'd like
to see is persistence: i.e. make sure you can store them in a
CacheStore.
How we do that efficiently will also affect design; for example I was
expecting to see a Counter as something which is stored in a Cache, so
inheriting details such as configured CacheStore(s) and number of
owners.
I'm ok to support persistence but I don't see any advantage. Does anyone
have a use case for it?
1:57 p.m.
On 03/15/2016 12:58 PM, Pedro Ruivo wrote:
On 03/14/2016 09:27 PM, Sanne Grinovero wrote:
> Rather than a split organized on the backing implementation details,
> I'd prefer to see a split based on purpose of the counter.
So, what about multiple CounterManager where in each implementation we
specified what semantic the Counter created ensures?
Please, don't do that. Different implementation = does the same thing
but has different internals. If it has different semantics (does
something different), use different interface. This way users will just
swap in 'more performant' implementation without thinking about the
consequences.
My 2c
R.
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
11:17 p.m.
What are the requirements? What are the distributed counters for? Is the counter to be
monotonically increasing? Can there be any missed values? Does the counter need to
increment and decrement? What is the *smallest* API you need initially?
There are two choices when implementing a distributed counter: use central coordination
(like JGroups counters), or use independent counters on separate machines that will
eventually converge to the correct value (CRDTs). Coordinated counters are expensive and
therefore slow, and can suffer from problems during network or cluster problems. For
example, what happens during a split brain? OTOH, CRDTs are decentralized so therefore are
very fast, easily merged, and fault tolerant; they’re excellent when counting things that
are occurring independently and therefore may be more suited for
monitoring/metrics/accumulators/etc. Both have very different behaviors under ideal and
failure scenarios, have different performance and consistency guarantees, and are useful
in different scenarios. Make sure you choose accordingly.
For information about CRDTs, make sure you’ve read the CRDT paper by Shapiro:
http://hal.upmc.fr/inria-00555588/document
<http://hal.upmc.fr/inria-00555588/document>
Randall
On Mar 14, 2016, at 2:14 PM, Pedro Ruivo <pedro(a)infinispan.org>
wrote:
Hi everybody,
Discussion about distributed counters.
== Public API ==
interface Counter
String getName() //counter name
long get() //current value. may return stale value due to concurrent
operations to other nodes.
void increment() //async or sync increment. default add(1)
void decrement() //async or sync decrement. default add(-1)
void add(long) //async or sync add.
void reset() //resets to initial value
Note: Tried to make the interface as simple as possible with support for
sync and async operations. To avoid any confusion, I consider an async
operation as happening somewhat in the future, i.e. eventually
increments/decrements.
The sync operation happens somewhat during the method execution.
interface AtomiCounter extends Counter
long addAndGet() //adds a returns the new value. sync operation
long incrementAndGet() //increments and returns the new value. sync
operation. default addAndGet(1)
long decrementAndGet() //decrements and returns the new value. sync
operation. default addAndGet(-1)
interface AdvancedCounter extends Counter
long getMin/MaxThreshold() //returns the min and max threshold value
void add/removeListener() //adds a listener that is invoked when the
value change. Can be extended to notify when it is "reseted" and when
the threshold is reached.
Note: should this interface be splitted?
== Details ==
This is what I have in mind. Two counter managers: one based on JGroups
counter and another one based on Infinispan cache.
The first one creates AtomicCounters and it first perfectly. All
counters are created with an initial value (zero by default)
The second generates counters with all the options available. It can mix
sync/async operation and all counters will be in the same cache. The
cache will be configure by us and it would be an internal cache. This
will use all the features available in the cache.
Configuration-wise, I'm thinking about 2 parameters: number of backups
and timeout (for sync operations).
So, comment bellow and let me know alternatives, improvement or if I
missed something.
ps. I also consider implement a counter based on JGroups-raft but I
believe it is an overkill.
ps2. sorry for the long email :( I tried to be shorter as possible.
Cheers,
Pedro
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
8:12 a.m.
On 14/03/16 23:17, Randall Hauch wrote:
What are the requirements? What are the distributed counters for? Is
the
counter to be monotonically increasing? Can there be any missed values?
Does the counter need to increment and decrement? What is the *smallest*
API you need initially?
There are two choices when implementing a distributed counter: use
central coordination (like JGroups counters), or use independent
counters on separate machines that will eventually converge to the
correct value (CRDTs). Coordinated counters are expensive and therefore
slow, and can suffer from problems during network or cluster problems.
The question is what do you get for this? If your app can't afford
duplicate counter values during a network partition, then - yes - there
is some overhead. CRDTs won't be able to guarantee this property. OTOH
CRDTs are fast when you only care about some sort of eventual
consistency, and don't need 'hard' consistency.
For example, what happens during a split brain? OTOH, CRDTs are
decentralized so therefore are very fast, easily merged, and fault
tolerant;
Yes, CRDTS are AP whereas jgroups-raft counters are CP. JGroups
counters, otoh, are CRAP (consistent, reliable, available and
partition-aware.
Take the last sentence with a grain of salt :-)
they’re excellent when counting things that are occurring
independently and therefore may be more suited for
monitoring/metrics/accumulators/etc. Both have very different behaviors
under ideal and failure scenarios, have different performance and
consistency guarantees, and are useful in different scenarios. Make sure
you choose accordingly.
For information about CRDTs, make sure you’ve read the CRDT paper by
Shapiro: http://hal.upmc.fr/inria-00555588/document
Randall
> On Mar 14, 2016, at 2:14 PM, Pedro Ruivo <pedro(a)infinispan.org
> <mailto:pedro@infinispan.org>> wrote:
>
> Hi everybody,
>
> Discussion about distributed counters.
>
> == Public API ==
>
> interface Counter
>
> String getName() //counter name
> long get()//current value. may return stale value due to concurrent
> operations to other nodes.
> void increment() //async or sync increment. default add(1)
> void decrement() //async or sync decrement. default add(-1)
> void add(long) //async or sync add.
> void reset() //resets to initial value
>
> Note: Tried to make the interface as simple as possible with support for
> sync and async operations. To avoid any confusion, I consider an async
> operation as happening somewhat in the future, i.e. eventually
> increments/decrements.
> The sync operation happens somewhat during the method execution.
>
> interface AtomiCounter extends Counter
>
> long addAndGet() //adds a returns the new value. sync operation
> long incrementAndGet() //increments and returns the new value. sync
> operation. default addAndGet(1)
> long decrementAndGet() //decrements and returns the new value. sync
> operation. default addAndGet(-1)
>
> interface AdvancedCounter extends Counter
>
> long getMin/MaxThreshold() //returns the min and max threshold value
> void add/removeListener() //adds a listener that is invoked when the
> value change. Can be extended to notify when it is "reseted" and when
> the threshold is reached.
>
> Note: should this interface be splitted?
>
> == Details ==
>
> This is what I have in mind. Two counter managers: one based on JGroups
> counter and another one based on Infinispan cache.
> The first one creates AtomicCounters and it first perfectly. All
> counters are created with an initial value (zero by default)
> The second generates counters with all the options available. It can mix
> sync/async operation and all counters will be in the same cache. The
> cache will be configure by us and it would be an internal cache. This
> will use all the features available in the cache.
>
> Configuration-wise, I'm thinking about 2 parameters: number of backups
> and timeout (for sync operations).
>
> So, comment bellow and let me know alternatives, improvement or if I
> missed something.
>
> ps. I also consider implement a counter based on JGroups-raft but I
> believe it is an overkill.
> ps2. sorry for the long email :( I tried to be shorter as possible.
>
> Cheers,
> Pedro
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Bela Ban, JGroups lead (http://www.jgroups.org)
3:17 p.m.
On Mar 15, 2016, at 2:12 AM, Bela Ban <bban(a)redhat.com> wrote:
On 14/03/16 23:17, Randall Hauch wrote:
> What are the requirements? What are the distributed counters for? Is the
> counter to be monotonically increasing? Can there be any missed values?
> Does the counter need to increment and decrement? What is the *smallest*
> API you need initially?
>
> There are two choices when implementing a distributed counter: use
> central coordination (like JGroups counters), or use independent
> counters on separate machines that will eventually converge to the
> correct value (CRDTs). Coordinated counters are expensive and therefore
> slow, and can suffer from problems during network or cluster problems.
The question is what do you get for this? If your app can't afford
duplicate counter values during a network partition, then - yes - there
is some overhead. CRDTs won't be able to guarantee this property. OTOH
CRDTs are fast when you only care about some sort of eventual
consistency, and don't need 'hard' consistency.
To be clear, I’m not saying they are interchangeable. They have very different properties,
which is why the requirements will help determine which of them (if any) are applicable.
> For example, what happens during a split brain? OTOH, CRDTs are
> decentralized so therefore are very fast, easily merged, and fault
> tolerant;
Yes, CRDTS are AP whereas jgroups-raft counters are CP. JGroups
counters, otoh, are CRAP (consistent, reliable, available and
partition-aware.
Take the last sentence with a grain of salt :-)
> they’re excellent when counting things that are occurring
> independently and therefore may be more suited for
> monitoring/metrics/accumulators/etc. Both have very different behaviors
> under ideal and failure scenarios, have different performance and
> consistency guarantees, and are useful in different scenarios. Make sure
> you choose accordingly.
>
> For information about CRDTs, make sure you’ve read the CRDT paper by
> Shapiro: http://hal.upmc.fr/inria-00555588/document
<http://hal.upmc.fr/inria-00555588/document>
>
> Randall
>
>> On Mar 14, 2016, at 2:14 PM, Pedro Ruivo <pedro(a)infinispan.org
<mailto:pedro@infinispan.org>
>> <mailto:pedro@infinispan.org <mailto:pedro@infinispan.org>>>
wrote:
>>
>> Hi everybody,
>>
>> Discussion about distributed counters.
>>
>> == Public API ==
>>
>> interface Counter
>>
>> String getName() //counter name
>> long get()//current value. may return stale value due to concurrent
>> operations to other nodes.
>> void increment() //async or sync increment. default add(1)
>> void decrement() //async or sync decrement. default add(-1)
>> void add(long) //async or sync add.
>> void reset() //resets to initial value
>>
>> Note: Tried to make the interface as simple as possible with support for
>> sync and async operations. To avoid any confusion, I consider an async
>> operation as happening somewhat in the future, i.e. eventually
>> increments/decrements.
>> The sync operation happens somewhat during the method execution.
>>
>> interface AtomiCounter extends Counter
>>
>> long addAndGet() //adds a returns the new value. sync operation
>> long incrementAndGet() //increments and returns the new value. sync
>> operation. default addAndGet(1)
>> long decrementAndGet() //decrements and returns the new value. sync
>> operation. default addAndGet(-1)
>>
>> interface AdvancedCounter extends Counter
>>
>> long getMin/MaxThreshold() //returns the min and max threshold value
>> void add/removeListener() //adds a listener that is invoked when the
>> value change. Can be extended to notify when it is "reseted" and when
>> the threshold is reached.
>>
>> Note: should this interface be splitted?
>>
>> == Details ==
>>
>> This is what I have in mind. Two counter managers: one based on JGroups
>> counter and another one based on Infinispan cache.
>> The first one creates AtomicCounters and it first perfectly. All
>> counters are created with an initial value (zero by default)
>> The second generates counters with all the options available. It can mix
>> sync/async operation and all counters will be in the same cache. The
>> cache will be configure by us and it would be an internal cache. This
>> will use all the features available in the cache.
>>
>> Configuration-wise, I'm thinking about 2 parameters: number of backups
>> and timeout (for sync operations).
>>
>> So, comment bellow and let me know alternatives, improvement or if I
>> missed something.
>>
>> ps. I also consider implement a counter based on JGroups-raft but I
>> believe it is an overkill.
>> ps2. sorry for the long email :( I tried to be shorter as possible.
>>
>> Cheers,
>> Pedro
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
<mailto:infinispan-dev@lists.jboss.org
<mailto:infinispan-dev@lists.jboss.org>>
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
<https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
<https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>
--
Bela Ban, JGroups lead (http://www.jgroups.org <http://www.jgroups.org/>)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
<https://lists.jboss.org/mailman/listinfo/infinispan-dev>
10:04 a.m.
For information about CRDTs, make sure you’ve read the CRDT paper by
Shapiro: http://hal.upmc.fr/inria-00555588/document
On that topic, I recently came across the paper "Scalable Eventually
Consistent Counters over Unreliable Networks" [1] which highlights some of
the scalability problems with CRDT counters at large scale: namely the
propagation and maintenance of a version vector that keeps growing
over time and requires broadcasts across all members. The paper then
describes how some implementors of CRDT counters tackled this limitation
by using "server side only" counters, and the inherent issues with that
approach. Finally it proposes an alternative called "Handoff counters" in
order
to overcome the scalability issue.
[1] http://arxiv.org/pdf/1307.3207v1.pdf
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
8 a.m.
What are the requirements? To offer a cluster wide counter from an
Infinispan cache? Or is this going to be used internally, by Infinispan?
I've implemented 2 counters, in JGroups and in jgroups-raft. Here are
their properties:
#1 JGroups counter:
- Cluster wide _named_ counters with atomicity guarantees
- A counter is always managed by the current coordinator
- In a cluster split, you can end up with different counter instances
(one per coordinator)
- When a cluster partition heals, the current coordinator's counter
'wins', ie. values from the other counters are lost, even if they were
updated last
#2 jgroups-raft counter:
- Counter is managed by the leader
- A counter will not be accessible when there's no leader (no majority):
this is CP in CAP
- Counters cannot diverge in split brain scenarios; there's always at
most 1 counter (or 0)
If you can live with multiple counters being present in light of network
partitiond, then pick #1 as this is faster than #2 (which requires disk
writes for updates), and is always available, whereas #2 can be
non-available.
I'm thinking about changing the way #1 works. Currently, it uses the
next-in-line to backup a counter value on each update. I'd like to
remove this and replace it with a reconciliation round when the
coordinator changes: the new coordinator asks all members for their
current counter values. Requires a bit more work on a view change, but
less in steady state mode. Perhaps I can make it configurable.
On 14/03/16 20:14, Pedro Ruivo wrote:
Hi everybody,
Discussion about distributed counters.
== Public API ==
interface Counter
String getName() //counter name
long get() //current value. may return stale value due to concurrent
operations to other nodes.
void increment() //async or sync increment. default add(1)
void decrement() //async or sync decrement. default add(-1)
void add(long) //async or sync add.
void reset() //resets to initial value
Note: Tried to make the interface as simple as possible with support for
sync and async operations. To avoid any confusion, I consider an async
operation as happening somewhat in the future, i.e. eventually
increments/decrements.
The sync operation happens somewhat during the method execution.
interface AtomiCounter extends Counter
long addAndGet() //adds a returns the new value. sync operation
long incrementAndGet() //increments and returns the new value. sync
operation. default addAndGet(1)
long decrementAndGet() //decrements and returns the new value. sync
operation. default addAndGet(-1)
interface AdvancedCounter extends Counter
long getMin/MaxThreshold() //returns the min and max threshold value
void add/removeListener() //adds a listener that is invoked when the
value change. Can be extended to notify when it is "reseted" and when
the threshold is reached.
Note: should this interface be splitted?
== Details ==
This is what I have in mind. Two counter managers: one based on JGroups
counter and another one based on Infinispan cache.
The first one creates AtomicCounters and it first perfectly. All
counters are created with an initial value (zero by default)
The second generates counters with all the options available. It can mix
sync/async operation and all counters will be in the same cache. The
cache will be configure by us and it would be an internal cache. This
will use all the features available in the cache.
Configuration-wise, I'm thinking about 2 parameters: number of backups
and timeout (for sync operations).
So, comment bellow and let me know alternatives, improvement or if I
missed something.
ps. I also consider implement a counter based on JGroups-raft but I
believe it is an overkill.
ps2. sorry for the long email :( I tried to be shorter as possible.
Cheers,
Pedro
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Bela Ban, JGroups lead (http://www.jgroups.org)
8:38 a.m.
On 15/03/2016 08:00, Bela Ban wrote:
What are the requirements? To offer a cluster wide counter from an
Infinispan cache? Or is this going to be used internally, by Infinispan?
The starting point is a requirement by apiman as a way to manage
throttling/quotas. Currently their requirement is not particularly
strict (i.e. synchronous), but they would like the ability to reset a
counter and possibly to receive an event when a certain threshold is
reached.
Obviously we'd like to cater to more use-cases, so the design should
take that into account.
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
10:12 a.m.
Good start but rather than coming out with a general design for a counter, I'd try to
start coming out with functionality needed for the most commonly use cases for counters,
e.g.
e.g. if you're counting number of visits to your website, you only need an
incrementing counter (and maybe a reset to be called at the end of the
day/week/month/year?) but you'll never decrement, precision not hugely important?
Could you assume you always increment by 1? It might be more efficient to increment by
N...
e.g. if you're counting number of users logged in at one point to your website, then
you need a counter that both increments and decrements, precision not hugely important?
You probably don't want reset operation? Moreover, could you assume you always
increment/decrement by 1?
e.g. if using a counter to generate a unique identifier that always increases, you need
only increment but precision must be guaranteed, so after increasing and retrieving an old
value can't be retrieved. You probably don't want reset operation? You can
probably assume that increment is only by 1.
...etc
And based on those, see what kind of API you need for each use case. There might be many
more use cases but I'd rather start there than head directly to specifying operations.
My main objective is to avoid that someone that wants to count number of visits to a
website does not end up calling a method that would not make sense for its use case. Doing
this might not be 100% avoidable but hope you see my point.
Cheers,
--
Galder Zamarreño
Infinispan, Red Hat
On 14 Mar 2016, at 20:14, Pedro Ruivo <pedro(a)infinispan.org>
wrote:
Hi everybody,
Discussion about distributed counters.
== Public API ==
interface Counter
String getName() //counter name
long get() //current value. may return stale value due to concurrent
operations to other nodes.
void increment() //async or sync increment. default add(1)
void decrement() //async or sync decrement. default add(-1)
void add(long) //async or sync add.
void reset() //resets to initial value
Note: Tried to make the interface as simple as possible with support for
sync and async operations. To avoid any confusion, I consider an async
operation as happening somewhat in the future, i.e. eventually
increments/decrements.
The sync operation happens somewhat during the method execution.
interface AtomiCounter extends Counter
long addAndGet() //adds a returns the new value. sync operation
long incrementAndGet() //increments and returns the new value. sync
operation. default addAndGet(1)
long decrementAndGet() //decrements and returns the new value. sync
operation. default addAndGet(-1)
interface AdvancedCounter extends Counter
long getMin/MaxThreshold() //returns the min and max threshold value
void add/removeListener() //adds a listener that is invoked when the
value change. Can be extended to notify when it is "reseted" and when
the threshold is reached.
Note: should this interface be splitted?
== Details ==
This is what I have in mind. Two counter managers: one based on JGroups
counter and another one based on Infinispan cache.
The first one creates AtomicCounters and it first perfectly. All
counters are created with an initial value (zero by default)
The second generates counters with all the options available. It can mix
sync/async operation and all counters will be in the same cache. The
cache will be configure by us and it would be an internal cache. This
will use all the features available in the cache.
Configuration-wise, I'm thinking about 2 parameters: number of backups
and timeout (for sync operations).
So, comment bellow and let me know alternatives, improvement or if I
missed something.
ps. I also consider implement a counter based on JGroups-raft but I
believe it is an overkill.
ps2. sorry for the long email :( I tried to be shorter as possible.
Cheers,
Pedro
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
11:16 a.m.
On 29/03/2016 10:12, Galder Zamarreño wrote:
Good start but rather than coming out with a general design for a
counter, I'd try to start coming out with functionality needed for the most commonly
use cases for counters, e.g.
e.g. if you're counting number of visits to your website, you only need an
incrementing counter (and maybe a reset to be called at the end of the
day/week/month/year?) but you'll never decrement, precision not hugely important?
Could you assume you always increment by 1? It might be more efficient to increment by
N...
e.g. if you're counting number of users logged in at one point to
your website, then you need a counter that both increments and decrements, precision not
hugely important? You probably don't want reset operation? Moreover, could you assume
you always increment/decrement by 1?
You want an "Adder" for both of these cases [1].
e.g. if using a counter to generate a unique identifier that always
increases, you need only increment but precision must be guaranteed, so after increasing
and retrieving an old value can't be retrieved. You probably don't want reset
operation? You can probably assume that increment is only by 1.
This is a sequence, and, even in this case, there are two possible variants:
- one where you just need uniqueness but monotonic increment is not
essential (nodes can retrieve preallocated ranges of ids on demand).
- one which is fully monotonic
Tristan
[1]
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/Lon...
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
7:25 p.m.
== Status update ==
Hi all,
The work in progress can be found in [1].
The main interface is:
CompletableFuture<Long> addAndGet(long)
The CompletableFuture will allow to cover a more use cases, for example,
if you need to count the number of visits, you can increment without
waiting it to complete. On other hand, if you need "unique ids", it is
possible to increment and wait for the result.
* Let me know any concerns with it.
Currently, two types of counters can be returned by the counter manager:
"unlimited" and limited (i.e. with thresholds).
"Unlimited" is actually limited to Long.MAX_VALUE but will never throw
any exception when reached.
The limited version throws a CounterException when the min or max
threshold is reached.
In both cases, it doesn't overflow the counter value.
Currently, I'm working in the notification mechanism. So, just ignore
that part of the code :)
Moving to the next problem, I need an opinion how to create the counter
manager.
The current implementation is using a private cache. It has the
advantage of starting the cache when a cache manager starts, however, it
is a bit difficult to configure (e.g. persistence). Also, it does not
allow different configuration for the counter (replicated vs
distributed, persisted or not).
Another alternative, it to construct the counter manager with the cache
manager and a cache name. This looks simpler and it allows the user to
configure it properly and have different counter manager with different
configurations.
* Comments are welcome.
Cheers,
Pedro
[1] https://github.com/pruivo/infinispan/tree/t_counters/distributed-counter
12:34 p.m.
Hi Pedro,
there have been many questions on this thread pointing out the need to
clarify the definition and which requirements it aims to solve. I
don't think we can comment on the API proposal without making that
clear first?
It seems there was some consensus that this doesn't help for the
APIMan use case; could we clarify the goal and use cases then?
thanks,
Sanne
On 29 March 2016 at 18:25, Pedro Ruivo <pedro(a)infinispan.org> wrote:
== Status update ==
Hi all,
The work in progress can be found in [1].
The main interface is:
CompletableFuture<Long> addAndGet(long)
The CompletableFuture will allow to cover a more use cases, for example,
if you need to count the number of visits, you can increment without
waiting it to complete. On other hand, if you need "unique ids", it is
possible to increment and wait for the result.
* Let me know any concerns with it.
Currently, two types of counters can be returned by the counter manager:
"unlimited" and limited (i.e. with thresholds).
"Unlimited" is actually limited to Long.MAX_VALUE but will never throw
any exception when reached.
The limited version throws a CounterException when the min or max
threshold is reached.
In both cases, it doesn't overflow the counter value.
Currently, I'm working in the notification mechanism. So, just ignore
that part of the code :)
Moving to the next problem, I need an opinion how to create the counter
manager.
The current implementation is using a private cache. It has the
advantage of starting the cache when a cache manager starts, however, it
is a bit difficult to configure (e.g. persistence). Also, it does not
allow different configuration for the counter (replicated vs
distributed, persisted or not).
Another alternative, it to construct the counter manager with the cache
manager and a cache name. This looks simpler and it allows the user to
configure it properly and have different counter manager with different
configurations.
* Comments are welcome.
Cheers,
Pedro
[1] https://github.com/pruivo/infinispan/tree/t_counters/distributed-counter
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
2:28 p.m.
I think the main interface definition (as it is) is fine, but it needs
to be subclassed into two different behaviours:
- "sync": the value returned by the CF is consistent across the cluster
- "eventually consistent": the value returned by the CF might be stale
(maybe return its latest "localh" value)
Tristan
On 30/03/2016 12:34, Sanne Grinovero wrote:
Hi Pedro,
there have been many questions on this thread pointing out the need to
clarify the definition and which requirements it aims to solve. I
don't think we can comment on the API proposal without making that
clear first?
It seems there was some consensus that this doesn't help for the
APIMan use case; could we clarify the goal and use cases then?
thanks,
Sanne
On 29 March 2016 at 18:25, Pedro Ruivo <pedro(a)infinispan.org> wrote:
> == Status update ==
>
> Hi all,
>
> The work in progress can be found in [1].
>
> The main interface is:
>
> CompletableFuture<Long> addAndGet(long)
>
> The CompletableFuture will allow to cover a more use cases, for example,
> if you need to count the number of visits, you can increment without
> waiting it to complete. On other hand, if you need "unique ids", it is
> possible to increment and wait for the result.
>
> * Let me know any concerns with it.
>
> Currently, two types of counters can be returned by the counter manager:
> "unlimited" and limited (i.e. with thresholds).
>
> "Unlimited" is actually limited to Long.MAX_VALUE but will never throw
> any exception when reached.
> The limited version throws a CounterException when the min or max
> threshold is reached.
> In both cases, it doesn't overflow the counter value.
>
> Currently, I'm working in the notification mechanism. So, just ignore
> that part of the code :)
>
> Moving to the next problem, I need an opinion how to create the counter
> manager.
>
> The current implementation is using a private cache. It has the
> advantage of starting the cache when a cache manager starts, however, it
> is a bit difficult to configure (e.g. persistence). Also, it does not
> allow different configuration for the counter (replicated vs
> distributed, persisted or not).
>
> Another alternative, it to construct the counter manager with the cache
> manager and a cache name. This looks simpler and it allows the user to
> configure it properly and have different counter manager with different
> configurations.
>
> * Comments are welcome.
>
> Cheers,
> Pedro
>
> [1] https://github.com/pruivo/infinispan/tree/t_counters/distributed-counter
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
3:06 p.m.
On 7 April 2016 at 13:28, Tristan Tarrant <ttarrant(a)redhat.com> wrote:
I think the main interface definition (as it is) is fine, but it
needs
to be subclassed into two different behaviours:
- "sync": the value returned by the CF is consistent across the cluster
- "eventually consistent": the value returned by the CF might be stale
(maybe return its latest "localh" value)
For the "eventually consistent" case, returning a local value might be
fine but you'd need to define also how writes are merged and what
guarantees it aims to provide (or which we want to intentionally
ignore).
And what about the sequence use case? I guess it can wait, just
tracking the need for that too.
Thanks,
Sanne
Tristan
On 30/03/2016 12:34, Sanne Grinovero wrote:
> Hi Pedro,
> there have been many questions on this thread pointing out the need to
> clarify the definition and which requirements it aims to solve. I
> don't think we can comment on the API proposal without making that
> clear first?
>
> It seems there was some consensus that this doesn't help for the
> APIMan use case; could we clarify the goal and use cases then?
>
> thanks,
> Sanne
>
> On 29 March 2016 at 18:25, Pedro Ruivo <pedro(a)infinispan.org> wrote:
>> == Status update ==
>>
>> Hi all,
>>
>> The work in progress can be found in [1].
>>
>> The main interface is:
>>
>> CompletableFuture<Long> addAndGet(long)
>>
>> The CompletableFuture will allow to cover a more use cases, for example,
>> if you need to count the number of visits, you can increment without
>> waiting it to complete. On other hand, if you need "unique ids", it is
>> possible to increment and wait for the result.
>>
>> * Let me know any concerns with it.
>>
>> Currently, two types of counters can be returned by the counter manager:
>> "unlimited" and limited (i.e. with thresholds).
>>
>> "Unlimited" is actually limited to Long.MAX_VALUE but will never throw
>> any exception when reached.
>> The limited version throws a CounterException when the min or max
>> threshold is reached.
>> In both cases, it doesn't overflow the counter value.
>>
>> Currently, I'm working in the notification mechanism. So, just ignore
>> that part of the code :)
>>
>> Moving to the next problem, I need an opinion how to create the counter
>> manager.
>>
>> The current implementation is using a private cache. It has the
>> advantage of starting the cache when a cache manager starts, however, it
>> is a bit difficult to configure (e.g. persistence). Also, it does not
>> allow different configuration for the counter (replicated vs
>> distributed, persisted or not).
>>
>> Another alternative, it to construct the counter manager with the cache
>> manager and a cache name. This looks simpler and it allows the user to
>> configure it properly and have different counter manager with different
>> configurations.
>>
>> * Comments are welcome.
>>
>> Cheers,
>> Pedro
>>
>> [1] https://github.com/pruivo/infinispan/tree/t_counters/distributed-counter
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
3:12 p.m.
On 07/04/2016 15:06, Sanne Grinovero wrote:
For the "eventually consistent" case, returning a local
value might be
fine but you'd need to define also how writes are merged and what
guarantees it aims to provide (or which we want to intentionally
ignore)
Fortunately a counter has commutative ops, so order is unimportant and
it makes things so much easier ;)
And what about the sequence use case? I guess it can wait, just
tracking the need for that too.
Yes, that is a separate thing as it needs far stricter guarantees.
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
3:49 p.m.
Hi all,
This is an update email.
I've just made a preview for the distributed counters. You can find it
in [1]
In this initial version I'm targeting a more consistent approach because
it covers a large uses cases (you'll lose on performance) and it's easy
to test. All the updates are performed atomically, the return value is a
consistent and it doesn't lose values. It supports limits, notifications
and resets.
Since performance is a hot topic (there is no free lunch unfortunately)
and an eventually consistent counter fits in a bunch of uses cases, I'm
planning to add another counter manager with this properties. But I'll
have to remove the reset (it is not commutative with inc/dec) and take
advantage of the functional API (without locking) to have CRDT
properties. Does it make sense to you?
Another task missing is comparing the performance between all the
implementations available and check if it match with the theory above.
Any feedback is more than welcome since there are a quite few things to
keep track on.
Cheers,
Pedro
[1] https://github.com/infinispan/infinispan/pull/4350
On 04/07/2016 02:12 PM, Tristan Tarrant wrote:
On 07/04/2016 15:06, Sanne Grinovero wrote:
> For the "eventually consistent" case, returning a local value might be
> fine but you'd need to define also how writes are merged and what
> guarantees it aims to provide (or which we want to intentionally
> ignore)
Fortunately a counter has commutative ops, so order is unimportant and
it makes things so much easier ;)
> And what about the sequence use case? I guess it can wait, just
> tracking the need for that too.
Yes, that is a separate thing as it needs far stricter guarantees.
Tristan
10:22 a.m.
Note that [1] will make the JGroupsCounter completely async. Currently,
the wrapper is not async.
[1] https://issues.jboss.org/browse/JGRP-2025
On 29/03/16 19:25, Pedro Ruivo wrote:
== Status update ==
Hi all,
The work in progress can be found in [1].
The main interface is:
CompletableFuture<Long> addAndGet(long)
The CompletableFuture will allow to cover a more use cases, for example,
if you need to count the number of visits, you can increment without
waiting it to complete. On other hand, if you need "unique ids", it is
possible to increment and wait for the result.
* Let me know any concerns with it.
Currently, two types of counters can be returned by the counter manager:
"unlimited" and limited (i.e. with thresholds).
"Unlimited" is actually limited to Long.MAX_VALUE but will never throw
any exception when reached.
The limited version throws a CounterException when the min or max
threshold is reached.
In both cases, it doesn't overflow the counter value.
Currently, I'm working in the notification mechanism. So, just ignore
that part of the code :)
Moving to the next problem, I need an opinion how to create the counter
manager.
The current implementation is using a private cache. It has the
advantage of starting the cache when a cache manager starts, however, it
is a bit difficult to configure (e.g. persistence). Also, it does not
allow different configuration for the counter (replicated vs
distributed, persisted or not).
Another alternative, it to construct the counter manager with the cache
manager and a cache name. This looks simpler and it allows the user to
configure it properly and have different counter manager with different
configurations.
* Comments are welcome.
Cheers,
Pedro
[1] https://github.com/pruivo/infinispan/tree/t_counters/distributed-counter
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Bela Ban, JGroups lead (http://www.jgroups.org)
2903
days inactive
2966
days old
infinispan-dev@lists.jboss.org
20 comments
8 participants
participants (8)
-
Bela Ban -
Galder Zamarreño -
Gustavo Fernandes -
Pedro Ruivo -
Radim Vansa -
Randall Hauch -
Sanne Grinovero -
Tristan Tarrant