From rory.odonnell at oracle.com Tue Mar 1 05:14:03 2016
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Tue, 1 Mar 2016 10:14:03 +0000
Subject: [infinispan-dev] Early Access builds of JDK 9 b107 & JDK 9 with
Project Jigsaw b106 (#4540) are available on java.net
Message-ID: <56D56B6B.4080305@oracle.com>
Hi Galder,
Early Access b107 for JDK 9 is
available on java.net, summary of changes are listed here
.
Among other fixes , the following are also included:
* Update class file version to 53 for JDK-9, more info here
* Add support for ES6 collections , more info here
Early Access b106 (#4540) for JDK 9 with
Project Jigsaw is available on java.net.
The March 2016 Quality Outreach Report is posted on here,
thanks again to those who
found & logged bugs against Early Access builds.
Rgds,Rory
--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160301/23485fe8/attachment.html
From gustavo at infinispan.org Wed Mar 2 04:42:37 2016
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Wed, 2 Mar 2016 09:42:37 +0000
Subject: [infinispan-dev] Infinispan 8.2.0.CR1 available
Message-ID:
Dear community,
Infinispan 8.2.0.CR1 has just been released!
For more information please go to:
http://blog.infinispan.org/2016/03/infinispan-820cr1-released.html
Cheers,
Gustavo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160302/0db881ae/attachment.html
From gustavo at infinispan.org Wed Mar 2 09:23:36 2016
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Wed, 2 Mar 2016 14:23:36 +0000
Subject: [infinispan-dev] Infinispan 8 presentation on DevConf.cz 2016
In-Reply-To: <2201094.oE6FtqIMlI@dhcp-10-40-5-198.brq.redhat.com>
References: <761888892.15555969.1454946567441.JavaMail.zimbra@redhat.com>
<956349635.15560183.1454947148356.JavaMail.zimbra@redhat.com>
<2201094.oE6FtqIMlI@dhcp-10-40-5-198.brq.redhat.com>
Message-ID:
On Tue, Feb 9, 2016 at 2:59 AM, Vojtech Juranek wrote:
> Hi Sebastian,
>
> > I'm probably blind - but could you tell me how to find your recording?
> All
> > I can see on the YT link is some Docker stuff....
>
> for some reason it doesn't work in some browsers well, the direct link to
> the
> presentation is [1].
> (and another ISPN related DevConf presentation is on [2])
> HTH
> Vojta
>
Nice Spark Streaming + Infinispan demo! This would make a great quickstart
project!
Gustavo
>
> [1] https://www.youtube.com/watch?v=r0__NEgldzI
> [2] https://www.youtube.com/watch?v=TVXeXM2g7So
> _______________________________________________
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160302/0d1b7594/attachment.html
From galder at redhat.com Wed Mar 2 11:23:18 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 2 Mar 2016 17:23:18 +0100
Subject: [infinispan-dev] HotRod exec operation tight coupling with
JBoss Marshaller
In-Reply-To:
References: <954084CA-04F0-4389-9A09-BB89B7BE8DED@redhat.com>
<351961288.55333152.1456752949592.JavaMail.zimbra@redhat.com>
Message-ID: <73260886-1B33-41BA-B1AB-830F7DDEB4FB@redhat.com>
Just an update: the prototype I worked on to add a 'datatype' metadata parameter to the script seems to be working, so I'll be sending a PR for inclusion in 8.2.
Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat
> On 29 Feb 2016, at 19:46, Galder Zamarre?o wrote:
>
> Hi all,
>
> I had a chat earlier with Tristan and he pointed me to the metadata information that can be passed in script header.
>
> Given that so far (until compatibility is supported), Javascript client only supports String key/values, a very simple solution to this problem would be to define a new metadata parameter, e.g. data-type, which can optionally define the type of key, value, parameters and returned object. E.g. in the javascript client case, I could just say: data-type=utf8 in the header of the script, and that would provide enough hints for the server to do its job by interpreting byte arrays dealt with in exec command as UTF-8 strings.
>
> This method is much more user friendly than having user plug a marshaller since it only requires a small change in the header of the script, as opposed to having to change server side configuration. I'm working on a prototype for this with hopes to include in 8.2.
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 29 Feb 2016, at 14:35, Vittorio Rigamonti wrote:
>>
>> Hi All, Hi Galder
>>
>> JBasicMarshaller.h is not an intent to go further along the way to implement a c++ JBossMarshaller, it has the only goal to provide a working 'exec' operation.
>> The c++ exec currently works only with integer and "small" string data types, user can easily extends the marshaller to other basic types.
>>
>> My current model of the exec use case is this one (maybe it's too simplistic so correct me if I'm wrong): a "user defined consumer" consumes data produced by a "user defined producer", so maybe it could worth to evaluate a solution where the framework provides a common standard for communication (stringified json? or JSObject for in memory 100% Java) and let the user handle it's own data.
>> This is a proposal approach specific to the exec scope, I can't say if it can be extended as a general approach where marshalling is involved.
>>
>> Cheers,
>> Vittorio
>>
>>
>>
>>
>> ----- Original Message -----
>> From: "Galder Zamarre?o"
>> To: "infinispan -Dev List"
>> Sent: Monday, February 29, 2016 11:28:55 AM
>> Subject: [infinispan-dev] HotRod exec operation tight coupling with JBoss Marshaller
>>
>> Hi all,
>>
>> While implement `exec` operation for the JS client, I've encountered an issue with how the exec parameters and return types are marshalled.
>>
>> The essence of the problem is that the server marshalls these objects instead of having the client drive how these are marshalled. As a result of this, for a JS or C++ client to be able to use `exec` with default configuration, they need to understand JBoss Marshaller format, which is not good.
>>
>> I'm not sure this would have been unavoidable due to the characteristics of `exec` but I wanted to see if we can find a good way to solve or get around this issue. Long term, we need better encoding handling both for incoming and returning types, but the question is whether we can find a way to better solve this until then. Here are some options:
>>
>> - For the C++ client, Vittorio has part implemented the JBoss Marshaller format [1], but I'm kinda reluctant to go down this path since that creates a lot of work for us as the number of types that can be discovered in a JBoss Marshaller format byte array are quite considerable [2]. We're bound to miss one of those and since clients could execute any script, the chances are high IMO...
>>
>> - An alternative would be for the JS/C++ clients to only support exec when the marshaller is one that enables compatibility mode. The idea here is that for compatibility mode to work, all clients involved are going to be set up with a marshaller that can work for all of them. Working on such marshaller is time better spent than on implementing the JBoss Marshaller format. We had a separate discussion on this topic in another dev thread...
>>
>> Any other ideas someone might have?
>>
>> Cheers,
>>
>> [1] https://github.com/infinispan/cpp-client/blob/master/include/infinispan/hotrod/JBasicMarshaller.h
>> [2] https://github.com/jboss-remoting/jboss-marshalling/blob/master/river/src/main/java/org/jboss/marshalling/river/Protocol.java
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From galder at redhat.com Wed Mar 2 11:35:14 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 2 Mar 2016 17:35:14 +0100
Subject: [infinispan-dev] HotRod exec operation tight coupling with
JBoss Marshaller
In-Reply-To: <73260886-1B33-41BA-B1AB-830F7DDEB4FB@redhat.com>
References: <954084CA-04F0-4389-9A09-BB89B7BE8DED@redhat.com>
<351961288.55333152.1456752949592.JavaMail.zimbra@redhat.com>
<73260886-1B33-41BA-B1AB-830F7DDEB4FB@redhat.com>
Message-ID:
FYI: https://issues.jboss.org/browse/ISPN-6307
--
Galder Zamarre?o
Infinispan, Red Hat
> On 2 Mar 2016, at 17:23, Galder Zamarre?o wrote:
>
> Just an update: the prototype I worked on to add a 'datatype' metadata parameter to the script seems to be working, so I'll be sending a PR for inclusion in 8.2.
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 29 Feb 2016, at 19:46, Galder Zamarre?o wrote:
>>
>> Hi all,
>>
>> I had a chat earlier with Tristan and he pointed me to the metadata information that can be passed in script header.
>>
>> Given that so far (until compatibility is supported), Javascript client only supports String key/values, a very simple solution to this problem would be to define a new metadata parameter, e.g. data-type, which can optionally define the type of key, value, parameters and returned object. E.g. in the javascript client case, I could just say: data-type=utf8 in the header of the script, and that would provide enough hints for the server to do its job by interpreting byte arrays dealt with in exec command as UTF-8 strings.
>>
>> This method is much more user friendly than having user plug a marshaller since it only requires a small change in the header of the script, as opposed to having to change server side configuration. I'm working on a prototype for this with hopes to include in 8.2.
>>
>> Cheers,
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>> On 29 Feb 2016, at 14:35, Vittorio Rigamonti wrote:
>>>
>>> Hi All, Hi Galder
>>>
>>> JBasicMarshaller.h is not an intent to go further along the way to implement a c++ JBossMarshaller, it has the only goal to provide a working 'exec' operation.
>>> The c++ exec currently works only with integer and "small" string data types, user can easily extends the marshaller to other basic types.
>>>
>>> My current model of the exec use case is this one (maybe it's too simplistic so correct me if I'm wrong): a "user defined consumer" consumes data produced by a "user defined producer", so maybe it could worth to evaluate a solution where the framework provides a common standard for communication (stringified json? or JSObject for in memory 100% Java) and let the user handle it's own data.
>>> This is a proposal approach specific to the exec scope, I can't say if it can be extended as a general approach where marshalling is involved.
>>>
>>> Cheers,
>>> Vittorio
>>>
>>>
>>>
>>>
>>> ----- Original Message -----
>>> From: "Galder Zamarre?o"
>>> To: "infinispan -Dev List"
>>> Sent: Monday, February 29, 2016 11:28:55 AM
>>> Subject: [infinispan-dev] HotRod exec operation tight coupling with JBoss Marshaller
>>>
>>> Hi all,
>>>
>>> While implement `exec` operation for the JS client, I've encountered an issue with how the exec parameters and return types are marshalled.
>>>
>>> The essence of the problem is that the server marshalls these objects instead of having the client drive how these are marshalled. As a result of this, for a JS or C++ client to be able to use `exec` with default configuration, they need to understand JBoss Marshaller format, which is not good.
>>>
>>> I'm not sure this would have been unavoidable due to the characteristics of `exec` but I wanted to see if we can find a good way to solve or get around this issue. Long term, we need better encoding handling both for incoming and returning types, but the question is whether we can find a way to better solve this until then. Here are some options:
>>>
>>> - For the C++ client, Vittorio has part implemented the JBoss Marshaller format [1], but I'm kinda reluctant to go down this path since that creates a lot of work for us as the number of types that can be discovered in a JBoss Marshaller format byte array are quite considerable [2]. We're bound to miss one of those and since clients could execute any script, the chances are high IMO...
>>>
>>> - An alternative would be for the JS/C++ clients to only support exec when the marshaller is one that enables compatibility mode. The idea here is that for compatibility mode to work, all clients involved are going to be set up with a marshaller that can work for all of them. Working on such marshaller is time better spent than on implementing the JBoss Marshaller format. We had a separate discussion on this topic in another dev thread...
>>>
>>> Any other ideas someone might have?
>>>
>>> Cheers,
>>>
>>> [1] https://github.com/infinispan/cpp-client/blob/master/include/infinispan/hotrod/JBasicMarshaller.h
>>> [2] https://github.com/jboss-remoting/jboss-marshalling/blob/master/river/src/main/java/org/jboss/marshalling/river/Protocol.java
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From mudokonman at gmail.com Thu Mar 3 10:19:51 2016
From: mudokonman at gmail.com (William Burns)
Date: Thu, 03 Mar 2016 15:19:51 +0000
Subject: [infinispan-dev] Lambda Serialization
In-Reply-To:
References:
Message-ID:
I now have a working branch that is using this for the new CacheStream
interface [1].
With this it allows users to use a stream without needing any casts for any
of the intermediate or terminal operations. Note I completely revamped the
BaseStreamTest [2] So in that case every example a user can find online
can be pretty much copy pasted without additional changes, which to me is
HUGE.
Unfortunately this causes the API to bloat quite a bit and I had to add a
bunch of Serializable* classes (ex. [3]). The former bloat issue seems
acceptable to me, I had thought about making a new separate API, but it
seems like it is unneeded to me. The latter issue I had tried defining the
generics on the method itself but the compiler can't quite figure out which
method to invoke still [4].
I am still planning on adding a CacheIntStream, CacheDoubleStream and
CacheLongStream interfaces as well. Without those users would need to do
casts on the subsequent primitive stream if they used any of the
mapTo or flatMapTo methods.
Another side benefit of this refactoring is we can easily add new
operations to the stream interfaces. We could add approximation methods
maybe that return after a certain timeout, histogram specific support among
others. I am open to whatever people think they would want added here.
Unfortunately, we can't easily add in a Map.Entry stream (similar to spark
PairRDD) without redoing a bunch more of the APIs and I don't know if we
have time to support that.
Any feedback would be great, hoping to get this ironed out soon before API
freeze :)
Cheers,
- Will
[1] https://github.com/wburns/infinispan/tree/ISPN-6272
[2]
https://github.com/wburns/infinispan/commit/09734d533a445df23df94f7a053b11bc496422ec#diff-170c50a8f618af028f238109f0f1392a
[3]
https://github.com/wburns/infinispan/blob/ISPN-6272/core/src/main/java/org/infinispan/util/SerializableFunction.java
[4] https://gist.github.com/wburns/dffe4f7543f68215f74b
On Wed, Feb 17, 2016 at 8:39 AM William Burns wrote:
> Actually I have a PR that will go in before the 8.2 Final release that
> uses this [1]. Specifically check out the ClusterExecutor interface. It
> doesn't have the issues of streams with overloading existing methods,
> however it adds both overloaded variants and you can see how the tests
> invoke those.
>
> [1] https://github.com/infinispan/infinispan/pull/4008
>
>
> On Wed, Feb 17, 2016 at 3:23 AM Galder Zamarre?o
> wrote:
>
>> Hey Will,
>>
>> A very interesting discovery!
>>
>> Do you have a branch were you've tried this out? I'd like to play with it
>> to see it in action and analyse the downsides more closely.
>>
>> Cheers,
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>> > On 9 Feb 2016, at 17:36, William Burns wrote:
>> >
>> > I wanted to propose a pretty simple way of making the lambdas
>> serializable by default that I stumbled upon while working on another issue.
>> >
>> > I noticed that in the method resolution of the compiler it does some
>> nice things [1]. To be more specific when you have 2 methods with the same
>> name but vary by argument types, it will attempt to pick the most
>> "specific" one. Specific in this case you can think of if I can cast one
>> argument type to the other but it can't be cast to this type, then this one
>> is most specific.
>> >
>> > Here is an example, given the following class
>> >
>> > interface SerializableFunction extends Serializable, Function> R>
>> >
>> > The stream interface already defines:
>> >
>> > Stream map(Function super T, ? extends R> mapper);
>> >
>> > But we could add this to the CacheStream interface
>> >
>> > CacheStream map(SerializableFunction super T, ? extends R> mapper);
>> >
>> > In this case you have 2 different map methods accessible from your
>> CacheStream instance. When passing a lambda the Java compiler will
>> automatically choose the most specific one (in this case the
>> SerializableFunction one since Function can't be cast to
>> SerializableFunction). This will then make the lambda automatically
>> Serializable. In this way nothing special has to be done (ie. explicit
>> cast) to make the instance Serializable.
>> >
>> > This allows anyone using our Cache interface to immediately get lambdas
>> that are Serializable when using Streams.
>> >
>> > The main problem however would be ambiguity because the Serialization
>> would only be applied assuming you are using a defined class of CacheStream
>> etc. Also this means there are 2 methods (but that seems fine to me), so
>> it could cause a bit of confusion. The non serialization method is still
>> helpful if people want to their own Externalizer, since their
>> implementation doesn't have to implement Serializable then.
>> >
>> > What do you guys think? It seems like a decent compromise to me.
>> >
>> > - Will
>> >
>> >
>> >
>> >
>> >
>> > [1]
>> https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.12.2.5
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160303/890eccee/attachment.html
From sanne at infinispan.org Thu Mar 3 10:25:29 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 3 Mar 2016 15:25:29 +0000
Subject: [infinispan-dev] Lambda Serialization
In-Reply-To:
References:
Message-ID:
On 3 March 2016 at 15:19, William Burns wrote:
> I now have a working branch that is using this for the new CacheStream
> interface [1].
>
> With this it allows users to use a stream without needing any casts for any
> of the intermediate or terminal operations. Note I completely revamped the
> BaseStreamTest [2] So in that case every example a user can find online can
> be pretty much copy pasted without additional changes, which to me is HUGE.
I agree, it's HUGE! Great work!
>
> Unfortunately this causes the API to bloat quite a bit and I had to add a
> bunch of Serializable* classes (ex. [3]). The former bloat issue seems
> acceptable to me, I had thought about making a new separate API, but it
> seems like it is unneeded to me. The latter issue I had tried defining the
> generics on the method itself but the compiler can't quite figure out which
> method to invoke still [4].
Rather than making many things serializable, did you consider to extend
our collection of JBoss Marshallers?
Maybe support for marshalling many of JDK's stream components could
be contributed directly to the Marshaller project.
>
> I am still planning on adding a CacheIntStream, CacheDoubleStream and
> CacheLongStream interfaces as well. Without those users would need to do
> casts on the subsequent primitive stream if they used any of the
> mapTo or flatMapTo methods.
>
> Another side benefit of this refactoring is we can easily add new operations
> to the stream interfaces. We could add approximation methods maybe that
> return after a certain timeout, histogram specific support among others. I
> am open to whatever people think they would want added here. Unfortunately,
> we can't easily add in a Map.Entry stream (similar to spark PairRDD) without
> redoing a bunch more of the APIs and I don't know if we have time to support
> that.
>
> Any feedback would be great, hoping to get this ironed out soon before API
> freeze :)
>
> Cheers,
>
> - Will
>
> [1] https://github.com/wburns/infinispan/tree/ISPN-6272
> [2]
> https://github.com/wburns/infinispan/commit/09734d533a445df23df94f7a053b11bc496422ec#diff-170c50a8f618af028f238109f0f1392a
> [3]
> https://github.com/wburns/infinispan/blob/ISPN-6272/core/src/main/java/org/infinispan/util/SerializableFunction.java
> [4] https://gist.github.com/wburns/dffe4f7543f68215f74b
>
>
>
> On Wed, Feb 17, 2016 at 8:39 AM William Burns wrote:
>>
>> Actually I have a PR that will go in before the 8.2 Final release that
>> uses this [1]. Specifically check out the ClusterExecutor interface. It
>> doesn't have the issues of streams with overloading existing methods,
>> however it adds both overloaded variants and you can see how the tests
>> invoke those.
>>
>> [1] https://github.com/infinispan/infinispan/pull/4008
>>
>>
>> On Wed, Feb 17, 2016 at 3:23 AM Galder Zamarre?o
>> wrote:
>>>
>>> Hey Will,
>>>
>>> A very interesting discovery!
>>>
>>> Do you have a branch were you've tried this out? I'd like to play with it
>>> to see it in action and analyse the downsides more closely.
>>>
>>> Cheers,
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>> > On 9 Feb 2016, at 17:36, William Burns wrote:
>>> >
>>> > I wanted to propose a pretty simple way of making the lambdas
>>> > serializable by default that I stumbled upon while working on another issue.
>>> >
>>> > I noticed that in the method resolution of the compiler it does some
>>> > nice things [1]. To be more specific when you have 2 methods with the same
>>> > name but vary by argument types, it will attempt to pick the most "specific"
>>> > one. Specific in this case you can think of if I can cast one argument type
>>> > to the other but it can't be cast to this type, then this one is most
>>> > specific.
>>> >
>>> > Here is an example, given the following class
>>> >
>>> > interface SerializableFunction extends Serializable, Function>> > R>
>>> >
>>> > The stream interface already defines:
>>> >
>>> > Stream map(Function super T, ? extends R> mapper);
>>> >
>>> > But we could add this to the CacheStream interface
>>> >
>>> > CacheStream map(SerializableFunction super T, ? extends R> mapper);
>>> >
>>> > In this case you have 2 different map methods accessible from your
>>> > CacheStream instance. When passing a lambda the Java compiler will
>>> > automatically choose the most specific one (in this case the
>>> > SerializableFunction one since Function can't be cast to
>>> > SerializableFunction). This will then make the lambda automatically
>>> > Serializable. In this way nothing special has to be done (ie. explicit
>>> > cast) to make the instance Serializable.
>>> >
>>> > This allows anyone using our Cache interface to immediately get lambdas
>>> > that are Serializable when using Streams.
>>> >
>>> > The main problem however would be ambiguity because the Serialization
>>> > would only be applied assuming you are using a defined class of CacheStream
>>> > etc. Also this means there are 2 methods (but that seems fine to me), so it
>>> > could cause a bit of confusion. The non serialization method is still
>>> > helpful if people want to their own Externalizer, since their implementation
>>> > doesn't have to implement Serializable then.
>>> >
>>> > What do you guys think? It seems like a decent compromise to me.
>>> >
>>> > - Will
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > [1]
>>> > https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.12.2.5
>>> >
>>> >
>>> > _______________________________________________
>>> > infinispan-dev mailing list
>>> > infinispan-dev at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From mudokonman at gmail.com Thu Mar 3 10:40:07 2016
From: mudokonman at gmail.com (William Burns)
Date: Thu, 03 Mar 2016 15:40:07 +0000
Subject: [infinispan-dev] Lambda Serialization
In-Reply-To:
References:
Message-ID:
On Thu, Mar 3, 2016 at 10:26 AM Sanne Grinovero
wrote:
> On 3 March 2016 at 15:19, William Burns wrote:
> > I now have a working branch that is using this for the new CacheStream
> > interface [1].
> >
> > With this it allows users to use a stream without needing any casts for
> any
> > of the intermediate or terminal operations. Note I completely revamped
> the
> > BaseStreamTest [2] So in that case every example a user can find online
> can
> > be pretty much copy pasted without additional changes, which to me is
> HUGE.
>
> I agree, it's HUGE! Great work!
>
Oh I forgot to mention the 1 caveat with this approach. If the user
defines their Cache or the various collections returned from it as the base
type (ie. Map, ConcurrentMap, Set, Collection) this automatic serialization
is lost and would require manual casting again. Normal method chaining
keeps this benefit though. This seems like an acceptable and unavoidable
issue to me.
>
> >
> > Unfortunately this causes the API to bloat quite a bit and I had to add a
> > bunch of Serializable* classes (ex. [3]). The former bloat issue seems
> > acceptable to me, I had thought about making a new separate API, but it
> > seems like it is unneeded to me. The latter issue I had tried defining
> the
> > generics on the method itself but the compiler can't quite figure out
> which
> > method to invoke still [4].
>
> Rather than making many things serializable, did you consider to extend
> our collection of JBoss Marshallers?
> Maybe support for marshalling many of JDK's stream components could
> be contributed directly to the Marshaller project.
>
I personally haven't looked at this aspect. To be honest, I was leaning on
Galder a bit more here, since he is much more familiar with the Marshalling
code. If we can do this instead I think it would be even bigger.
Unfortunately I don't know how feasible it is.
>
> >
> > I am still planning on adding a CacheIntStream, CacheDoubleStream and
> > CacheLongStream interfaces as well. Without those users would need to do
> > casts on the subsequent primitive stream if they used any of the
> > mapTo or flatMapTo methods.
> >
> > Another side benefit of this refactoring is we can easily add new
> operations
> > to the stream interfaces. We could add approximation methods maybe that
> > return after a certain timeout, histogram specific support among
> others. I
> > am open to whatever people think they would want added here.
> Unfortunately,
> > we can't easily add in a Map.Entry stream (similar to spark PairRDD)
> without
> > redoing a bunch more of the APIs and I don't know if we have time to
> support
> > that.
> >
> > Any feedback would be great, hoping to get this ironed out soon before
> API
> > freeze :)
> >
> > Cheers,
> >
> > - Will
> >
> > [1] https://github.com/wburns/infinispan/tree/ISPN-6272
> > [2]
> >
> https://github.com/wburns/infinispan/commit/09734d533a445df23df94f7a053b11bc496422ec#diff-170c50a8f618af028f238109f0f1392a
> > [3]
> >
> https://github.com/wburns/infinispan/blob/ISPN-6272/core/src/main/java/org/infinispan/util/SerializableFunction.java
> > [4] https://gist.github.com/wburns/dffe4f7543f68215f74b
> >
> >
> >
> > On Wed, Feb 17, 2016 at 8:39 AM William Burns
> wrote:
> >>
> >> Actually I have a PR that will go in before the 8.2 Final release that
> >> uses this [1]. Specifically check out the ClusterExecutor interface.
> It
> >> doesn't have the issues of streams with overloading existing methods,
> >> however it adds both overloaded variants and you can see how the tests
> >> invoke those.
> >>
> >> [1] https://github.com/infinispan/infinispan/pull/4008
> >>
> >>
> >> On Wed, Feb 17, 2016 at 3:23 AM Galder Zamarre?o
> >> wrote:
> >>>
> >>> Hey Will,
> >>>
> >>> A very interesting discovery!
> >>>
> >>> Do you have a branch were you've tried this out? I'd like to play with
> it
> >>> to see it in action and analyse the downsides more closely.
> >>>
> >>> Cheers,
> >>> --
> >>> Galder Zamarre?o
> >>> Infinispan, Red Hat
> >>>
> >>> > On 9 Feb 2016, at 17:36, William Burns wrote:
> >>> >
> >>> > I wanted to propose a pretty simple way of making the lambdas
> >>> > serializable by default that I stumbled upon while working on
> another issue.
> >>> >
> >>> > I noticed that in the method resolution of the compiler it does some
> >>> > nice things [1]. To be more specific when you have 2 methods with
> the same
> >>> > name but vary by argument types, it will attempt to pick the most
> "specific"
> >>> > one. Specific in this case you can think of if I can cast one
> argument type
> >>> > to the other but it can't be cast to this type, then this one is most
> >>> > specific.
> >>> >
> >>> > Here is an example, given the following class
> >>> >
> >>> > interface SerializableFunction extends Serializable,
> Function >>> > R>
> >>> >
> >>> > The stream interface already defines:
> >>> >
> >>> > Stream map(Function super T, ? extends R> mapper);
> >>> >
> >>> > But we could add this to the CacheStream interface
> >>> >
> >>> > CacheStream map(SerializableFunction super T, ? extends R>
> mapper);
> >>> >
> >>> > In this case you have 2 different map methods accessible from your
> >>> > CacheStream instance. When passing a lambda the Java compiler will
> >>> > automatically choose the most specific one (in this case the
> >>> > SerializableFunction one since Function can't be cast to
> >>> > SerializableFunction). This will then make the lambda automatically
> >>> > Serializable. In this way nothing special has to be done (ie.
> explicit
> >>> > cast) to make the instance Serializable.
> >>> >
> >>> > This allows anyone using our Cache interface to immediately get
> lambdas
> >>> > that are Serializable when using Streams.
> >>> >
> >>> > The main problem however would be ambiguity because the Serialization
> >>> > would only be applied assuming you are using a defined class of
> CacheStream
> >>> > etc. Also this means there are 2 methods (but that seems fine to
> me), so it
> >>> > could cause a bit of confusion. The non serialization method is
> still
> >>> > helpful if people want to their own Externalizer, since their
> implementation
> >>> > doesn't have to implement Serializable then.
> >>> >
> >>> > What do you guys think? It seems like a decent compromise to me.
> >>> >
> >>> > - Will
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > [1]
> >>> >
> https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.12.2.5
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > infinispan-dev mailing list
> >>> > infinispan-dev at lists.jboss.org
> >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160303/958191c4/attachment.html
From vjuranek at redhat.com Thu Mar 3 10:54:52 2016
From: vjuranek at redhat.com (Vojtech Juranek)
Date: Thu, 03 Mar 2016 16:54:52 +0100
Subject: [infinispan-dev] Infinispan 8 presentation on DevConf.cz 2016
In-Reply-To:
References: <761888892.15555969.1454946567441.JavaMail.zimbra@redhat.com>
<2201094.oE6FtqIMlI@dhcp-10-40-5-198.brq.redhat.com>
Message-ID: <4875847.8Vk2cQq9QQ@localhost.localdomain>
On Wednesday 02 March 2016 14:23:36 Gustavo Fernandes wrote:
> Nice Spark Streaming + Infinispan demo! This would make a great quickstart
> project!
thanks. I'll try to improve it little bit and convert to a quickstart.
Vojta
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part.
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160303/62155f25/attachment.bin
From galder at redhat.com Fri Mar 4 08:44:35 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 4 Mar 2016 14:44:35 +0100
Subject: [infinispan-dev] Shade source jar issue?
Message-ID:
Hi guys,
I've been trying to test out a embedded shaded jar snapshot but I'm having issues with discrepancies between jar file and source file.
It appears that the source file is not updated, e.g.
If I take the jar, expand it and grep, I see this:
$ javap org.infinispan.CacheStream | grep collect
public abstract R1 collect(java.util.stream.Collector super R, A, R1>);
public abstract R1 collect(org.infinispan.util.SerializableSupplier, org.infinispan.util.SerializableBiConsumer, org.infinispan.util.SerializableBiConsumer);
When I look at the source jar, I see:
$ grep collect ./org/infinispan/CacheStream.java
...
R1 collect(Collector super R, A, R1> collector);
Are you aware of any issues with shade source jars?
Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat
From galder at redhat.com Fri Mar 4 08:52:06 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 4 Mar 2016 14:52:06 +0100
Subject: [infinispan-dev] Shade source jar issue?
In-Reply-To:
References:
Message-ID:
Actually, I have same issue building core/ with:
$ mvn -DskipTests=true install
Looking into it...
Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat
> On 4 Mar 2016, at 14:44, Galder Zamarre?o wrote:
>
> Hi guys,
>
> I've been trying to test out a embedded shaded jar snapshot but I'm having issues with discrepancies between jar file and source file.
>
> It appears that the source file is not updated, e.g.
>
> If I take the jar, expand it and grep, I see this:
>
> $ javap org.infinispan.CacheStream | grep collect
> public abstract R1 collect(java.util.stream.Collector super R, A, R1>);
> public abstract R1 collect(org.infinispan.util.SerializableSupplier, org.infinispan.util.SerializableBiConsumer, org.infinispan.util.SerializableBiConsumer);
>
> When I look at the source jar, I see:
>
> $ grep collect ./org/infinispan/CacheStream.java
> ...
> R1 collect(Collector super R, A, R1> collector);
>
> Are you aware of any issues with shade source jars?
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From mudokonman at gmail.com Fri Mar 4 08:52:56 2016
From: mudokonman at gmail.com (William Burns)
Date: Fri, 04 Mar 2016 13:52:56 +0000
Subject: [infinispan-dev] Lambda Serialization
In-Reply-To:
References:
Message-ID:
On Thu, Mar 3, 2016 at 10:40 AM William Burns wrote:
> On Thu, Mar 3, 2016 at 10:26 AM Sanne Grinovero
> wrote:
>
>> On 3 March 2016 at 15:19, William Burns wrote:
>> > I now have a working branch that is using this for the new CacheStream
>> > interface [1].
>> >
>> > With this it allows users to use a stream without needing any casts for
>> any
>> > of the intermediate or terminal operations. Note I completely revamped
>> the
>> > BaseStreamTest [2] So in that case every example a user can find
>> online can
>> > be pretty much copy pasted without additional changes, which to me is
>> HUGE.
>>
>> I agree, it's HUGE! Great work!
>>
>
> Oh I forgot to mention the 1 caveat with this approach. If the user
> defines their Cache or the various collections returned from it as the base
> type (ie. Map, ConcurrentMap, Set, Collection) this automatic serialization
> is lost and would require manual casting again. Normal method chaining
> keeps this benefit though. This seems like an acceptable and unavoidable
> issue to me.
>
Sorry also these changes still require the use of the CacheCollectors when
using [1] like we have had in the past it. It just removes the need for all
of the various casting (ie. (Serializable & Function super R>) for the
other methods.
[1]
https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#collect-java.util.stream.Collector-
>
>
>>
>> >
>> > Unfortunately this causes the API to bloat quite a bit and I had to add
>> a
>> > bunch of Serializable* classes (ex. [3]). The former bloat issue seems
>> > acceptable to me, I had thought about making a new separate API, but it
>> > seems like it is unneeded to me. The latter issue I had tried defining
>> the
>> > generics on the method itself but the compiler can't quite figure out
>> which
>> > method to invoke still [4].
>>
>> Rather than making many things serializable, did you consider to extend
>> our collection of JBoss Marshallers?
>> Maybe support for marshalling many of JDK's stream components could
>> be contributed directly to the Marshaller project.
>>
>
> I personally haven't looked at this aspect. To be honest, I was leaning
> on Galder a bit more here, since he is much more familiar with the
> Marshalling code. If we can do this instead I think it would be even
> bigger. Unfortunately I don't know how feasible it is.
>
>
>>
>> >
>> > I am still planning on adding a CacheIntStream, CacheDoubleStream and
>> > CacheLongStream interfaces as well. Without those users would need to
>> do
>> > casts on the subsequent primitive stream if they used any of the
>> > mapTo or flatMapTo methods.
>> >
>> > Another side benefit of this refactoring is we can easily add new
>> operations
>> > to the stream interfaces. We could add approximation methods maybe that
>> > return after a certain timeout, histogram specific support among
>> others. I
>> > am open to whatever people think they would want added here.
>> Unfortunately,
>> > we can't easily add in a Map.Entry stream (similar to spark PairRDD)
>> without
>> > redoing a bunch more of the APIs and I don't know if we have time to
>> support
>> > that.
>> >
>> > Any feedback would be great, hoping to get this ironed out soon before
>> API
>> > freeze :)
>> >
>> > Cheers,
>> >
>> > - Will
>> >
>> > [1] https://github.com/wburns/infinispan/tree/ISPN-6272
>> > [2]
>> >
>> https://github.com/wburns/infinispan/commit/09734d533a445df23df94f7a053b11bc496422ec#diff-170c50a8f618af028f238109f0f1392a
>> > [3]
>> >
>> https://github.com/wburns/infinispan/blob/ISPN-6272/core/src/main/java/org/infinispan/util/SerializableFunction.java
>> > [4] https://gist.github.com/wburns/dffe4f7543f68215f74b
>> >
>> >
>> >
>> > On Wed, Feb 17, 2016 at 8:39 AM William Burns
>> wrote:
>> >>
>> >> Actually I have a PR that will go in before the 8.2 Final release that
>> >> uses this [1]. Specifically check out the ClusterExecutor interface.
>> It
>> >> doesn't have the issues of streams with overloading existing methods,
>> >> however it adds both overloaded variants and you can see how the tests
>> >> invoke those.
>> >>
>> >> [1] https://github.com/infinispan/infinispan/pull/4008
>> >>
>> >>
>> >> On Wed, Feb 17, 2016 at 3:23 AM Galder Zamarre?o
>> >> wrote:
>> >>>
>> >>> Hey Will,
>> >>>
>> >>> A very interesting discovery!
>> >>>
>> >>> Do you have a branch were you've tried this out? I'd like to play
>> with it
>> >>> to see it in action and analyse the downsides more closely.
>> >>>
>> >>> Cheers,
>> >>> --
>> >>> Galder Zamarre?o
>> >>> Infinispan, Red Hat
>> >>>
>> >>> > On 9 Feb 2016, at 17:36, William Burns
>> wrote:
>> >>> >
>> >>> > I wanted to propose a pretty simple way of making the lambdas
>> >>> > serializable by default that I stumbled upon while working on
>> another issue.
>> >>> >
>> >>> > I noticed that in the method resolution of the compiler it does some
>> >>> > nice things [1]. To be more specific when you have 2 methods with
>> the same
>> >>> > name but vary by argument types, it will attempt to pick the most
>> "specific"
>> >>> > one. Specific in this case you can think of if I can cast one
>> argument type
>> >>> > to the other but it can't be cast to this type, then this one is
>> most
>> >>> > specific.
>> >>> >
>> >>> > Here is an example, given the following class
>> >>> >
>> >>> > interface SerializableFunction extends Serializable,
>> Function> >>> > R>
>> >>> >
>> >>> > The stream interface already defines:
>> >>> >
>> >>> > Stream map(Function super T, ? extends R> mapper);
>> >>> >
>> >>> > But we could add this to the CacheStream interface
>> >>> >
>> >>> > CacheStream map(SerializableFunction super T, ? extends R>
>> mapper);
>> >>> >
>> >>> > In this case you have 2 different map methods accessible from your
>> >>> > CacheStream instance. When passing a lambda the Java compiler will
>> >>> > automatically choose the most specific one (in this case the
>> >>> > SerializableFunction one since Function can't be cast to
>> >>> > SerializableFunction). This will then make the lambda automatically
>> >>> > Serializable. In this way nothing special has to be done (ie.
>> explicit
>> >>> > cast) to make the instance Serializable.
>> >>> >
>> >>> > This allows anyone using our Cache interface to immediately get
>> lambdas
>> >>> > that are Serializable when using Streams.
>> >>> >
>> >>> > The main problem however would be ambiguity because the
>> Serialization
>> >>> > would only be applied assuming you are using a defined class of
>> CacheStream
>> >>> > etc. Also this means there are 2 methods (but that seems fine to
>> me), so it
>> >>> > could cause a bit of confusion. The non serialization method is
>> still
>> >>> > helpful if people want to their own Externalizer, since their
>> implementation
>> >>> > doesn't have to implement Serializable then.
>> >>> >
>> >>> > What do you guys think? It seems like a decent compromise to me.
>> >>> >
>> >>> > - Will
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > [1]
>> >>> >
>> https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.12.2.5
>> >>> >
>> >>> >
>> >>> > _______________________________________________
>> >>> > infinispan-dev mailing list
>> >>> > infinispan-dev at lists.jboss.org
>> >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> infinispan-dev mailing list
>> >>> infinispan-dev at lists.jboss.org
>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160304/33e5d4a7/attachment.html
From galder at redhat.com Fri Mar 4 09:07:20 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 4 Mar 2016 15:07:20 +0100
Subject: [infinispan-dev] Shade source jar issue?
In-Reply-To:
References:
Message-ID: <89425F26-3028-4E46-AA46-E8F76ADAB03E@redhat.com>
Needs -Pextras :)
--
Galder Zamarre?o
Infinispan, Red Hat
> On 4 Mar 2016, at 14:52, Galder Zamarre?o wrote:
>
> Actually, I have same issue building core/ with:
>
> $ mvn -DskipTests=true install
>
> Looking into it...
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 4 Mar 2016, at 14:44, Galder Zamarre?o wrote:
>>
>> Hi guys,
>>
>> I've been trying to test out a embedded shaded jar snapshot but I'm having issues with discrepancies between jar file and source file.
>>
>> It appears that the source file is not updated, e.g.
>>
>> If I take the jar, expand it and grep, I see this:
>>
>> $ javap org.infinispan.CacheStream | grep collect
>> public abstract R1 collect(java.util.stream.Collector super R, A, R1>);
>> public abstract R1 collect(org.infinispan.util.SerializableSupplier, org.infinispan.util.SerializableBiConsumer, org.infinispan.util.SerializableBiConsumer);
>>
>> When I look at the source jar, I see:
>>
>> $ grep collect ./org/infinispan/CacheStream.java
>> ...
>> R1 collect(Collector super R, A, R1> collector);
>>
>> Are you aware of any issues with shade source jars?
>>
>> Cheers,
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From galder at redhat.com Fri Mar 4 10:07:39 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 4 Mar 2016 16:07:39 +0100
Subject: [infinispan-dev] Lambda Serialization
In-Reply-To:
References:
Message-ID:
--
Galder Zamarre?o
Infinispan, Red Hat
> On 3 Mar 2016, at 16:40, William Burns wrote:
>
>
>
> On Thu, Mar 3, 2016 at 10:26 AM Sanne Grinovero wrote:
> On 3 March 2016 at 15:19, William Burns wrote:
> > I now have a working branch that is using this for the new CacheStream
> > interface [1].
> >
> > With this it allows users to use a stream without needing any casts for any
> > of the intermediate or terminal operations. Note I completely revamped the
> > BaseStreamTest [2] So in that case every example a user can find online can
> > be pretty much copy pasted without additional changes, which to me is HUGE.
>
> I agree, it's HUGE! Great work!
^ Awesome stuff! I've just tried it out and it's a huge improvement particularly for out-of-the-box experience and live demos.
>
> Oh I forgot to mention the 1 caveat with this approach. If the user defines their Cache or the various collections returned from it as the base type (ie. Map, ConcurrentMap, Set, Collection) this automatic serialization is lost and would require manual casting again. Normal method chaining keeps this benefit though. This seems like an acceptable and unavoidable issue to me.
>
>
> >
> > Unfortunately this causes the API to bloat quite a bit and I had to add a
> > bunch of Serializable* classes (ex. [3]). The former bloat issue seems
> > acceptable to me, I had thought about making a new separate API, but it
> > seems like it is unneeded to me. The latter issue I had tried defining the
> > generics on the method itself but the compiler can't quite figure out which
> > method to invoke still [4].
>
> Rather than making many things serializable, did you consider to extend
> our collection of JBoss Marshallers?
> Maybe support for marshalling many of JDK's stream components could
> be contributed directly to the Marshaller project.
>
> I personally haven't looked at this aspect. To be honest, I was leaning on Galder a bit more here, since he is much more familiar with the Marshalling code. If we can do this instead I think it would be even bigger. Unfortunately I don't know how feasible it is.
I'm not sure this is that important. With Will's changes we're trying to improve the out-of-the-box experience by being able to marshall lambdas as they are.
The user can still provide Externalizers for them to make them ultra fast. Also, an Externalizer has no default read/write object implementation, so not sure how we could implement what you suggest in a generic way.
>
>
> >
> > I am still planning on adding a CacheIntStream, CacheDoubleStream and
> > CacheLongStream interfaces as well. Without those users would need to do
> > casts on the subsequent primitive stream if they used any of the
> > mapTo or flatMapTo methods.
> >
> > Another side benefit of this refactoring is we can easily add new operations
> > to the stream interfaces. We could add approximation methods maybe that
> > return after a certain timeout, histogram specific support among others. I
> > am open to whatever people think they would want added here. Unfortunately,
> > we can't easily add in a Map.Entry stream (similar to spark PairRDD) without
> > redoing a bunch more of the APIs and I don't know if we have time to support
> > that.
> >
> > Any feedback would be great, hoping to get this ironed out soon before API
> > freeze :)
> >
> > Cheers,
> >
> > - Will
> >
> > [1] https://github.com/wburns/infinispan/tree/ISPN-6272
> > [2]
> > https://github.com/wburns/infinispan/commit/09734d533a445df23df94f7a053b11bc496422ec#diff-170c50a8f618af028f238109f0f1392a
> > [3]
> > https://github.com/wburns/infinispan/blob/ISPN-6272/core/src/main/java/org/infinispan/util/SerializableFunction.java
> > [4] https://gist.github.com/wburns/dffe4f7543f68215f74b
> >
> >
> >
> > On Wed, Feb 17, 2016 at 8:39 AM William Burns wrote:
> >>
> >> Actually I have a PR that will go in before the 8.2 Final release that
> >> uses this [1]. Specifically check out the ClusterExecutor interface. It
> >> doesn't have the issues of streams with overloading existing methods,
> >> however it adds both overloaded variants and you can see how the tests
> >> invoke those.
> >>
> >> [1] https://github.com/infinispan/infinispan/pull/4008
> >>
> >>
> >> On Wed, Feb 17, 2016 at 3:23 AM Galder Zamarre?o
> >> wrote:
> >>>
> >>> Hey Will,
> >>>
> >>> A very interesting discovery!
> >>>
> >>> Do you have a branch were you've tried this out? I'd like to play with it
> >>> to see it in action and analyse the downsides more closely.
> >>>
> >>> Cheers,
> >>> --
> >>> Galder Zamarre?o
> >>> Infinispan, Red Hat
> >>>
> >>> > On 9 Feb 2016, at 17:36, William Burns wrote:
> >>> >
> >>> > I wanted to propose a pretty simple way of making the lambdas
> >>> > serializable by default that I stumbled upon while working on another issue.
> >>> >
> >>> > I noticed that in the method resolution of the compiler it does some
> >>> > nice things [1]. To be more specific when you have 2 methods with the same
> >>> > name but vary by argument types, it will attempt to pick the most "specific"
> >>> > one. Specific in this case you can think of if I can cast one argument type
> >>> > to the other but it can't be cast to this type, then this one is most
> >>> > specific.
> >>> >
> >>> > Here is an example, given the following class
> >>> >
> >>> > interface SerializableFunction extends Serializable, Function >>> > R>
> >>> >
> >>> > The stream interface already defines:
> >>> >
> >>> > Stream map(Function super T, ? extends R> mapper);
> >>> >
> >>> > But we could add this to the CacheStream interface
> >>> >
> >>> > CacheStream map(SerializableFunction super T, ? extends R> mapper);
> >>> >
> >>> > In this case you have 2 different map methods accessible from your
> >>> > CacheStream instance. When passing a lambda the Java compiler will
> >>> > automatically choose the most specific one (in this case the
> >>> > SerializableFunction one since Function can't be cast to
> >>> > SerializableFunction). This will then make the lambda automatically
> >>> > Serializable. In this way nothing special has to be done (ie. explicit
> >>> > cast) to make the instance Serializable.
> >>> >
> >>> > This allows anyone using our Cache interface to immediately get lambdas
> >>> > that are Serializable when using Streams.
> >>> >
> >>> > The main problem however would be ambiguity because the Serialization
> >>> > would only be applied assuming you are using a defined class of CacheStream
> >>> > etc. Also this means there are 2 methods (but that seems fine to me), so it
> >>> > could cause a bit of confusion. The non serialization method is still
> >>> > helpful if people want to their own Externalizer, since their implementation
> >>> > doesn't have to implement Serializable then.
> >>> >
> >>> > What do you guys think? It seems like a decent compromise to me.
> >>> >
> >>> > - Will
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > [1]
> >>> > https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.12.2.5
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > infinispan-dev mailing list
> >>> > infinispan-dev at lists.jboss.org
> >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From slaskawi at redhat.com Mon Mar 7 09:31:34 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Mon, 7 Mar 2016 15:31:34 +0100
Subject: [infinispan-dev] Uber jars - how do we want to use them
Message-ID:
Hey!
Uber jars have been around for quite a while but some old problems are
still biting our ankles. The biggest problem is JBoss Logging... But before
diving into the technical discussion, I would like to clarify how do we
treat Uber Jars and how other (let's call them "extra") modules should
integrate with them...
How Uber Jars look like:
- All Uber Jar dependencies are declared as:
- provided - those are APIs which should be available on your
container
- optional - all other dependencies
- compile - JTA (I'm not sure why, but probably because this is a
required API). I think it should be changed to provided at some point.
- Uber Jars have (almost) no dependencies (apart from JTA)
- Uber Jars contain everything to get a standard application running
with ISPN in a single jar
Now the biggest question is how to integrate other extra modules with them
- for example Spring or Cache Stores? Here are the options:
- Not integrate at all... users should use standard (small) jars in such
use cases
- This one has a big impact on our users, so I would assume no
- All extra modules should shade everything they need and provide their
own Uber Jars (like Spring Uber Jar = ISPN Core + Spring)
- This one might be really hard to maintain, so I would also vote for
no.
- All extra modules should integrate with small jars and Uber Jars (but
not both at the same time). An example application might want to add
Spring-embedded and Infinispan-embedded to it's classpath and it's ready to
go. Alternatively it could add Spring-embedded + infinispan-core + whatever
is needed. Adding both infinispan-embedded and infinispan-core is
prohibited.
- Easy for the users, a bit harder for us. My vote goes here.
Could you please share some thoughts whether this makes sense or not? Once
we achieve common understanding - I'll propose some technical solutions.
Thanks
Sebastian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160307/24081d2c/attachment.html
From ttarrant at redhat.com Mon Mar 7 15:01:36 2016
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 7 Mar 2016 21:01:36 +0100
Subject: [infinispan-dev] Infinispan Weekly IRC meeting logs 2016-03-07
Message-ID: <56DDDE20.1020505@redhat.com>
Hi all,
get the meeting logs from
http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2016/infinispan.2016-03-07-15.01.log.html
Cheers
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
From pedro at infinispan.org Tue Mar 8 18:07:46 2016
From: pedro at infinispan.org (Pedro Ruivo)
Date: Tue, 8 Mar 2016 23:07:46 +0000
Subject: [infinispan-dev] Infinispan 8.2.0.Final is out!
Message-ID: <56DF5B42.5040503@infinispan.org>
Dear community,
The Infinispan team is proud to announce the release of Infinispan
8.2.0.Final.
Checkout what is new in this version here:
http://blog.infinispan.org/2016/03/infinispan-820final-is-out.html
Cheers,
The Infinispan team.
From sanne at infinispan.org Fri Mar 11 07:07:55 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 11 Mar 2016 12:07:55 +0000
Subject: [infinispan-dev] Removal of auto-detection of indexed entities
Message-ID:
When booting Infinispan Server v. 8.2.0.Final I see this message being logged:
ISPN000403: No indexable classes were defined for this indexed cache;
switching to autodetection (support for autodetection will be removed
in Infinispan 9.0).
Does it mean we can finally remove the infamous, super-complex to
maintain and generally hated "feature" in Hibernate Search to
dynamically add new entities on the fly?
From anistor at redhat.com Fri Mar 11 08:00:05 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Fri, 11 Mar 2016 15:00:05 +0200
Subject: [infinispan-dev] Removal of auto-detection of indexed entities
In-Reply-To:
References:
Message-ID: <56E2C155.2000402@redhat.com>
Hi Sanne,
yes, but there is a small unsolved issue ATM and the implementation
needs more polishing before we get to Ispn 9.0; infinispan and probably
search too will need a bit more tweaking.
Current approach is that the search factory is first created with 0
classes and then after initialization of the cache and the index manager
has completed we reconfigure the search factory again, exactly once, and
all known indexed classes are added at once. We cannot add the indexed
classes at the initial creation time because that would cause infinispan
directory to try to create the needed caches (for locking, metadata,...)
while the user's cache has not even finished initializing. These leads
to deadlock in our component registry. This approach if far more
efficient and less error prone than the previous but still not ideal.
Adrian
On 03/11/2016 02:07 PM, Sanne Grinovero wrote:
> When booting Infinispan Server v. 8.2.0.Final I see this message being logged:
>
> ISPN000403: No indexable classes were defined for this indexed cache;
> switching to autodetection (support for autodetection will be removed
> in Infinispan 9.0).
>
> Does it mean we can finally remove the infamous, super-complex to
> maintain and generally hated "feature" in Hibernate Search to
> dynamically add new entities on the fly?
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From sanne at infinispan.org Fri Mar 11 08:13:48 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 11 Mar 2016 13:13:48 +0000
Subject: [infinispan-dev] Removal of auto-detection of indexed entities
In-Reply-To: <56E2C155.2000402@redhat.com>
References:
<56E2C155.2000402@redhat.com>
Message-ID:
On 11 March 2016 at 13:00, Adrian Nistor wrote:
> Hi Sanne,
>
> yes, but there is a small unsolved issue ATM and the implementation
> needs more polishing before we get to Ispn 9.0; infinispan and probably
> search too will need a bit more tweaking.
>
> Current approach is that the search factory is first created with 0
> classes and then after initialization of the cache and the index manager
> has completed we reconfigure the search factory again, exactly once, and
> all known indexed classes are added at once. We cannot add the indexed
> classes at the initial creation time because that would cause infinispan
> directory to try to create the needed caches (for locking, metadata,...)
> while the user's cache has not even finished initializing. These leads
> to deadlock in our component registry. This approach if far more
> efficient and less error prone than the previous but still not ideal.
Sounds like this would be solved by initializing the SearchIntegrator
lazily (only once) ?
Let us know what the Search team could do, it would be very nice for
us to be able to
drop this requirement as it makes any other change we're working on more complex
to implement.
Thanks!
Sanne
>
> Adrian
>
> On 03/11/2016 02:07 PM, Sanne Grinovero wrote:
>> When booting Infinispan Server v. 8.2.0.Final I see this message being logged:
>>
>> ISPN000403: No indexable classes were defined for this indexed cache;
>> switching to autodetection (support for autodetection will be removed
>> in Infinispan 9.0).
>>
>> Does it mean we can finally remove the infamous, super-complex to
>> maintain and generally hated "feature" in Hibernate Search to
>> dynamically add new entities on the fly?
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From gunnar at hibernate.org Fri Mar 11 08:42:27 2016
From: gunnar at hibernate.org (Gunnar Morling)
Date: Fri, 11 Mar 2016 14:42:27 +0100
Subject: [infinispan-dev] Removal of auto-detection of indexed entities
In-Reply-To:
References:
<56E2C155.2000402@redhat.com>
Message-ID:
What Sanne says :)
This is causing great pain in HSEARCH, so if we managed to get rid of
this, that'd be just awesome!
2016-03-11 14:13 GMT+01:00 Sanne Grinovero :
> On 11 March 2016 at 13:00, Adrian Nistor wrote:
>> Hi Sanne,
>>
>> yes, but there is a small unsolved issue ATM and the implementation
>> needs more polishing before we get to Ispn 9.0; infinispan and probably
>> search too will need a bit more tweaking.
>>
>> Current approach is that the search factory is first created with 0
>> classes and then after initialization of the cache and the index manager
>> has completed we reconfigure the search factory again, exactly once, and
>> all known indexed classes are added at once. We cannot add the indexed
>> classes at the initial creation time because that would cause infinispan
>> directory to try to create the needed caches (for locking, metadata,...)
>> while the user's cache has not even finished initializing. These leads
>> to deadlock in our component registry. This approach if far more
>> efficient and less error prone than the previous but still not ideal.
>
> Sounds like this would be solved by initializing the SearchIntegrator
> lazily (only once) ?
>
> Let us know what the Search team could do, it would be very nice for
> us to be able to
> drop this requirement as it makes any other change we're working on more complex
> to implement.
>
> Thanks!
> Sanne
>
>>
>> Adrian
>>
>> On 03/11/2016 02:07 PM, Sanne Grinovero wrote:
>>> When booting Infinispan Server v. 8.2.0.Final I see this message being logged:
>>>
>>> ISPN000403: No indexable classes were defined for this indexed cache;
>>> switching to autodetection (support for autodetection will be removed
>>> in Infinispan 9.0).
>>>
>>> Does it mean we can finally remove the infamous, super-complex to
>>> maintain and generally hated "feature" in Hibernate Search to
>>> dynamically add new entities on the fly?
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From anistor at redhat.com Fri Mar 11 09:02:45 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Fri, 11 Mar 2016 16:02:45 +0200
Subject: [infinispan-dev] Removal of auto-detection of indexed entities
In-Reply-To:
References:
<56E2C155.2000402@redhat.com>
Message-ID: <56E2D005.9010408@redhat.com>
The laziness could be tried first in Infinispan, then if it does not
work we might need some help from hibernate-search. That might save
hibernate-search from adding another hack for infinispan.
Thanks!
On 03/11/2016 03:13 PM, Sanne Grinovero wrote:
> On 11 March 2016 at 13:00, Adrian Nistor wrote:
>> Hi Sanne,
>>
>> yes, but there is a small unsolved issue ATM and the implementation
>> needs more polishing before we get to Ispn 9.0; infinispan and probably
>> search too will need a bit more tweaking.
>>
>> Current approach is that the search factory is first created with 0
>> classes and then after initialization of the cache and the index manager
>> has completed we reconfigure the search factory again, exactly once, and
>> all known indexed classes are added at once. We cannot add the indexed
>> classes at the initial creation time because that would cause infinispan
>> directory to try to create the needed caches (for locking, metadata,...)
>> while the user's cache has not even finished initializing. These leads
>> to deadlock in our component registry. This approach if far more
>> efficient and less error prone than the previous but still not ideal.
> Sounds like this would be solved by initializing the SearchIntegrator
> lazily (only once) ?
>
> Let us know what the Search team could do, it would be very nice for
> us to be able to
> drop this requirement as it makes any other change we're working on more complex
> to implement.
>
> Thanks!
> Sanne
>
>> Adrian
>>
>> On 03/11/2016 02:07 PM, Sanne Grinovero wrote:
>>> When booting Infinispan Server v. 8.2.0.Final I see this message being logged:
>>>
>>> ISPN000403: No indexable classes were defined for this indexed cache;
>>> switching to autodetection (support for autodetection will be removed
>>> in Infinispan 9.0).
>>>
>>> Does it mean we can finally remove the infamous, super-complex to
>>> maintain and generally hated "feature" in Hibernate Search to
>>> dynamically add new entities on the fly?
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From anistor at redhat.com Fri Mar 11 09:09:27 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Fri, 11 Mar 2016 16:09:27 +0200
Subject: [infinispan-dev] Removal of auto-detection of indexed entities
In-Reply-To: <56E2D005.9010408@redhat.com>
References:
<56E2C155.2000402@redhat.com>
<56E2D005.9010408@redhat.com>
Message-ID: <56E2D197.30305@redhat.com>
BTW, these changes were added a bit late in the Infinispan 8.2
development so I had to keep them at the minimum, otherwise I would have
added lazy search factory initialization already.
But since we already have some strange random mystery deadlocks at
server start (which do not seem related, until I'm proven wrong :)) I
was reluctant to do it.
On 03/11/2016 04:02 PM, Adrian Nistor wrote:
> The laziness could be tried first in Infinispan, then if it does not
> work we might need some help from hibernate-search. That might save
> hibernate-search from adding another hack for infinispan.
>
> Thanks!
>
> On 03/11/2016 03:13 PM, Sanne Grinovero wrote:
>> On 11 March 2016 at 13:00, Adrian Nistor wrote:
>>> Hi Sanne,
>>>
>>> yes, but there is a small unsolved issue ATM and the implementation
>>> needs more polishing before we get to Ispn 9.0; infinispan and probably
>>> search too will need a bit more tweaking.
>>>
>>> Current approach is that the search factory is first created with 0
>>> classes and then after initialization of the cache and the index manager
>>> has completed we reconfigure the search factory again, exactly once, and
>>> all known indexed classes are added at once. We cannot add the indexed
>>> classes at the initial creation time because that would cause infinispan
>>> directory to try to create the needed caches (for locking, metadata,...)
>>> while the user's cache has not even finished initializing. These leads
>>> to deadlock in our component registry. This approach if far more
>>> efficient and less error prone than the previous but still not ideal.
>> Sounds like this would be solved by initializing the SearchIntegrator
>> lazily (only once) ?
>>
>> Let us know what the Search team could do, it would be very nice for
>> us to be able to
>> drop this requirement as it makes any other change we're working on more complex
>> to implement.
>>
>> Thanks!
>> Sanne
>>
>>> Adrian
>>>
>>> On 03/11/2016 02:07 PM, Sanne Grinovero wrote:
>>>> When booting Infinispan Server v. 8.2.0.Final I see this message being logged:
>>>>
>>>> ISPN000403: No indexable classes were defined for this indexed cache;
>>>> switching to autodetection (support for autodetection will be removed
>>>> in Infinispan 9.0).
>>>>
>>>> Does it mean we can finally remove the infamous, super-complex to
>>>> maintain and generally hated "feature" in Hibernate Search to
>>>> dynamically add new entities on the fly?
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From galder at redhat.com Fri Mar 11 12:20:22 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 11 Mar 2016 19:20:22 +0200
Subject: [infinispan-dev] Uber jars - how do we want to use them
In-Reply-To:
References:
Message-ID:
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160311/fd1542e1/attachment.html
From ttarrant at redhat.com Mon Mar 14 03:42:53 2016
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 14 Mar 2016 08:42:53 +0100
Subject: [infinispan-dev] Uber jars - how do we want to use them
In-Reply-To:
References:
Message-ID: <56E66B7D.70901@redhat.com>
On 11/03/2016 18:20, Galder Zamarre?o wrote:
> Are uber jars really that useful? From my own experience they often get
The number of users who don't use a dependency management system (Maven,
Ivy, Gradle) is quite a lot higher than you'd expect.
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
From slaskawi at redhat.com Mon Mar 14 06:21:25 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Mon, 14 Mar 2016 11:21:25 +0100
Subject: [infinispan-dev] Uber jars - how do we want to use them
In-Reply-To: <56E66B7D.70901@redhat.com>
References:
<56E66B7D.70901@redhat.com>
Message-ID:
I took a look at Nexus download statistics and Infinispan Uberjars are
about 7% of our downloads (of course this calculation has been based on our
JBoss Nexus instance and we have no data from other mirrors).
So, once we are clear how Uber Jars should work... let's take a look at one
of the problems:
- Many of our modules use JBoss Logging
- Uber jars relocate dependencies (like the one above) into infinispan
package. The result for JBoss Logging would be:
infinispan.org.jboss.logging*
- Most of our modules are compiled with small jars (like Spring
integration), so the compile dependency for Spring integration is
org.jboss.logging (not infinispan.org.jboss.logging)
- If someone wants to use Spring with Uber Jars (which should be
absolutely fine), he gets a clash... Spring module can't find JBoss Logging
There are several options to solve that:
1. Stop relocating dependencies
- Pro: everything should work without problems
- Con: our dependencies will clash with client's dependencies
2. Wrap 3rd party dependency with our own wrappers. In case of JBoss
Logging - develop our own facade and put it in common.
- Pro: 3rd party dependencies will not leak between our modules
- Con: A lot of effort and rather small gain
3. Develop 3rd party dependencies as Uber Jars. In case of Spring - we
will have a Spring Uber Jar. I think Dan put this idea on the table.
- Pro: Lots of Uber Jars for everyone
- Con: How to use small jars in this scenario?
4. Mark all module dependencies as provided and let users add small/uber
jars themselves.
- Pro: It should give us the best results with the smallest amount of
time
- Cons: Not all scenarios will be solved. We will also need to stop
relocating some dependencies (like logging facades which are used almost
everywhere)
Having in mind our download statistics I would go for #4 (even though it
doesn't solve the problem entirely). However #2 seems like a better fit for
long term maintenance and we should proceed with this direction if number
of Uber Jars users will grow (I can set a reminder in a 3-4 months time).
Does it sound reasonable?
Thanks
Sebastian
On Mon, Mar 14, 2016 at 8:42 AM, Tristan Tarrant
wrote:
>
>
> On 11/03/2016 18:20, Galder Zamarre?o wrote:
> > Are uber jars really that useful? From my own experience they often get
>
> The number of users who don't use a dependency management system (Maven,
> Ivy, Gradle) is quite a lot higher than you'd expect.
>
> Tristan
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160314/7f0010b9/attachment.html
From galder at redhat.com Mon Mar 14 06:47:59 2016
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Mon, 14 Mar 2016 11:47:59 +0100
Subject: [infinispan-dev] Uber jars - how do we want to use them
In-Reply-To: <56E66B7D.70901@redhat.com>
References:
<56E66B7D.70901@redhat.com>
Message-ID: <9EC129F2-B9C8-44BB-9EEE-F6899F0AC49F@redhat.com>
--
Galder Zamarre?o
Infinispan, Red Hat
> On 14 Mar 2016, at 08:42, Tristan Tarrant wrote:
>
>
>
> On 11/03/2016 18:20, Galder Zamarre?o wrote:
>> Are uber jars really that useful? From my own experience they often get
>
> The number of users who don't use a dependency management system (Maven,
> Ivy, Gradle) is quite a lot higher than you'd expect.
Good point, so the uber jars are mostly for those use cases then?
That could be the recommendation:
- Using dependency management? Use normal jars, otherwise use uber jars
Cheers,
>
> Tristan
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From sanne at infinispan.org Mon Mar 14 06:50:00 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Mon, 14 Mar 2016 10:50:00 +0000
Subject: [infinispan-dev] Uber jars - how do we want to use them
In-Reply-To:
References:
<56E66B7D.70901@redhat.com>
Message-ID:
Hi,
uber jars were introduced as an answer to complaints such as "there
are too many jars" but I still think this was the wrong answer.. too
many issues so please stop this: it's not helping usability to
understand which jars are needed, and it makes things worse with
runtime errors not matching source code.
It was my impression that uber jars were a temporary solution to
improve things, while the real improvements - to actually need less at
runtime, or provide better errors pointing out what's needed - would
take more time.
So rather than stubbornly spend time maintaining this approach which
is so obviously broken, I'd rather see time spent in pursuing the
better improvements so that we can finally drop this.
Sebastian: I din't understand the idea behind proposal 2#: what does
it meant to provide our own facade to JBoss Logging? One will still
need to have JBoss Logging at runtime, right?
Thanks,
Sanne
On 14 March 2016 at 10:21, Sebastian Laskawiec wrote:
> I took a look at Nexus download statistics and Infinispan Uberjars are about
> 7% of our downloads (of course this calculation has been based on our JBoss
> Nexus instance and we have no data from other mirrors).
>
> So, once we are clear how Uber Jars should work... let's take a look at one
> of the problems:
>
> Many of our modules use JBoss Logging
> Uber jars relocate dependencies (like the one above) into infinispan
> package. The result for JBoss Logging would be:
> infinispan.org.jboss.logging*
> Most of our modules are compiled with small jars (like Spring integration),
> so the compile dependency for Spring integration is org.jboss.logging (not
> infinispan.org.jboss.logging)
> If someone wants to use Spring with Uber Jars (which should be absolutely
> fine), he gets a clash... Spring module can't find JBoss Logging
>
> There are several options to solve that:
>
> Stop relocating dependencies
>
> Pro: everything should work without problems
> Con: our dependencies will clash with client's dependencies
>
> Wrap 3rd party dependency with our own wrappers. In case of JBoss Logging -
> develop our own facade and put it in common.
>
> Pro: 3rd party dependencies will not leak between our modules
> Con: A lot of effort and rather small gain
>
> Develop 3rd party dependencies as Uber Jars. In case of Spring - we will
> have a Spring Uber Jar. I think Dan put this idea on the table.
>
> Pro: Lots of Uber Jars for everyone
> Con: How to use small jars in this scenario?
>
> Mark all module dependencies as provided and let users add small/uber jars
> themselves.
>
> Pro: It should give us the best results with the smallest amount of time
> Cons: Not all scenarios will be solved. We will also need to stop relocating
> some dependencies (like logging facades which are used almost everywhere)
>
> Having in mind our download statistics I would go for #4 (even though it
> doesn't solve the problem entirely). However #2 seems like a better fit for
> long term maintenance and we should proceed with this direction if number of
> Uber Jars users will grow (I can set a reminder in a 3-4 months time).
>
> Does it sound reasonable?
>
> Thanks
> Sebastian
>
> On Mon, Mar 14, 2016 at 8:42 AM, Tristan Tarrant
> wrote:
>>
>>
>>
>> On 11/03/2016 18:20, Galder Zamarre?o wrote:
>> > Are uber jars really that useful? From my own experience they often get
>>
>> The number of users who don't use a dependency management system (Maven,
>> Ivy, Gradle) is quite a lot higher than you'd expect.
>>
>> Tristan
>> --
>> Tristan Tarrant
>> Infinispan Lead
>> JBoss, a division of Red Hat
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From sanne at infinispan.org Mon Mar 14 06:57:07 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Mon, 14 Mar 2016 10:57:07 +0000
Subject: [infinispan-dev] Uber jars - how do we want to use them
In-Reply-To:
References:
<56E66B7D.70901@redhat.com>
Message-ID:
To make a more practical proposal:
give me a way to start a local, non transactional Cache which doesn't
require any of:
- JGroups
- JBoss Marshalling
- JTA API
- Infinispan Commons
If I don't have - for example - JGroups and am starting a clustered
cache, I will see an exception "Enabling Clustering on Cache {1}
requires a Transport Module such as
org.infinispan:transport-jgroups:9.0.0.Final. No transport module
found on classpath."
On 14 March 2016 at 10:50, Sanne Grinovero wrote:
> Hi,
> uber jars were introduced as an answer to complaints such as "there
> are too many jars" but I still think this was the wrong answer.. too
> many issues so please stop this: it's not helping usability to
> understand which jars are needed, and it makes things worse with
> runtime errors not matching source code.
>
> It was my impression that uber jars were a temporary solution to
> improve things, while the real improvements - to actually need less at
> runtime, or provide better errors pointing out what's needed - would
> take more time.
>
> So rather than stubbornly spend time maintaining this approach which
> is so obviously broken, I'd rather see time spent in pursuing the
> better improvements so that we can finally drop this.
>
> Sebastian: I din't understand the idea behind proposal 2#: what does
> it meant to provide our own facade to JBoss Logging? One will still
> need to have JBoss Logging at runtime, right?
>
> Thanks,
> Sanne
>
> On 14 March 2016 at 10:21, Sebastian Laskawiec wrote:
>> I took a look at Nexus download statistics and Infinispan Uberjars are about
>> 7% of our downloads (of course this calculation has been based on our JBoss
>> Nexus instance and we have no data from other mirrors).
>>
>> So, once we are clear how Uber Jars should work... let's take a look at one
>> of the problems:
>>
>> Many of our modules use JBoss Logging
>> Uber jars relocate dependencies (like the one above) into infinispan
>> package. The result for JBoss Logging would be:
>> infinispan.org.jboss.logging*
>> Most of our modules are compiled with small jars (like Spring integration),
>> so the compile dependency for Spring integration is org.jboss.logging (not
>> infinispan.org.jboss.logging)
>> If someone wants to use Spring with Uber Jars (which should be absolutely
>> fine), he gets a clash... Spring module can't find JBoss Logging
>>
>> There are several options to solve that:
>>
>> Stop relocating dependencies
>>
>> Pro: everything should work without problems
>> Con: our dependencies will clash with client's dependencies
>>
>> Wrap 3rd party dependency with our own wrappers. In case of JBoss Logging -
>> develop our own facade and put it in common.
>>
>> Pro: 3rd party dependencies will not leak between our modules
>> Con: A lot of effort and rather small gain
>>
>> Develop 3rd party dependencies as Uber Jars. In case of Spring - we will
>> have a Spring Uber Jar. I think Dan put this idea on the table.
>>
>> Pro: Lots of Uber Jars for everyone
>> Con: How to use small jars in this scenario?
>>
>> Mark all module dependencies as provided and let users add small/uber jars
>> themselves.
>>
>> Pro: It should give us the best results with the smallest amount of time
>> Cons: Not all scenarios will be solved. We will also need to stop relocating
>> some dependencies (like logging facades which are used almost everywhere)
>>
>> Having in mind our download statistics I would go for #4 (even though it
>> doesn't solve the problem entirely). However #2 seems like a better fit for
>> long term maintenance and we should proceed with this direction if number of
>> Uber Jars users will grow (I can set a reminder in a 3-4 months time).
>>
>> Does it sound reasonable?
>>
>> Thanks
>> Sebastian
>>
>> On Mon, Mar 14, 2016 at 8:42 AM, Tristan Tarrant
>> wrote:
>>>
>>>
>>>
>>> On 11/03/2016 18:20, Galder Zamarre?o wrote:
>>> > Are uber jars really that useful? From my own experience they often get
>>>
>>> The number of users who don't use a dependency management system (Maven,
>>> Ivy, Gradle) is quite a lot higher than you'd expect.
>>>
>>> Tristan
>>> --
>>> Tristan Tarrant
>>> Infinispan Lead
>>> JBoss, a division of Red Hat
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From dan.berindei at gmail.com Mon Mar 14 08:03:52 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 14 Mar 2016 14:03:52 +0200
Subject: [infinispan-dev] Uber jars - how do we want to use them
In-Reply-To:
References:
<56E66B7D.70901@redhat.com>
Message-ID:
On Mon, Mar 14, 2016 at 12:21 PM, Sebastian Laskawiec
wrote:
> I took a look at Nexus download statistics and Infinispan Uberjars are about
> 7% of our downloads (of course this calculation has been based on our JBoss
> Nexus instance and we have no data from other mirrors).
>
> So, once we are clear how Uber Jars should work... let's take a look at one
> of the problems:
>
> Many of our modules use JBoss Logging
> Uber jars relocate dependencies (like the one above) into infinispan
> package. The result for JBoss Logging would be:
> infinispan.org.jboss.logging*
> Most of our modules are compiled with small jars (like Spring integration),
> so the compile dependency for Spring integration is org.jboss.logging (not
> infinispan.org.jboss.logging)
> If someone wants to use Spring with Uber Jars (which should be absolutely
> fine), he gets a clash... Spring module can't find JBoss Logging
>
> There are several options to solve that:
>
> Stop relocating dependencies
>
> Pro: everything should work without problems
> Con: our dependencies will clash with client's dependencies
>
> Wrap 3rd party dependency with our own wrappers. In case of JBoss Logging -
> develop our own facade and put it in common.
>
> Pro: 3rd party dependencies will not leak between our modules
> Con: A lot of effort and rather small gain
>
> Develop 3rd party dependencies as Uber Jars. In case of Spring - we will
> have a Spring Uber Jar. I think Dan put this idea on the table.
>
> Pro: Lots of Uber Jars for everyone
> Con: How to use small jars in this scenario?
>
> Mark all module dependencies as provided and let users add small/uber jars
> themselves.
>
> Pro: It should give us the best results with the smallest amount of time
> Cons: Not all scenarios will be solved. We will also need to stop relocating
> some dependencies (like logging facades which are used almost everywhere)
>
> Having in mind our download statistics I would go for #4 (even though it
> doesn't solve the problem entirely). However #2 seems like a better fit for
> long term maintenance and we should proceed with this direction if number of
> Uber Jars users will grow (I can set a reminder in a 3-4 months time).
>
> Does it sound reasonable?
+1 for option 4. The embedded uber-jar doesn't expose any class in
org.jboss.logging, so it doesn't provide jboss-logging. The Spring
module should depend on jboss-logging itself, not assume that it's
going to be provided by another module.
One complication we have now is that we've made jboss-logging a part
of our API: one module's i18n Log interface can extend another
module's Log, and that means it extends the BasicLogger from
jboss-logging. I believe the Log interfaces should be private, and
each module should extend BasicLogger directly.
Cheers
Dan
From emmanuel at hibernate.org Mon Mar 14 10:02:01 2016
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Mon, 14 Mar 2016 15:02:01 +0100
Subject: [infinispan-dev] Uber jars - how do we want to use them
In-Reply-To:
References:
<56E66B7D.70901@redhat.com>
Message-ID: <20160314140201.GQ47051@hibernate.org>
On Mon 2016-03-14 11:21, Sebastian Laskawiec wrote:
> I took a look at Nexus download statistics and Infinispan Uberjars are
> about 7% of our downloads (of course this calculation has been based on our
> JBoss Nexus instance and we have no data from other mirrors).
It's actually the opposite, the way nexus stats work, the stats you see
from our nexus are actually the maven central stats minus direct hits
from our nexus instance.
Don't ask.
From pedro at infinispan.org Mon Mar 14 15:14:26 2016
From: pedro at infinispan.org (Pedro Ruivo)
Date: Mon, 14 Mar 2016 19:14:26 +0000
Subject: [infinispan-dev] Distributed Counter Discussion
Message-ID: <56E70D92.1010206@infinispan.org>
Hi everybody,
Discussion about distributed counters.
== Public API ==
interface Counter
String getName() //counter name
long get() //current value. may return stale value due to concurrent
operations to other nodes.
void increment() //async or sync increment. default add(1)
void decrement() //async or sync decrement. default add(-1)
void add(long) //async or sync add.
void reset() //resets to initial value
Note: Tried to make the interface as simple as possible with support for
sync and async operations. To avoid any confusion, I consider an async
operation as happening somewhat in the future, i.e. eventually
increments/decrements.
The sync operation happens somewhat during the method execution.
interface AtomiCounter extends Counter
long addAndGet() //adds a returns the new value. sync operation
long incrementAndGet() //increments and returns the new value. sync
operation. default addAndGet(1)
long decrementAndGet() //decrements and returns the new value. sync
operation. default addAndGet(-1)
interface AdvancedCounter extends Counter
long getMin/MaxThreshold() //returns the min and max threshold value
void add/removeListener() //adds a listener that is invoked when the
value change. Can be extended to notify when it is "reseted" and when
the threshold is reached.
Note: should this interface be splitted?
== Details ==
This is what I have in mind. Two counter managers: one based on JGroups
counter and another one based on Infinispan cache.
The first one creates AtomicCounters and it first perfectly. All
counters are created with an initial value (zero by default)
The second generates counters with all the options available. It can mix
sync/async operation and all counters will be in the same cache. The
cache will be configure by us and it would be an internal cache. This
will use all the features available in the cache.
Configuration-wise, I'm thinking about 2 parameters: number of backups
and timeout (for sync operations).
So, comment bellow and let me know alternatives, improvement or if I
missed something.
ps. I also consider implement a counter based on JGroups-raft but I
believe it is an overkill.
ps2. sorry for the long email :( I tried to be shorter as possible.
Cheers,
Pedro
From sanne at infinispan.org Mon Mar 14 17:27:44 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Mon, 14 Mar 2016 21:27:44 +0000
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56E70D92.1010206@infinispan.org>
References: <56E70D92.1010206@infinispan.org>
Message-ID:
Great starting point!
Some comments inline:
On 14 March 2016 at 19:14, Pedro Ruivo wrote:
> Hi everybody,
>
> Discussion about distributed counters.
>
> == Public API ==
>
> interface Counter
As a user, how do I get a Counter instance? From a the CacheContainer interface?
Will they have their own configuration section in the configuration file?
>
> String getName() //counter name
> long get() //current value. may return stale value due to concurrent
> operations to other nodes.
This is what puzzles me the most. I'm not sure if the feature is
actually useful, unless we can clearly state how far outdated the
value could be.
I think a slightly more formal definition would be in order. For
example I think it would be acceptable to say that this will return a
value from the range of values the primary owner of this counter was
holding in the timeframe between the method is being invoked and the
time the value is returned.
Could it optionally be integrated with Total Order ? Transactions?
> void increment() //async or sync increment. default add(1)
> void decrement() //async or sync decrement. default add(-1)
> void add(long) //async or sync add.
> void reset() //resets to initial value
>
> Note: Tried to make the interface as simple as possible with support for
> sync and async operations. To avoid any confusion, I consider an async
> operation as happening somewhat in the future, i.e. eventually
> increments/decrements.
> The sync operation happens somewhat during the method execution.
>
> interface AtomiCounter extends Counter
>
> long addAndGet() //adds a returns the new value. sync operation
> long incrementAndGet() //increments and returns the new value. sync
> operation. default addAndGet(1)
> long decrementAndGet() //decrements and returns the new value. sync
> operation. default addAndGet(-1)
>
> interface AdvancedCounter extends Counter
>
> long getMin/MaxThreshold() //returns the min and max threshold value
"threshold" ??
> void add/removeListener() //adds a listener that is invoked when the
> value change. Can be extended to notify when it is "reseted" and when
> the threshold is reached.
>
> Note: should this interface be splitted?
I'd prefer a single interface, with reduced redundancy.
For example, is there really a benefit in having a "void increment()"
and also a "long addAndGet()" ? [Besides The fact that only the first
one can benefit of an async option]
Besides, I am no longer sure that it's good thing that methods in
Infinispan can be async vs sync depending on configuration switches;
I'd rather make it explicit in the signature and simplify the
configuration by removing such a flag.
Making the methods which are async-capable to look "explicitly async"
should also allow us to add completeable futures & similar.
>
> == Details ==
>
> This is what I have in mind. Two counter managers: one based on JGroups
> counter and another one based on Infinispan cache.
> The first one creates AtomicCounters and it first perfectly. All
> counters are created with an initial value (zero by default)
> The second generates counters with all the options available. It can mix
> sync/async operation and all counters will be in the same cache. The
> cache will be configure by us and it would be an internal cache. This
> will use all the features available in the cache.
Rather than a split organized on the backing implementation details,
I'd prefer to see a split based on purpose of the counter.
For example JGroups has different styles of counters: one might need
one to do an atomic increment which is monotonically increasing across
the whole cluster - from the point of view from an omniscent observer
- but in many cases we just need the generation of a guarantee of a
monotonic unique number: that allows for example to have each node
pre-allocate a range of values and pre-fetch a new block while its
running out of values.
For some systems this is not acceptable because a failing server might
result in some of its allocated numbers to not ever be used, so
creating gaps; for others it's not acceptable that the figures
returned by the "monotonic counter" are not monotonic across each
other when confronting results from different cluster nodes, but for
many use cases that's acceptable. My point being that we need to be
able to choose between these possible semantics.
As an Infinispan consumer being able to express such details is
essential, while having to choose between "backed by JGroups counters"
or not is irrelevant and actually makes it harder.
In terms of Infinispan integration, the most important point I'd like
to see is persistence: i.e. make sure you can store them in a
CacheStore.
How we do that efficiently will also affect design; for example I was
expecting to see a Counter as something which is stored in a Cache, so
inheriting details such as configured CacheStore(s) and number of
owners.
Thanks for starting this!
Sanne
>
> Configuration-wise, I'm thinking about 2 parameters: number of backups
> and timeout (for sync operations).
>
> So, comment bellow and let me know alternatives, improvement or if I
> missed something.
>
> ps. I also consider implement a counter based on JGroups-raft but I
> believe it is an overkill.
> ps2. sorry for the long email :( I tried to be shorter as possible.
>
> Cheers,
> Pedro
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From rhauch at redhat.com Mon Mar 14 18:17:25 2016
From: rhauch at redhat.com (Randall Hauch)
Date: Mon, 14 Mar 2016 17:17:25 -0500
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56E70D92.1010206@infinispan.org>
References: <56E70D92.1010206@infinispan.org>
Message-ID: <80FC3F6D-F7F8-4CBE-90CC-FEED3ADE86F1@redhat.com>
What are the requirements? What are the distributed counters for? Is the counter to be monotonically increasing? Can there be any missed values? Does the counter need to increment and decrement? What is the *smallest* API you need initially?
There are two choices when implementing a distributed counter: use central coordination (like JGroups counters), or use independent counters on separate machines that will eventually converge to the correct value (CRDTs). Coordinated counters are expensive and therefore slow, and can suffer from problems during network or cluster problems. For example, what happens during a split brain? OTOH, CRDTs are decentralized so therefore are very fast, easily merged, and fault tolerant; they?re excellent when counting things that are occurring independently and therefore may be more suited for monitoring/metrics/accumulators/etc. Both have very different behaviors under ideal and failure scenarios, have different performance and consistency guarantees, and are useful in different scenarios. Make sure you choose accordingly.
For information about CRDTs, make sure you?ve read the CRDT paper by Shapiro: http://hal.upmc.fr/inria-00555588/document
Randall
> On Mar 14, 2016, at 2:14 PM, Pedro Ruivo wrote:
>
> Hi everybody,
>
> Discussion about distributed counters.
>
> == Public API ==
>
> interface Counter
>
> String getName() //counter name
> long get() //current value. may return stale value due to concurrent
> operations to other nodes.
> void increment() //async or sync increment. default add(1)
> void decrement() //async or sync decrement. default add(-1)
> void add(long) //async or sync add.
> void reset() //resets to initial value
>
> Note: Tried to make the interface as simple as possible with support for
> sync and async operations. To avoid any confusion, I consider an async
> operation as happening somewhat in the future, i.e. eventually
> increments/decrements.
> The sync operation happens somewhat during the method execution.
>
> interface AtomiCounter extends Counter
>
> long addAndGet() //adds a returns the new value. sync operation
> long incrementAndGet() //increments and returns the new value. sync
> operation. default addAndGet(1)
> long decrementAndGet() //decrements and returns the new value. sync
> operation. default addAndGet(-1)
>
> interface AdvancedCounter extends Counter
>
> long getMin/MaxThreshold() //returns the min and max threshold value
> void add/removeListener() //adds a listener that is invoked when the
> value change. Can be extended to notify when it is "reseted" and when
> the threshold is reached.
>
> Note: should this interface be splitted?
>
> == Details ==
>
> This is what I have in mind. Two counter managers: one based on JGroups
> counter and another one based on Infinispan cache.
> The first one creates AtomicCounters and it first perfectly. All
> counters are created with an initial value (zero by default)
> The second generates counters with all the options available. It can mix
> sync/async operation and all counters will be in the same cache. The
> cache will be configure by us and it would be an internal cache. This
> will use all the features available in the cache.
>
> Configuration-wise, I'm thinking about 2 parameters: number of backups
> and timeout (for sync operations).
>
> So, comment bellow and let me know alternatives, improvement or if I
> missed something.
>
> ps. I also consider implement a counter based on JGroups-raft but I
> believe it is an overkill.
> ps2. sorry for the long email :( I tried to be shorter as possible.
>
> Cheers,
> Pedro
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160314/fcdd1c15/attachment.html
From bban at redhat.com Tue Mar 15 03:00:55 2016
From: bban at redhat.com (Bela Ban)
Date: Tue, 15 Mar 2016 08:00:55 +0100
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56E70D92.1010206@infinispan.org>
References: <56E70D92.1010206@infinispan.org>
Message-ID: <56E7B327.2050604@redhat.com>
What are the requirements? To offer a cluster wide counter from an
Infinispan cache? Or is this going to be used internally, by Infinispan?
I've implemented 2 counters, in JGroups and in jgroups-raft. Here are
their properties:
#1 JGroups counter:
- Cluster wide _named_ counters with atomicity guarantees
- A counter is always managed by the current coordinator
- In a cluster split, you can end up with different counter instances
(one per coordinator)
- When a cluster partition heals, the current coordinator's counter
'wins', ie. values from the other counters are lost, even if they were
updated last
#2 jgroups-raft counter:
- Counter is managed by the leader
- A counter will not be accessible when there's no leader (no majority):
this is CP in CAP
- Counters cannot diverge in split brain scenarios; there's always at
most 1 counter (or 0)
If you can live with multiple counters being present in light of network
partitiond, then pick #1 as this is faster than #2 (which requires disk
writes for updates), and is always available, whereas #2 can be
non-available.
I'm thinking about changing the way #1 works. Currently, it uses the
next-in-line to backup a counter value on each update. I'd like to
remove this and replace it with a reconciliation round when the
coordinator changes: the new coordinator asks all members for their
current counter values. Requires a bit more work on a view change, but
less in steady state mode. Perhaps I can make it configurable.
On 14/03/16 20:14, Pedro Ruivo wrote:
> Hi everybody,
>
> Discussion about distributed counters.
>
> == Public API ==
>
> interface Counter
>
> String getName() //counter name
> long get() //current value. may return stale value due to concurrent
> operations to other nodes.
> void increment() //async or sync increment. default add(1)
> void decrement() //async or sync decrement. default add(-1)
> void add(long) //async or sync add.
> void reset() //resets to initial value
>
> Note: Tried to make the interface as simple as possible with support for
> sync and async operations. To avoid any confusion, I consider an async
> operation as happening somewhat in the future, i.e. eventually
> increments/decrements.
> The sync operation happens somewhat during the method execution.
>
> interface AtomiCounter extends Counter
>
> long addAndGet() //adds a returns the new value. sync operation
> long incrementAndGet() //increments and returns the new value. sync
> operation. default addAndGet(1)
> long decrementAndGet() //decrements and returns the new value. sync
> operation. default addAndGet(-1)
>
> interface AdvancedCounter extends Counter
>
> long getMin/MaxThreshold() //returns the min and max threshold value
> void add/removeListener() //adds a listener that is invoked when the
> value change. Can be extended to notify when it is "reseted" and when
> the threshold is reached.
>
> Note: should this interface be splitted?
>
> == Details ==
>
> This is what I have in mind. Two counter managers: one based on JGroups
> counter and another one based on Infinispan cache.
> The first one creates AtomicCounters and it first perfectly. All
> counters are created with an initial value (zero by default)
> The second generates counters with all the options available. It can mix
> sync/async operation and all counters will be in the same cache. The
> cache will be configure by us and it would be an internal cache. This
> will use all the features available in the cache.
>
> Configuration-wise, I'm thinking about 2 parameters: number of backups
> and timeout (for sync operations).
>
> So, comment bellow and let me know alternatives, improvement or if I
> missed something.
>
> ps. I also consider implement a counter based on JGroups-raft but I
> believe it is an overkill.
> ps2. sorry for the long email :( I tried to be shorter as possible.
>
> Cheers,
> Pedro
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
--
Bela Ban, JGroups lead (http://www.jgroups.org)
From bban at redhat.com Tue Mar 15 03:12:30 2016
From: bban at redhat.com (Bela Ban)
Date: Tue, 15 Mar 2016 08:12:30 +0100
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <80FC3F6D-F7F8-4CBE-90CC-FEED3ADE86F1@redhat.com>
References: <56E70D92.1010206@infinispan.org>
<80FC3F6D-F7F8-4CBE-90CC-FEED3ADE86F1@redhat.com>
Message-ID: <56E7B5DE.6010606@redhat.com>
On 14/03/16 23:17, Randall Hauch wrote:
> What are the requirements? What are the distributed counters for? Is the
> counter to be monotonically increasing? Can there be any missed values?
> Does the counter need to increment and decrement? What is the *smallest*
> API you need initially?
>
> There are two choices when implementing a distributed counter: use
> central coordination (like JGroups counters), or use independent
> counters on separate machines that will eventually converge to the
> correct value (CRDTs). Coordinated counters are expensive and therefore
> slow, and can suffer from problems during network or cluster problems.
The question is what do you get for this? If your app can't afford
duplicate counter values during a network partition, then - yes - there
is some overhead. CRDTs won't be able to guarantee this property. OTOH
CRDTs are fast when you only care about some sort of eventual
consistency, and don't need 'hard' consistency.
> For example, what happens during a split brain? OTOH, CRDTs are
> decentralized so therefore are very fast, easily merged, and fault
> tolerant;
Yes, CRDTS are AP whereas jgroups-raft counters are CP. JGroups
counters, otoh, are CRAP (consistent, reliable, available and
partition-aware.
Take the last sentence with a grain of salt :-)
> they?re excellent when counting things that are occurring
> independently and therefore may be more suited for
> monitoring/metrics/accumulators/etc. Both have very different behaviors
> under ideal and failure scenarios, have different performance and
> consistency guarantees, and are useful in different scenarios. Make sure
> you choose accordingly.
>
> For information about CRDTs, make sure you?ve read the CRDT paper by
> Shapiro: http://hal.upmc.fr/inria-00555588/document
>
> Randall
>
>> On Mar 14, 2016, at 2:14 PM, Pedro Ruivo > > wrote:
>>
>> Hi everybody,
>>
>> Discussion about distributed counters.
>>
>> == Public API ==
>>
>> interface Counter
>>
>> String getName() //counter name
>> long get()//current value. may return stale value due to concurrent
>> operations to other nodes.
>> void increment() //async or sync increment. default add(1)
>> void decrement() //async or sync decrement. default add(-1)
>> void add(long) //async or sync add.
>> void reset() //resets to initial value
>>
>> Note: Tried to make the interface as simple as possible with support for
>> sync and async operations. To avoid any confusion, I consider an async
>> operation as happening somewhat in the future, i.e. eventually
>> increments/decrements.
>> The sync operation happens somewhat during the method execution.
>>
>> interface AtomiCounter extends Counter
>>
>> long addAndGet() //adds a returns the new value. sync operation
>> long incrementAndGet() //increments and returns the new value. sync
>> operation. default addAndGet(1)
>> long decrementAndGet() //decrements and returns the new value. sync
>> operation. default addAndGet(-1)
>>
>> interface AdvancedCounter extends Counter
>>
>> long getMin/MaxThreshold() //returns the min and max threshold value
>> void add/removeListener() //adds a listener that is invoked when the
>> value change. Can be extended to notify when it is "reseted" and when
>> the threshold is reached.
>>
>> Note: should this interface be splitted?
>>
>> == Details ==
>>
>> This is what I have in mind. Two counter managers: one based on JGroups
>> counter and another one based on Infinispan cache.
>> The first one creates AtomicCounters and it first perfectly. All
>> counters are created with an initial value (zero by default)
>> The second generates counters with all the options available. It can mix
>> sync/async operation and all counters will be in the same cache. The
>> cache will be configure by us and it would be an internal cache. This
>> will use all the features available in the cache.
>>
>> Configuration-wise, I'm thinking about 2 parameters: number of backups
>> and timeout (for sync operations).
>>
>> So, comment bellow and let me know alternatives, improvement or if I
>> missed something.
>>
>> ps. I also consider implement a counter based on JGroups-raft but I
>> believe it is an overkill.
>> ps2. sorry for the long email :( I tried to be shorter as possible.
>>
>> Cheers,
>> Pedro
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
--
Bela Ban, JGroups lead (http://www.jgroups.org)
From ttarrant at redhat.com Tue Mar 15 03:38:37 2016
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Tue, 15 Mar 2016 08:38:37 +0100
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56E7B327.2050604@redhat.com>
References: <56E70D92.1010206@infinispan.org> <56E7B327.2050604@redhat.com>
Message-ID: <56E7BBFD.4080403@redhat.com>
On 15/03/2016 08:00, Bela Ban wrote:
> What are the requirements? To offer a cluster wide counter from an
> Infinispan cache? Or is this going to be used internally, by Infinispan?
The starting point is a requirement by apiman as a way to manage
throttling/quotas. Currently their requirement is not particularly
strict (i.e. synchronous), but they would like the ability to reset a
counter and possibly to receive an event when a certain threshold is
reached.
Obviously we'd like to cater to more use-cases, so the design should
take that into account.
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
From rvansa at redhat.com Tue Mar 15 04:26:27 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 15 Mar 2016 09:26:27 +0100
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To:
References: <56E70D92.1010206@infinispan.org>
Message-ID: <56E7C733.8080809@redhat.com>
I second Sanne's opinion about the sync/async API: make the API express
the synchronicity directly. I would even propose that for synchronous
methods, there's only the CompletableFuture or
CompletableFuture variant; we are already reaching the concurrency
limits of application that lets threads block, so let's give user a hint
that he should use such notification API. If anyone prefers the sync
variant, he can always use get().
Let's settle on some nomenclature, too, because these JGroups-counters
and RAFT-counters don't have commonly known properties. It almost seems
that the term *counter* is so overloaded that we shouldn't use it all.
There is a widely known j.u.c.AtomicLong, so if we want to implement
that (strict JMM-like properties), let's call it AtomicLong. Does not
have to follow j.u.c.AtomicLong API, but it's definitely CP.
Something providing unique values should be called Sequence. This will
be probably the one batching ranges (therefore possibly with gaps). By
default non-monotonic, but monotonicity could be a ctor arg (in a
similar way as fairness is set for j.u.concurrent.* classes).
As for CRDTs, I can't imagine how this could be easily built on top of
current 'passive' cache (without any syncing). But as for names *CRDT
counter* is explanatory enough.
If someone needs quotas, let's create Quota according to their needs
(soft and hard threshold, fluent operation below soft, some jitter when
above that). It seems that this will be closest to Sequence by reserving
some ranges. Don't let them shoot themselves into foot with some liger
counter.
And I hope you'll build these on top of the functional API, with at most
one RPC per operation.
Radim
On 03/14/2016 10:27 PM, Sanne Grinovero wrote:
> Great starting point!
> Some comments inline:
>
> On 14 March 2016 at 19:14, Pedro Ruivo wrote:
>> Hi everybody,
>>
>> Discussion about distributed counters.
>>
>> == Public API ==
>>
>> interface Counter
> As a user, how do I get a Counter instance? From a the CacheContainer interface?
>
> Will they have their own configuration section in the configuration file?
>
>> String getName() //counter name
>> long get() //current value. may return stale value due to concurrent
>> operations to other nodes.
> This is what puzzles me the most. I'm not sure if the feature is
> actually useful, unless we can clearly state how far outdated the
> value could be.
>
> I think a slightly more formal definition would be in order. For
> example I think it would be acceptable to say that this will return a
> value from the range of values the primary owner of this counter was
> holding in the timeframe between the method is being invoked and the
> time the value is returned.
>
> Could it optionally be integrated with Total Order ? Transactions?
>
>> void increment() //async or sync increment. default add(1)
>> void decrement() //async or sync decrement. default add(-1)
>> void add(long) //async or sync add.
>> void reset() //resets to initial value
>>
>> Note: Tried to make the interface as simple as possible with support for
>> sync and async operations. To avoid any confusion, I consider an async
>> operation as happening somewhat in the future, i.e. eventually
>> increments/decrements.
>> The sync operation happens somewhat during the method execution.
>>
>> interface AtomiCounter extends Counter
>>
>> long addAndGet() //adds a returns the new value. sync operation
>> long incrementAndGet() //increments and returns the new value. sync
>> operation. default addAndGet(1)
>> long decrementAndGet() //decrements and returns the new value. sync
>> operation. default addAndGet(-1)
>>
>> interface AdvancedCounter extends Counter
>>
>> long getMin/MaxThreshold() //returns the min and max threshold value
> "threshold" ??
>
>> void add/removeListener() //adds a listener that is invoked when the
>> value change. Can be extended to notify when it is "reseted" and when
>> the threshold is reached.
>>
>> Note: should this interface be splitted?
> I'd prefer a single interface, with reduced redundancy.
> For example, is there really a benefit in having a "void increment()"
> and also a "long addAndGet()" ? [Besides The fact that only the first
> one can benefit of an async option]
>
> Besides, I am no longer sure that it's good thing that methods in
> Infinispan can be async vs sync depending on configuration switches;
> I'd rather make it explicit in the signature and simplify the
> configuration by removing such a flag.
>
> Making the methods which are async-capable to look "explicitly async"
> should also allow us to add completeable futures & similar.
>
>> == Details ==
>>
>> This is what I have in mind. Two counter managers: one based on JGroups
>> counter and another one based on Infinispan cache.
>> The first one creates AtomicCounters and it first perfectly. All
>> counters are created with an initial value (zero by default)
>> The second generates counters with all the options available. It can mix
>> sync/async operation and all counters will be in the same cache. The
>> cache will be configure by us and it would be an internal cache. This
>> will use all the features available in the cache.
> Rather than a split organized on the backing implementation details,
> I'd prefer to see a split based on purpose of the counter.
>
> For example JGroups has different styles of counters: one might need
> one to do an atomic increment which is monotonically increasing across
> the whole cluster - from the point of view from an omniscent observer
> - but in many cases we just need the generation of a guarantee of a
> monotonic unique number: that allows for example to have each node
> pre-allocate a range of values and pre-fetch a new block while its
> running out of values.
>
> For some systems this is not acceptable because a failing server might
> result in some of its allocated numbers to not ever be used, so
> creating gaps; for others it's not acceptable that the figures
> returned by the "monotonic counter" are not monotonic across each
> other when confronting results from different cluster nodes, but for
> many use cases that's acceptable. My point being that we need to be
> able to choose between these possible semantics.
>
> As an Infinispan consumer being able to express such details is
> essential, while having to choose between "backed by JGroups counters"
> or not is irrelevant and actually makes it harder.
>
> In terms of Infinispan integration, the most important point I'd like
> to see is persistence: i.e. make sure you can store them in a
> CacheStore.
> How we do that efficiently will also affect design; for example I was
> expecting to see a Counter as something which is stored in a Cache, so
> inheriting details such as configured CacheStore(s) and number of
> owners.
>
> Thanks for starting this!
>
> Sanne
>
>> Configuration-wise, I'm thinking about 2 parameters: number of backups
>> and timeout (for sync operations).
>>
>> So, comment bellow and let me know alternatives, improvement or if I
>> missed something.
>>
>> ps. I also consider implement a counter based on JGroups-raft but I
>> believe it is an overkill.
>> ps2. sorry for the long email :( I tried to be shorter as possible.
>>
>> Cheers,
>> Pedro
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa
JBoss Performance Team
From gustavo at infinispan.org Tue Mar 15 05:04:43 2016
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Tue, 15 Mar 2016 09:04:43 +0000
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <80FC3F6D-F7F8-4CBE-90CC-FEED3ADE86F1@redhat.com>
References: <56E70D92.1010206@infinispan.org>
<80FC3F6D-F7F8-4CBE-90CC-FEED3ADE86F1@redhat.com>
Message-ID:
> For information about CRDTs, make sure you?ve read the CRDT paper by
> Shapiro: http://hal.upmc.fr/inria-00555588/document
>
On that topic, I recently came across the paper "Scalable Eventually
Consistent Counters over Unreliable Networks" [1] which highlights some of
the scalability problems with CRDT counters at large scale: namely the
propagation and maintenance of a version vector that keeps growing
over time and requires broadcasts across all members. The paper then
describes how some implementors of CRDT counters tackled this limitation
by using "server side only" counters, and the inherent issues with that
approach. Finally it proposes an alternative called "Handoff counters" in
order
to overcome the scalability issue.
[1] http://arxiv.org/pdf/1307.3207v1.pdf
_______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160315/20a88929/attachment.html
From pedro at infinispan.org Tue Mar 15 07:58:20 2016
From: pedro at infinispan.org (Pedro Ruivo)
Date: Tue, 15 Mar 2016 11:58:20 +0000
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To:
References: <56E70D92.1010206@infinispan.org>
Message-ID: <56E7F8DC.9060000@infinispan.org>
Comments inline
On 03/14/2016 09:27 PM, Sanne Grinovero wrote:
> On 14 March 2016 at 19:14, Pedro Ruivo wrote:
>
> As a user, how do I get a Counter instance? From a the CacheContainer interface?
You can get a Counter from a CounterManager. The CounterManager is
associated to the CacheManager.
I have two options to get the CounterManager:
CacheManager.as(CouterManager.class), as discussed during the Rome
meeting (2015). This will change the API in the core but it allows to
introduce the other building blocked (locks, sequences, latches, ...)
The other alternative I have in mind is CounterMager.get(CacheManager).
The only advantage I see is no need to change the API in core.
guys... comment what alternative do you prefer...
>
> Will they have their own configuration section in the configuration file?
I have no idea here... probably something like the cache store
configurations would work.
>> long get() //current value. may return stale value due to concurrent
>> operations to other nodes.
>
> This is what puzzles me the most. I'm not sure if the feature is
> actually useful, unless we can clearly state how far outdated the
> value could be.
>
> I think a slightly more formal definition would be in order. For
> example I think it would be acceptable to say that this will return a
> value from the range of values the primary owner of this counter was
> holding in the timeframe between the method is being invoked and the
> time the value is returned.
>
> Could it optionally be integrated with Total Order ? Transactions?
That's why we are having this discussion. If someone heard some
requirements from an user, it would be good to write them here.
IMO, at least, the get() must include all the finished operations in the
local node.
>>
>> long getMin/MaxThreshold() //returns the min and max threshold value
>
> "threshold" ??
See Tristan's reply.
>
> I'd prefer a single interface, with reduced redundancy.
> For example, is there really a benefit in having a "void increment()"
> and also a "long addAndGet()" ? [Besides The fact that only the first
> one can benefit of an async option]
>
> Besides, I am no longer sure that it's good thing that methods in
> Infinispan can be async vs sync depending on configuration switches;
> I'd rather make it explicit in the signature and simplify the
> configuration by removing such a flag.
>
> Making the methods which are async-capable to look "explicitly async"
> should also allow us to add completeable futures & similar.
So, lets revamp the API. We can drop the AtomiCounter and create a
simple Counter interface like:
String getName()
CompletableFuture addAndGet()
long get()
void reset()
It is the simplest we can get and it covers all the semantics we need so
far. If something does not fit, we can start improving it.
>
> Rather than a split organized on the backing implementation details,
> I'd prefer to see a split based on purpose of the counter.
So, what about multiple CounterManager where in each implementation we
specified what semantic the Counter created ensures?
>
> For example JGroups has different styles of counters: one might need
> one to do an atomic increment which is monotonically increasing across
> the whole cluster - from the point of view from an omniscent observer
> - but in many cases we just need the generation of a guarantee of a
> monotonic unique number: that allows for example to have each node
> pre-allocate a range of values and pre-fetch a new block while its
> running out of values.
you lost me in the pre-allocate range of values. I think you are talking
about sequences (we also discussed that in Rome) but I don't think it
should be the primary objective of a counter.
>
> For some systems this is not acceptable because a failing server might
> result in some of its allocated numbers to not ever be used, so
> creating gaps; for others it's not acceptable that the figures
> returned by the "monotonic counter" are not monotonic across each
> other when confronting results from different cluster nodes, but for
> many use cases that's acceptable. My point being that we need to be
> able to choose between these possible semantics.
>
> As an Infinispan consumer being able to express such details is
> essential, while having to choose between "backed by JGroups counters"
> or not is irrelevant and actually makes it harder.
>
> In terms of Infinispan integration, the most important point I'd like
> to see is persistence: i.e. make sure you can store them in a
> CacheStore.
> How we do that efficiently will also affect design; for example I was
> expecting to see a Counter as something which is stored in a Cache, so
> inheriting details such as configured CacheStore(s) and number of
> owners.
I'm ok to support persistence but I don't see any advantage. Does anyone
have a use case for it?
From rvansa at redhat.com Tue Mar 15 08:57:31 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 15 Mar 2016 13:57:31 +0100
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56E7F8DC.9060000@infinispan.org>
References: <56E70D92.1010206@infinispan.org>
<56E7F8DC.9060000@infinispan.org>
Message-ID: <56E806BB.8010803@redhat.com>
On 03/15/2016 12:58 PM, Pedro Ruivo wrote:
>
> On 03/14/2016 09:27 PM, Sanne Grinovero wrote:
>> Rather than a split organized on the backing implementation details,
>> I'd prefer to see a split based on purpose of the counter.
> So, what about multiple CounterManager where in each implementation we
> specified what semantic the Counter created ensures?
>
Please, don't do that. Different implementation = does the same thing
but has different internals. If it has different semantics (does
something different), use different interface. This way users will just
swap in 'more performant' implementation without thinking about the
consequences.
My 2c
R.
--
Radim Vansa
JBoss Performance Team
From rhauch at redhat.com Tue Mar 15 10:15:22 2016
From: rhauch at redhat.com (Randall Hauch)
Date: Tue, 15 Mar 2016 09:15:22 -0500
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56E7C733.8080809@redhat.com>
References: <56E70D92.1010206@infinispan.org>
<56E7C733.8080809@redhat.com>
Message-ID: <2B729151-6A22-49A9-9FC1-473220A3AF13@redhat.com>
> On Mar 15, 2016, at 3:26 AM, Radim Vansa wrote:
>
> I second Sanne's opinion about the sync/async API: make the API express
> the synchronicity directly. I would even propose that for synchronous
> methods, there's only the CompletableFuture or
> CompletableFuture variant; we are already reaching the concurrency
> limits of application that lets threads block, so let's give user a hint
> that he should use such notification API. If anyone prefers the sync
> variant, he can always use get().
>
> Let's settle on some nomenclature, too, because these JGroups-counters
> and RAFT-counters don't have commonly known properties. It almost seems
> that the term *counter* is so overloaded that we shouldn't use it all.
>
> There is a widely known j.u.c.AtomicLong, so if we want to implement
> that (strict JMM-like properties), let's call it AtomicLong. Does not
> have to follow j.u.c.AtomicLong API, but it's definitely CP.
>
> Something providing unique values should be called Sequence. This will
> be probably the one batching ranges (therefore possibly with gaps). By
> default non-monotonic, but monotonicity could be a ctor arg (in a
> similar way as fairness is set for j.u.concurrent.* classes).
+1 for differentiating in the API between a counter that a client increments (or resets) and a sequence generator that a client can use to obtain (non)monotonic values.
>
> As for CRDTs, I can't imagine how this could be easily built on top of
> current 'passive' cache (without any syncing). But as for names *CRDT
> counter* is explanatory enough.
>
> If someone needs quotas, let's create Quota according to their needs
> (soft and hard threshold, fluent operation below soft, some jitter when
> above that). It seems that this will be closest to Sequence by reserving
> some ranges. Don't let them shoot themselves into foot with some liger
> counter.
+1 for creating a specific interface to enable quotas. Perhaps it is implemented with some more general purpose functionality that might be exposed in the future, but for now why not keep it narrowly focused.
>
> And I hope you'll build these on top of the functional API, with at most
> one RPC per operation.
>
> Radim
>
> On 03/14/2016 10:27 PM, Sanne Grinovero wrote:
>> Great starting point!
>> Some comments inline:
>>
>> On 14 March 2016 at 19:14, Pedro Ruivo wrote:
>>> Hi everybody,
>>>
>>> Discussion about distributed counters.
>>>
>>> == Public API ==
>>>
>>> interface Counter
>> As a user, how do I get a Counter instance? From a the CacheContainer interface?
>>
>> Will they have their own configuration section in the configuration file?
>>
>>> String getName() //counter name
>>> long get() //current value. may return stale value due to concurrent
>>> operations to other nodes.
>> This is what puzzles me the most. I'm not sure if the feature is
>> actually useful, unless we can clearly state how far outdated the
>> value could be.
>>
>> I think a slightly more formal definition would be in order. For
>> example I think it would be acceptable to say that this will return a
>> value from the range of values the primary owner of this counter was
>> holding in the timeframe between the method is being invoked and the
>> time the value is returned.
>>
>> Could it optionally be integrated with Total Order ? Transactions?
>>
>>> void increment() //async or sync increment. default add(1)
>>> void decrement() //async or sync decrement. default add(-1)
>>> void add(long) //async or sync add.
>>> void reset() //resets to initial value
>>>
>>> Note: Tried to make the interface as simple as possible with support for
>>> sync and async operations. To avoid any confusion, I consider an async
>>> operation as happening somewhat in the future, i.e. eventually
>>> increments/decrements.
>>> The sync operation happens somewhat during the method execution.
>>>
>>> interface AtomiCounter extends Counter
>>>
>>> long addAndGet() //adds a returns the new value. sync operation
>>> long incrementAndGet() //increments and returns the new value. sync
>>> operation. default addAndGet(1)
>>> long decrementAndGet() //decrements and returns the new value. sync
>>> operation. default addAndGet(-1)
>>>
>>> interface AdvancedCounter extends Counter
>>>
>>> long getMin/MaxThreshold() //returns the min and max threshold value
>> "threshold" ??
>>
>>> void add/removeListener() //adds a listener that is invoked when the
>>> value change. Can be extended to notify when it is "reseted" and when
>>> the threshold is reached.
>>>
>>> Note: should this interface be splitted?
>> I'd prefer a single interface, with reduced redundancy.
>> For example, is there really a benefit in having a "void increment()"
>> and also a "long addAndGet()" ? [Besides The fact that only the first
>> one can benefit of an async option]
>>
>> Besides, I am no longer sure that it's good thing that methods in
>> Infinispan can be async vs sync depending on configuration switches;
>> I'd rather make it explicit in the signature and simplify the
>> configuration by removing such a flag.
>>
>> Making the methods which are async-capable to look "explicitly async"
>> should also allow us to add completeable futures & similar.
>>
>>> == Details ==
>>>
>>> This is what I have in mind. Two counter managers: one based on JGroups
>>> counter and another one based on Infinispan cache.
>>> The first one creates AtomicCounters and it first perfectly. All
>>> counters are created with an initial value (zero by default)
>>> The second generates counters with all the options available. It can mix
>>> sync/async operation and all counters will be in the same cache. The
>>> cache will be configure by us and it would be an internal cache. This
>>> will use all the features available in the cache.
>> Rather than a split organized on the backing implementation details,
>> I'd prefer to see a split based on purpose of the counter.
>>
>> For example JGroups has different styles of counters: one might need
>> one to do an atomic increment which is monotonically increasing across
>> the whole cluster - from the point of view from an omniscent observer
>> - but in many cases we just need the generation of a guarantee of a
>> monotonic unique number: that allows for example to have each node
>> pre-allocate a range of values and pre-fetch a new block while its
>> running out of values.
>>
>> For some systems this is not acceptable because a failing server might
>> result in some of its allocated numbers to not ever be used, so
>> creating gaps; for others it's not acceptable that the figures
>> returned by the "monotonic counter" are not monotonic across each
>> other when confronting results from different cluster nodes, but for
>> many use cases that's acceptable. My point being that we need to be
>> able to choose between these possible semantics.
>>
>> As an Infinispan consumer being able to express such details is
>> essential, while having to choose between "backed by JGroups counters"
>> or not is irrelevant and actually makes it harder.
>>
>> In terms of Infinispan integration, the most important point I'd like
>> to see is persistence: i.e. make sure you can store them in a
>> CacheStore.
>> How we do that efficiently will also affect design; for example I was
>> expecting to see a Counter as something which is stored in a Cache, so
>> inheriting details such as configured CacheStore(s) and number of
>> owners.
>>
>> Thanks for starting this!
>>
>> Sanne
>>
>>> Configuration-wise, I'm thinking about 2 parameters: number of backups
>>> and timeout (for sync operations).
>>>
>>> So, comment bellow and let me know alternatives, improvement or if I
>>> missed something.
>>>
>>> ps. I also consider implement a counter based on JGroups-raft but I
>>> believe it is an overkill.
>>> ps2. sorry for the long email :( I tried to be shorter as possible.
>>>
>>> Cheers,
>>> Pedro
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From rhauch at redhat.com Tue Mar 15 10:17:45 2016
From: rhauch at redhat.com (Randall Hauch)
Date: Tue, 15 Mar 2016 09:17:45 -0500
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56E7B5DE.6010606@redhat.com>
References: <56E70D92.1010206@infinispan.org>
<80FC3F6D-F7F8-4CBE-90CC-FEED3ADE86F1@redhat.com>
<56E7B5DE.6010606@redhat.com>
Message-ID: <69D897A4-54A4-4BE0-A954-996106318CFA@redhat.com>
> On Mar 15, 2016, at 2:12 AM, Bela Ban wrote:
>
>
>
> On 14/03/16 23:17, Randall Hauch wrote:
>> What are the requirements? What are the distributed counters for? Is the
>> counter to be monotonically increasing? Can there be any missed values?
>> Does the counter need to increment and decrement? What is the *smallest*
>> API you need initially?
>>
>> There are two choices when implementing a distributed counter: use
>> central coordination (like JGroups counters), or use independent
>> counters on separate machines that will eventually converge to the
>> correct value (CRDTs). Coordinated counters are expensive and therefore
>> slow, and can suffer from problems during network or cluster problems.
>
> The question is what do you get for this? If your app can't afford
> duplicate counter values during a network partition, then - yes - there
> is some overhead. CRDTs won't be able to guarantee this property. OTOH
> CRDTs are fast when you only care about some sort of eventual
> consistency, and don't need 'hard' consistency.
To be clear, I?m not saying they are interchangeable. They have very different properties, which is why the requirements will help determine which of them (if any) are applicable.
>
>
>> For example, what happens during a split brain? OTOH, CRDTs are
>> decentralized so therefore are very fast, easily merged, and fault
>> tolerant;
>
> Yes, CRDTS are AP whereas jgroups-raft counters are CP. JGroups
> counters, otoh, are CRAP (consistent, reliable, available and
> partition-aware.
>
> Take the last sentence with a grain of salt :-)
>
>> they?re excellent when counting things that are occurring
>> independently and therefore may be more suited for
>> monitoring/metrics/accumulators/etc. Both have very different behaviors
>> under ideal and failure scenarios, have different performance and
>> consistency guarantees, and are useful in different scenarios. Make sure
>> you choose accordingly.
>>
>> For information about CRDTs, make sure you?ve read the CRDT paper by
>> Shapiro: http://hal.upmc.fr/inria-00555588/document
>>
>> Randall
>>
>>> On Mar 14, 2016, at 2:14 PM, Pedro Ruivo
>>> >> wrote:
>>>
>>> Hi everybody,
>>>
>>> Discussion about distributed counters.
>>>
>>> == Public API ==
>>>
>>> interface Counter
>>>
>>> String getName() //counter name
>>> long get()//current value. may return stale value due to concurrent
>>> operations to other nodes.
>>> void increment() //async or sync increment. default add(1)
>>> void decrement() //async or sync decrement. default add(-1)
>>> void add(long) //async or sync add.
>>> void reset() //resets to initial value
>>>
>>> Note: Tried to make the interface as simple as possible with support for
>>> sync and async operations. To avoid any confusion, I consider an async
>>> operation as happening somewhat in the future, i.e. eventually
>>> increments/decrements.
>>> The sync operation happens somewhat during the method execution.
>>>
>>> interface AtomiCounter extends Counter
>>>
>>> long addAndGet() //adds a returns the new value. sync operation
>>> long incrementAndGet() //increments and returns the new value. sync
>>> operation. default addAndGet(1)
>>> long decrementAndGet() //decrements and returns the new value. sync
>>> operation. default addAndGet(-1)
>>>
>>> interface AdvancedCounter extends Counter
>>>
>>> long getMin/MaxThreshold() //returns the min and max threshold value
>>> void add/removeListener() //adds a listener that is invoked when the
>>> value change. Can be extended to notify when it is "reseted" and when
>>> the threshold is reached.
>>>
>>> Note: should this interface be splitted?
>>>
>>> == Details ==
>>>
>>> This is what I have in mind. Two counter managers: one based on JGroups
>>> counter and another one based on Infinispan cache.
>>> The first one creates AtomicCounters and it first perfectly. All
>>> counters are created with an initial value (zero by default)
>>> The second generates counters with all the options available. It can mix
>>> sync/async operation and all counters will be in the same cache. The
>>> cache will be configure by us and it would be an internal cache. This
>>> will use all the features available in the cache.
>>>
>>> Configuration-wise, I'm thinking about 2 parameters: number of backups
>>> and timeout (for sync operations).
>>>
>>> So, comment bellow and let me know alternatives, improvement or if I
>>> missed something.
>>>
>>> ps. I also consider implement a counter based on JGroups-raft but I
>>> believe it is an overkill.
>>> ps2. sorry for the long email :( I tried to be shorter as possible.
>>>
>>> Cheers,
>>> Pedro
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org >
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org )
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160315/c385e9ac/attachment-0001.html
From sanne at infinispan.org Tue Mar 15 15:22:57 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 15 Mar 2016 19:22:57 +0000
Subject: [infinispan-dev] XA transactions on a single CacheManager?
Message-ID:
Hi all,
I just noticed that when I'm making changes to multiple caches within
the same transaction, the transaction manager will treat this as XA
transactions.
That seems suboptimal as they are all managed by the same resource; is
there a configuration I'm missing or should I open a JIRA to ask for
an improvement?
I'm not just asking for performance reasons: having to setup a
transaction manager is an annoying configuration step for Java SE
users.
Thanks,
Sanne
From vjuranek at redhat.com Tue Mar 15 18:15:14 2016
From: vjuranek at redhat.com (Vojtech Juranek)
Date: Tue, 15 Mar 2016 23:15:14 +0100
Subject: [infinispan-dev] ISPN embedded tutorial broken
Message-ID: <6207433.jbZTtoz8p6@localhost>
Hi,
I noticed that ISPN embedded tutorial (weather app) [1] is broken. It fails
with NPE when fetching weather information from OpenWeatherMap. It's broken as
OpenWeatherMap started to request user registration [2]. I checked couple of
other free weather service (like Weather underground, Yahoo Weather, World
weather online) and all of them seem to require some kind of user
registration.
Shall we register on OpenWeatherMap (I didn't investigate if API token can be
pushed to GitHub) or switch to RandomWeatherService as default service?
Thanks
Vojta
[1] http://infinispan.org/tutorials/embedded/
[2] http://openweathermap.org/faq#error401
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part.
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160315/288d318d/attachment.bin
From slaskawi at redhat.com Wed Mar 16 02:22:56 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 16 Mar 2016 07:22:56 +0100
Subject: [infinispan-dev] ISPN embedded tutorial broken
In-Reply-To: <6207433.jbZTtoz8p6@localhost>
References: <6207433.jbZTtoz8p6@localhost>
Message-ID:
+1 for registering a user and putting API key in the repo.
On Tue, Mar 15, 2016 at 11:15 PM, Vojtech Juranek
wrote:
> Hi,
> I noticed that ISPN embedded tutorial (weather app) [1] is broken. It fails
> with NPE when fetching weather information from OpenWeatherMap. It's
> broken as
> OpenWeatherMap started to request user registration [2]. I checked couple
> of
> other free weather service (like Weather underground, Yahoo Weather, World
> weather online) and all of them seem to require some kind of user
> registration.
>
> Shall we register on OpenWeatherMap (I didn't investigate if API token can
> be
> pushed to GitHub) or switch to RandomWeatherService as default service?
>
> Thanks
> Vojta
>
> [1] http://infinispan.org/tutorials/embedded/
> [2] http://openweathermap.org/faq#error401
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160316/6ba81151/attachment.html
From galder at redhat.com Wed Mar 16 03:56:12 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 16 Mar 2016 08:56:12 +0100
Subject: [infinispan-dev] Verifying Consistent Hash based routing using stats
In-Reply-To:
References:
Message-ID:
Hey Vittorio,
Expanding this to dev list in case anyone has interest. Latest commit is in [1]. The relevant code that verifies Consistent Hash (CH) based routing is in [2]. The way it works is this:
Assuming I have 3 nodes, A, B and C, with 2 as number of copies for distribution, I do the following:
0. Get stats from nodes A, B, C.
1. Generate a key K1 whose owners are A,B.
2. Generate a key K2 whose owners are B,C.
3. Generate a key K3 whose owners are C,A.
4. The primary node is always the first node of the owners, so if CH routing works fine, K1 would targeted for node A, K2 for node B and K3 for node C.
5. Call puts on keys K1, K2 and K3.
6. Get stats from nodes A, B, C.
7. For all nodes, "stores" value in latest stats should be +1 compared to initial stats. This would pass even if with a round-robin load balance policy.
8. For all nodes, "currentNumberOfEntries" value in latest stats should be (Number of copies) * ("stores"). This is the key assumption that guarantees CH routing works as expected.
^ To be more precise, this can only be guaranteed under certain circumstances: First, the ownership needs to be spread around, so a node does not take owner ship of 3 keys and another of only 1. We can guarantee that based on how we generate the keys. The second part is that if a put happens in a non-owner, this non-owner node will increase its "stores" values but not its "currentNumberOfEntries" since the key does not belong in this node. So, if any put request happened in a non-owner, there would be an inbalance in the ratio of currentNumberOfEntries to "stores".
Hope this helps.
Cheers,
[1] https://github.com/galderz/js-client/commit/c57c285561b73e67b7ebce04de67d99086b5b726
[2] https://github.com/galderz/js-client/blob/t_steps/spec/infinispan_cluster_spec.js#L72
--
Galder Zamarre?o
Infinispan, Red Hat
> On 16 Mar 2016, at 08:25, Galder Zamarre?o wrote:
>
> Hey,
>
> Sorry I missed your reply yesterday about the location of the code.
>
> I've not pushed my branch yet, I'll do that this morning. As soon as it's on Github I'll let you know :)
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
From ttarrant at redhat.com Wed Mar 16 04:20:20 2016
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Wed, 16 Mar 2016 09:20:20 +0100
Subject: [infinispan-dev] ISPN embedded tutorial broken
In-Reply-To: <6207433.jbZTtoz8p6@localhost>
References: <6207433.jbZTtoz8p6@localhost>
Message-ID: <56E91744.6040401@redhat.com>
I'd rather go for RandomWeatherService.
Tristan
On 15/03/2016 23:15, Vojtech Juranek wrote:
> Hi,
> I noticed that ISPN embedded tutorial (weather app) [1] is broken. It fails
> with NPE when fetching weather information from OpenWeatherMap. It's broken as
> OpenWeatherMap started to request user registration [2]. I checked couple of
> other free weather service (like Weather underground, Yahoo Weather, World
> weather online) and all of them seem to require some kind of user
> registration.
>
> Shall we register on OpenWeatherMap (I didn't investigate if API token can be
> pushed to GitHub) or switch to RandomWeatherService as default service?
>
> Thanks
> Vojta
>
> [1] http://infinispan.org/tutorials/embedded/
> [2] http://openweathermap.org/faq#error401
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
From rvansa at redhat.com Wed Mar 16 04:51:45 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 16 Mar 2016 09:51:45 +0100
Subject: [infinispan-dev] XA transactions on a single CacheManager?
In-Reply-To:
References:
Message-ID: <56E91EA1.4020700@redhat.com>
Quite related, though for NON_XA: https://issues.jboss.org/browse/ISPN-5600
R.
On 03/15/2016 08:22 PM, Sanne Grinovero wrote:
> Hi all,
>
> I just noticed that when I'm making changes to multiple caches within
> the same transaction, the transaction manager will treat this as XA
> transactions.
>
> That seems suboptimal as they are all managed by the same resource; is
> there a configuration I'm missing or should I open a JIRA to ask for
> an improvement?
>
> I'm not just asking for performance reasons: having to setup a
> transaction manager is an annoying configuration step for Java SE
> users.
>
> Thanks,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa
JBoss Performance Team
From galder at redhat.com Wed Mar 16 06:26:50 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 16 Mar 2016 11:26:50 +0100
Subject: [infinispan-dev] Infinispan Javascript client 0.1.0 is out now!
In-Reply-To: <56CC887B.4090905@redhat.com>
References: <36BD5510-94EC-47BD-AA21-54C459165CD9@redhat.com>
<56C6E0EA.8050305@redhat.com>
<56C6EBBC.3080605@redhat.com>
<56CC887B.4090905@redhat.com>
Message-ID:
AFAIK, we're stuck with `function(client) { return ... ;}` since we need to support ECMAScript 5.
ECMAScript 6 brings in lambda syntax which will make this more palatable:
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Functions/Arrow_functions
Your suggestion would not really work since it would mean passing a value instead of a function, and we need the latter to be able to chain functions lazily.
Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat
> On 23 Feb 2016, at 17:27, Radim Vansa wrote:
>
> Thanks, though it was pretty clear even after your first reply.
>
> One more thing: it seems to me that there is certain redundancy in
> declaring
>
> function(client) { return ... ;}
>
> where only the ellipsis is the business logic. I would recommend
> including helper with higher-order functions, that would give remove the
> need for the above when you just called ispn API, please see how I've
> modified
>
> var put = before.then(function() { return client.put('k', 'v'); }); //
> from [2]
>
> into
>
> var put = before.then(cache().put('k', 'v')); // [3]
>
> ([3] includes stub of the helper). Or is this too non-JS-ish (TBH I
> haven't tested if that is valid JS)?
>
> Radim
>
> [3] https://gist.github.com/rvansa/ba5be94ba3bf7ae39a91
>
> On 02/23/2016 04:35 PM, Galder Zamarre?o wrote:
>> Radim, once again thanks for the invaluable feedback.
>>
>> As promised, I will ammend the README to make it much more clear but I found a very easy example that I can show here as I was working on the JS client testsuite.
>>
>> Imagine this scenario: I want to verify that after doing a put 'stores' statistic goes up by 1. The obvious thing to do here is this:
>>
>> 1. invoke stats to find out initial 'stores' value, then...
>> 2. invoke put, then...
>> 3. invoke stats again and compare with initial 'stores' value.
>>
>> Using Promises, this can be coded in two different ways. The first one is using nesting, e.g. [1]. This is a bit ugly but this is how you'd do have to do it if you were using function callbacks instead of Promises.
>>
>> The alternative is to chain Promises, taking into account that Promise's then() return a Promise or a directly value from within it. Here's an alternative solution that reads more cleanly [2].
>>
>> Here we can see how the first then() callback returns a value directly, which are the statistics before the put operation. The next two .then() calls return Promises, first a promise for the put, and the second a promise of the stats retrieved after put operation.
>>
>> Finally, we call Promise.all() which takes N promises and calls back .then() when all those promises have been fullfilled, and then passes the results of those promises as parameter. Within this last .then() callback, we can do the necessary assertions compare before and after stastistic values.
>>
>> Hope this clarifies things further.
>>
>> Cheers,
>>
>> p.s. 'stats' operation support is work in progress.
>>
>> [1] https://gist.github.com/galderz/65ed0b2a4e02b8f4e0c1
>> [2] https://gist.github.com/galderz/c349d3aa64a39684d7d7
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>> On 19 Feb 2016, at 11:17, Radim Vansa wrote:
>>>
>>> All right, this example looks much better. And the ability to use native
>>> variables rather than keys into the session is much more comfortable.
>>> Thanks for the explanation!
>>>
>>> Radim
>>>
>>> On 02/19/2016 10:53 AM, Galder Zamarre?o wrote:
>>>> --
>>>> Galder Zamarre?o
>>>> Infinispan, Red Hat
>>>>
>>>>> On 19 Feb 2016, at 10:31, Radim Vansa wrote:
>>>>>
>>>>> Just checked the readme, and TBH I wouldn't be too enthusiastic about
>>>>> the deep nesting of chained operations (basically each follow-up
>>>>> operation requires further level of nesting). Is that the way async
>>>>> handlers are usually written in JS?
>>>> I think the README I've produce is a bit confusing, I'll work to improve for future versions. In the mean time, let me explain this:
>>>>
>>>> The reason Promises are used instead of callback parameters is precisely to avoid this kind of deep nesting.
>>>>
>>>> The code below shows 4 (but could be N really) operations which are chained only 2 levels deep:
>>>>
>>>> connected.then(function(client) {
>>>> ... <- Do something with the client
>>>> var putCall = client.put('key', 'value');
>>>> var getCall = putCall.then(function() {
>>>> return client.get('key');
>>>> });
>>>> var containsKeyCall = getCall.then(function() {
>>>> return client.containsKey('key');
>>>> });
>>>> var removeCall = containsKeyCall.then(function(contains) {
>>>> return client.remove('key');
>>>> })
>>>> });
>>>>
>>>> The key aspect of this is that no matter how many operations you want to chain together, the nesting levels are constant. You could have 100 operations chained together, you would still only have 2 levels deep.
>>>>
>>>> The README file might not make this so obvious and hence I'll work to improve on that.
>>>>
>>>> Also, the API above is using Promise API which will be standard in ES6. Even though Infinispan JS Client is based on ES5, the `promise` module sticks to the forthcoming standard Promise API, and hence we're using something that in a few years will be standard, which is a good thing. Conceptually, Promise API is similar to Scala's Future or Java's CompletableFuture.
>>>>
>>>> The other alternative that's used in Node.js is passing callbacks as parameters to each put/get call... and that does cause very deep nesting, because if you want to chain 100 operations together, you need 100 nested calls! Plus not centralised error handling, and many other disadvantages.
>>>>
>>>>> Last few weeks I am working with Gatling and the syntax to describe
>>>>> scenarios = chained operations is quite nice (though it's Scala). Would
>>>>> it be possible to have a syntax like this?
>>>>>
>>>>> connected
>>>>> .then(function(client) {
>>>>> client.put('key', 'value');
>>>>> .then(function(client) {
>>>>> client.get('key').use(function(value) {
>>>>> console.log(':get(`key`) = ' + value);
>>>>> }).save('p0'); // this would be the same as use(function(value)
>>>>> { client.save('p0') })
>>>>> }.
>>>>> .then(function(client)) {
>>>>> client.remove('key', client.load('p0')).use(function(success) {
>>>>> console.log(':remove(`key`) = ' + success);
>>>>> });
>>>>> }).exec();
>>>> Independent of the correctness of the syntax, this API can be split in several concepts:
>>>>
>>>> 1. The first one is that for those operations that have not a return, e.g. a put, you're passing on the client instance. I think that sounds fine.
>>>>
>>>> 2. For your API to work, get() would need to return something other than Promise that has a use/save methods. I'm very reluctant to do anything like that since it's not standard and would require more upfront understanding from the user. Right now, we provide a Promise, a concept that's very well understood in the JS community and that will become standard in ES6.
>>>>
>>>>> Note that I've used conditional remove to demonstrate load of previously
>>>>> saved value. Also, I am mixing the concept of client = cache and
>>>>> session, but maybe these could go to single object.
>>>>>
>>>>> Just my 2c
>>>>>
>>>>> Radim
>>>>>
>>>>> On 02/18/2016 12:04 PM, Galder Zamarre?o wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Following on Tristan's anouncement, we've also released Infinispan Javascript client version 0.1.0. You can read all about it here:
>>>>>> http://blog.infinispan.org/2016/02/infinispan-javascript-client-010-is-out.html
>>>>>>
>>>>>> Cheers,
>>>>>> --
>>>>>> Galder Zamarre?o
>>>>>> Infinispan, Red Hat
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> --
>>>>> Radim Vansa
>>>>> JBoss Performance Team
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Radim Vansa
>>> JBoss Performance Team
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From sanne at infinispan.org Thu Mar 17 07:25:20 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 17 Mar 2016 11:25:20 +0000
Subject: [infinispan-dev] Practical example of difficulties with dependencies
Message-ID:
Since we recently discussed the purpose of the "UberJar"s , I think it
would be a great exercise to look at this discussion:
- https://developer.jboss.org/message/953081
and ask ourselves what we could have done to make the first steps of
this new user less miserable.
Personally I think we could go a long way with some clear
documentation about our dependencies;
it's not hard to inject version properties from the pom files into the
docs, so we could have a little reminder about - for example - which
version of Hibernate is expected to have the JPACacheStore working.
Also, I think it's clear that uber jars - at least in this case - are
not even close to the solution, unless you intend to include all of
Hibernate ORM and transitive dependencies..
Thanks,
Sanne
From slaskawi at redhat.com Thu Mar 17 07:48:04 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 17 Mar 2016 12:48:04 +0100
Subject: [infinispan-dev] Practical example of difficulties with
dependencies
In-Reply-To:
References:
Message-ID:
Hey!
If Hibernate had an Uber jar - one could add it to the classpath and fix
the problem :) Of course I'm not serious here...
So what are you proposing? One of the proposed solutions (in "Uber jars -
how do we want to use them" email thread) was to limit number of
dependencies and throw errors in the runtime if something is missing. I'm
slightly positive to the first part but really negative to the second (it
only delays the problem until you run the application).
Another idea (a bit similar) is to squash most of the ISPN modules into
infinispan-core and move all "extras" (like Spring support) in separate
repository (infinispan-extras organization?). Such projects would have a
dependency to core. But with this approach we get all the bad things from
coarse grained layout.
The question I'm asking here is - if not Uber Jars, than what? Ideas are
more than welcome!
Thanks
Sebastian
On Thu, Mar 17, 2016 at 12:25 PM, Sanne Grinovero
wrote:
> Since we recently discussed the purpose of the "UberJar"s , I think it
> would be a great exercise to look at this discussion:
> - https://developer.jboss.org/message/953081
>
> and ask ourselves what we could have done to make the first steps of
> this new user less miserable.
>
> Personally I think we could go a long way with some clear
> documentation about our dependencies;
> it's not hard to inject version properties from the pom files into the
> docs, so we could have a little reminder about - for example - which
> version of Hibernate is expected to have the JPACacheStore working.
>
> Also, I think it's clear that uber jars - at least in this case - are
> not even close to the solution, unless you intend to include all of
> Hibernate ORM and transitive dependencies..
>
> Thanks,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160317/3d81f053/attachment.html
From ttarrant at redhat.com Thu Mar 17 08:22:58 2016
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Thu, 17 Mar 2016 13:22:58 +0100
Subject: [infinispan-dev] Practical example of difficulties with
dependencies
In-Reply-To:
References:
Message-ID: <56EAA1A2.4060109@redhat.com>
On 17/03/2016 12:25, Sanne Grinovero wrote:
> Since we recently discussed the purpose of the "UberJar"s , I think it
> would be a great exercise to look at this discussion:
> - https://developer.jboss.org/message/953081
>
> and ask ourselves what we could have done to make the first steps of
> this new user less miserable.
>
> Personally I think we could go a long way with some clear
> documentation about our dependencies;
> it's not hard to inject version properties from the pom files into the
> docs, so we could have a little reminder about - for example - which
> version of Hibernate is expected to have the JPACacheStore working.
>
> Also, I think it's clear that uber jars - at least in this case - are
> not even close to the solution, unless you intend to include all of
> Hibernate ORM and transitive dependencies..
While you are right that we could be clearer, and providing a printout
of mvn dependency:dependency-[list|tree] would help some users, managing
dependencies manually always comes loaded with a number of issues:
duplicate jars, multiple versions, incompatible versions, useless jars,
etc. So, to me, this is an NP complete problem.
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
From sanne at infinispan.org Thu Mar 17 09:32:06 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 17 Mar 2016 13:32:06 +0000
Subject: [infinispan-dev] Practical example of difficulties with
dependencies
In-Reply-To: <56EAA1A2.4060109@redhat.com>
References:
<56EAA1A2.4060109@redhat.com>
Message-ID:
On 17 March 2016 at 12:22, Tristan Tarrant wrote:
> On 17/03/2016 12:25, Sanne Grinovero wrote:
>> Since we recently discussed the purpose of the "UberJar"s , I think it
>> would be a great exercise to look at this discussion:
>> - https://developer.jboss.org/message/953081
>>
>> and ask ourselves what we could have done to make the first steps of
>> this new user less miserable.
>>
>> Personally I think we could go a long way with some clear
>> documentation about our dependencies;
>> it's not hard to inject version properties from the pom files into the
>> docs, so we could have a little reminder about - for example - which
>> version of Hibernate is expected to have the JPACacheStore working.
>>
>> Also, I think it's clear that uber jars - at least in this case - are
>> not even close to the solution, unless you intend to include all of
>> Hibernate ORM and transitive dependencies..
>
> While you are right that we could be clearer, and providing a printout
> of mvn dependency:dependency-[list|tree] would help some users, managing
> dependencies manually always comes loaded with a number of issues:
> duplicate jars, multiple versions, incompatible versions, useless jars,
> etc. So, to me, this is an NP complete problem.
Sure, I don't expect neither yourself or the Infinispan team as a
whole to have a magic solution.
Just wondering what could be "good enough" without being too hard to maintain,
maybe environment/goal driven? For example: to run on Websphere, you
will need this list.
The Wildfly modules are a great way to package; maybe we should
provide (and test) packages for other containers?
BTW Maven helps a bit but it's not providing the magic solution
either, especially
not when deploying on containers and it's not clear what is already
going to be provided,
but also for example it doesn't help with the different versions of
JEE API specifications which we,
i.e. geronimo and Narayana publish different artifact identifiers, and
even within the same project
they will publish a different API version as a different artifact.
N.B. Uber Jars didn't help at all here, and they potentially add even
more confusion or duplication.
Sanne
>
> Tristan
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
From eric.wittmann at redhat.com Thu Mar 17 13:06:04 2016
From: eric.wittmann at redhat.com (Eric Wittmann)
Date: Thu, 17 Mar 2016 13:06:04 -0400
Subject: [infinispan-dev] Distributed Counter Discussion
Message-ID: <56EAE3FC.6030401@redhat.com>
Greetings. Apologies for coming in a bit late on this conversation.
Tristan pointed me to it a couple of days ago and unfortunately I'm just
now getting time to reply.
I can try to quickly give an overview of apiman's (JBoss API Management
Gateway) requirements.
What we're trying to do is implement support for Limiting policies:
* Rate Limiting/Throttling (e.g. limit of 100 requests per second)
* Quotas (e.g. limit of 100,000,000 requests per month)
* Transfer Quotas (e.g. limit of 2.5GB of data downloaded per day)
We will need to support multiple backing implementations of the Rate
Limiter, and we're trying to get Infinispan to be one of those
implementations.
In no particular order, we would need the following characteristics:
- Can be "squishy" for quotas and transfer quotas: If you
get 100,001,017 requests that's OK
- Strict would be cool as an option: Hard-fail when the
counter reaches the limit - no chance it will go over.
- Lots of individual counters: users may publish 100s of
APIs to the Gateway, and each API may be consumed by
100s or 1000s of users/client. Depending on configuration
of the policy, *each* user/client has a separate limit.
- Counters need to be created dynamically: users can
add APIs via the Management UI, configure them to add
policies (e.g. a Quota policy) and then publish them to
a running Gateway, at which point end users can invoke
the API through the Gateway, which will use a counter
to enforce the Quota.
- Counter values reset at the end of a time boundary: for
example, at the end of the month the counter value for
the example quota above would reset to 0.
- Don't care (right now) what the counter value is: at the
moment we simply need to know if some counter max value
has been reached. In the future we would like to know
when a max value is being "approached" (e.g. to notify a
user)
- Should be persistent: it would not be ideal for e.g. per-
month quota values to be lost on server restart.
That's all the high level requirements I can think of off the top of my
head, and after reading all of the current messages in this thread. :)
-Eric
From bban at redhat.com Fri Mar 18 02:32:02 2016
From: bban at redhat.com (Bela Ban)
Date: Fri, 18 Mar 2016 07:32:02 +0100
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56EAE3FC.6030401@redhat.com>
References: <56EAE3FC.6030401@redhat.com>
Message-ID: <56EBA0E2.7070106@redhat.com>
Stupid question: whay do you need a distributed counter for this? Is the
service you're monitoring replicated in a cluster?
On 17/03/16 18:06, Eric Wittmann wrote:
> Greetings. Apologies for coming in a bit late on this conversation.
> Tristan pointed me to it a couple of days ago and unfortunately I'm just
> now getting time to reply.
>
> I can try to quickly give an overview of apiman's (JBoss API Management
> Gateway) requirements.
>
> What we're trying to do is implement support for Limiting policies:
>
> * Rate Limiting/Throttling (e.g. limit of 100 requests per second)
> * Quotas (e.g. limit of 100,000,000 requests per month)
> * Transfer Quotas (e.g. limit of 2.5GB of data downloaded per day)
>
> We will need to support multiple backing implementations of the Rate
> Limiter, and we're trying to get Infinispan to be one of those
> implementations.
>
> In no particular order, we would need the following characteristics:
>
> - Can be "squishy" for quotas and transfer quotas: If you
> get 100,001,017 requests that's OK
> - Strict would be cool as an option: Hard-fail when the
> counter reaches the limit - no chance it will go over.
> - Lots of individual counters: users may publish 100s of
> APIs to the Gateway, and each API may be consumed by
> 100s or 1000s of users/client. Depending on configuration
> of the policy, *each* user/client has a separate limit.
> - Counters need to be created dynamically: users can
> add APIs via the Management UI, configure them to add
> policies (e.g. a Quota policy) and then publish them to
> a running Gateway, at which point end users can invoke
> the API through the Gateway, which will use a counter
> to enforce the Quota.
> - Counter values reset at the end of a time boundary: for
> example, at the end of the month the counter value for
> the example quota above would reset to 0.
> - Don't care (right now) what the counter value is: at the
> moment we simply need to know if some counter max value
> has been reached. In the future we would like to know
> when a max value is being "approached" (e.g. to notify a
> user)
> - Should be persistent: it would not be ideal for e.g. per-
> month quota values to be lost on server restart.
>
> That's all the high level requirements I can think of off the top of my
> head, and after reading all of the current messages in this thread. :)
>
> -Eric
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
--
Bela Ban, JGroups lead (http://www.jgroups.org)
From eric.wittmann at redhat.com Fri Mar 18 09:19:11 2016
From: eric.wittmann at redhat.com (Eric Wittmann)
Date: Fri, 18 Mar 2016 09:19:11 -0400
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56EBA0E2.7070106@redhat.com>
References: <56EAE3FC.6030401@redhat.com> <56EBA0E2.7070106@redhat.com>
Message-ID: <56EC004F.7040101@redhat.com>
Yes, precisely. The API Gateway itself is clustered. It services a
large volume of inbound traffic which it reverse-proxies to appropriate
back-end APIs after applying policies such as security, rate limiting,
caching, etc.
-Eric
On 3/18/2016 2:32 AM, Bela Ban wrote:
> Stupid question: whay do you need a distributed counter for this? Is the
> service you're monitoring replicated in a cluster?
>
> On 17/03/16 18:06, Eric Wittmann wrote:
>> Greetings. Apologies for coming in a bit late on this conversation.
>> Tristan pointed me to it a couple of days ago and unfortunately I'm just
>> now getting time to reply.
>>
>> I can try to quickly give an overview of apiman's (JBoss API Management
>> Gateway) requirements.
>>
>> What we're trying to do is implement support for Limiting policies:
>>
>> * Rate Limiting/Throttling (e.g. limit of 100 requests per second)
>> * Quotas (e.g. limit of 100,000,000 requests per month)
>> * Transfer Quotas (e.g. limit of 2.5GB of data downloaded per day)
>>
>> We will need to support multiple backing implementations of the Rate
>> Limiter, and we're trying to get Infinispan to be one of those
>> implementations.
>>
>> In no particular order, we would need the following characteristics:
>>
>> - Can be "squishy" for quotas and transfer quotas: If you
>> get 100,001,017 requests that's OK
>> - Strict would be cool as an option: Hard-fail when the
>> counter reaches the limit - no chance it will go over.
>> - Lots of individual counters: users may publish 100s of
>> APIs to the Gateway, and each API may be consumed by
>> 100s or 1000s of users/client. Depending on configuration
>> of the policy, *each* user/client has a separate limit.
>> - Counters need to be created dynamically: users can
>> add APIs via the Management UI, configure them to add
>> policies (e.g. a Quota policy) and then publish them to
>> a running Gateway, at which point end users can invoke
>> the API through the Gateway, which will use a counter
>> to enforce the Quota.
>> - Counter values reset at the end of a time boundary: for
>> example, at the end of the month the counter value for
>> the example quota above would reset to 0.
>> - Don't care (right now) what the counter value is: at the
>> moment we simply need to know if some counter max value
>> has been reached. In the future we would like to know
>> when a max value is being "approached" (e.g. to notify a
>> user)
>> - Should be persistent: it would not be ideal for e.g. per-
>> month quota values to be lost on server restart.
>>
>> That's all the high level requirements I can think of off the top of my
>> head, and after reading all of the current messages in this thread. :)
>>
>> -Eric
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
From bban at redhat.com Fri Mar 18 09:31:11 2016
From: bban at redhat.com (Bela Ban)
Date: Fri, 18 Mar 2016 14:31:11 +0100
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56EC004F.7040101@redhat.com>
References: <56EAE3FC.6030401@redhat.com> <56EBA0E2.7070106@redhat.com>
<56EC004F.7040101@redhat.com>
Message-ID: <56EC031F.10809@redhat.com>
So actually you don't care if you have multiple counters in case of a
network split, but you do care that the numbers of different counters
get reconciled when a network partition heals.
Example
- C1: 1000
- Network split: C1: 1000, C2: 1000
- Different clients update counters on both sides of the partition: C1:
1500 C2: 1600
- Network split disappears, reconciling C1 to 2100: 1000 +500 +600. This
means the 500 added to C1 should have been added to C2 as well, and the
600 to C2 should have been added to C1
If such a behavior would be acceptable, then we could do without CP and
live with AP
On 18/03/16 14:19, Eric Wittmann wrote:
> Yes, precisely. The API Gateway itself is clustered. It services a
> large volume of inbound traffic which it reverse-proxies to appropriate
> back-end APIs after applying policies such as security, rate limiting,
> caching, etc.
>
> -Eric
>
> On 3/18/2016 2:32 AM, Bela Ban wrote:
>> Stupid question: whay do you need a distributed counter for this? Is the
>> service you're monitoring replicated in a cluster?
>>
>> On 17/03/16 18:06, Eric Wittmann wrote:
>>> Greetings. Apologies for coming in a bit late on this conversation.
>>> Tristan pointed me to it a couple of days ago and unfortunately I'm just
>>> now getting time to reply.
>>>
>>> I can try to quickly give an overview of apiman's (JBoss API Management
>>> Gateway) requirements.
>>>
>>> What we're trying to do is implement support for Limiting policies:
>>>
>>> * Rate Limiting/Throttling (e.g. limit of 100 requests per second)
>>> * Quotas (e.g. limit of 100,000,000 requests per month)
>>> * Transfer Quotas (e.g. limit of 2.5GB of data downloaded per day)
>>>
>>> We will need to support multiple backing implementations of the Rate
>>> Limiter, and we're trying to get Infinispan to be one of those
>>> implementations.
>>>
>>> In no particular order, we would need the following characteristics:
>>>
>>> - Can be "squishy" for quotas and transfer quotas: If you
>>> get 100,001,017 requests that's OK
>>> - Strict would be cool as an option: Hard-fail when the
>>> counter reaches the limit - no chance it will go over.
>>> - Lots of individual counters: users may publish 100s of
>>> APIs to the Gateway, and each API may be consumed by
>>> 100s or 1000s of users/client. Depending on configuration
>>> of the policy, *each* user/client has a separate limit.
>>> - Counters need to be created dynamically: users can
>>> add APIs via the Management UI, configure them to add
>>> policies (e.g. a Quota policy) and then publish them to
>>> a running Gateway, at which point end users can invoke
>>> the API through the Gateway, which will use a counter
>>> to enforce the Quota.
>>> - Counter values reset at the end of a time boundary: for
>>> example, at the end of the month the counter value for
>>> the example quota above would reset to 0.
>>> - Don't care (right now) what the counter value is: at the
>>> moment we simply need to know if some counter max value
>>> has been reached. In the future we would like to know
>>> when a max value is being "approached" (e.g. to notify a
>>> user)
>>> - Should be persistent: it would not be ideal for e.g. per-
>>> month quota values to be lost on server restart.
>>>
>>> That's all the high level requirements I can think of off the top of my
>>> head, and after reading all of the current messages in this thread. :)
>>>
>>> -Eric
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
--
Bela Ban, JGroups lead (http://www.jgroups.org)
From eric.wittmann at redhat.com Fri Mar 18 10:19:11 2016
From: eric.wittmann at redhat.com (Eric Wittmann)
Date: Fri, 18 Mar 2016 10:19:11 -0400
Subject: [infinispan-dev] Distributed Counter Discussion
In-Reply-To: <56EC031F.10809@redhat.com>
References: <56EAE3FC.6030401@redhat.com> <56EBA0E2.7070106@redhat.com>
<56EC004F.7040101@redhat.com> <56EC031F.10809@redhat.com>
Message-ID: <56EC0E5F.1020103@redhat.com>
Agreed. :)
On 3/18/2016 9:31 AM, Bela Ban wrote:
> So actually you don't care if you have multiple counters in case of a
> network split, but you do care that the numbers of different counters
> get reconciled when a network partition heals.
>
> Example
> - C1: 1000
> - Network split: C1: 1000, C2: 1000
> - Different clients update counters on both sides of the partition: C1:
> 1500 C2: 1600
> - Network split disappears, reconciling C1 to 2100: 1000 +500 +600. This
> means the 500 added to C1 should have been added to C2 as well, and the
> 600 to C2 should have been added to C1
>
> If such a behavior would be acceptable, then we could do without CP and
> live with AP
>
> On 18/03/16 14:19, Eric Wittmann wrote:
>> Yes, precisely. The API Gateway itself is clustered. It services a
>> large volume of inbound traffic which it reverse-proxies to appropriate
>> back-end APIs after applying policies such as security, rate limiting,
>> caching, etc.
>>
>> -Eric
>>
>> On 3/18/2016 2:32 AM, Bela Ban wrote:
>>> Stupid question: whay do you need a distributed counter for this? Is the
>>> service you're monitoring replicated in a cluster?
>>>
>>> On 17/03/16 18:06, Eric Wittmann wrote:
>>>> Greetings. Apologies for coming in a bit late on this conversation.
>>>> Tristan pointed me to it a couple of days ago and unfortunately I'm just
>>>> now getting time to reply.
>>>>
>>>> I can try to quickly give an overview of apiman's (JBoss API Management
>>>> Gateway) requirements.
>>>>
>>>> What we're trying to do is implement support for Limiting policies:
>>>>
>>>> * Rate Limiting/Throttling (e.g. limit of 100 requests per second)
>>>> * Quotas (e.g. limit of 100,000,000 requests per month)
>>>> * Transfer Quotas (e.g. limit of 2.5GB of data downloaded per day)
>>>>
>>>> We will need to support multiple backing implementations of the Rate
>>>> Limiter, and we're trying to get Infinispan to be one of those
>>>> implementations.
>>>>
>>>> In no particular order, we would need the following characteristics:
>>>>
>>>> - Can be "squishy" for quotas and transfer quotas: If you
>>>> get 100,001,017 requests that's OK
>>>> - Strict would be cool as an option: Hard-fail when the
>>>> counter reaches the limit - no chance it will go over.
>>>> - Lots of individual counters: users may publish 100s of
>>>> APIs to the Gateway, and each API may be consumed by
>>>> 100s or 1000s of users/client. Depending on configuration
>>>> of the policy, *each* user/client has a separate limit.
>>>> - Counters need to be created dynamically: users can
>>>> add APIs via the Management UI, configure them to add
>>>> policies (e.g. a Quota policy) and then publish them to
>>>> a running Gateway, at which point end users can invoke
>>>> the API through the Gateway, which will use a counter
>>>> to enforce the Quota.
>>>> - Counter values reset at the end of a time boundary: for
>>>> example, at the end of the month the counter value for
>>>> the example quota above would reset to 0.
>>>> - Don't care (right now) what the counter value is: at the
>>>> moment we simply need to know if some counter max value
>>>> has been reached. In the future we would like to know
>>>> when a max value is being "approached" (e.g. to notify a
>>>> user)
>>>> - Should be persistent: it would not be ideal for e.g. per-
>>>> month quota values to be lost on server restart.
>>>>
>>>> That's all the high level requirements I can think of off the top of my
>>>> head, and after reading all of the current messages in this thread. :)
>>>>
>>>> -Eric
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
From rory.odonnell at oracle.com Sat Mar 19 03:40:02 2016
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Sat, 19 Mar 2016 13:10:02 +0530
Subject: [infinispan-dev] Early Access builds of JDK 9 b110 & JDK 9 with
Project Jigsaw b110 (#4664) are available on java.net
Message-ID: <56ED0252.2090307@oracle.com>
Hi Galder,
Early Access b110