From galder at redhat.com  Wed Aug  3 03:22:16 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 3 Aug 2016 09:22:16 +0200
Subject: [infinispan-dev] Video for DevNation talk on JBoss Data Grid,
	Node.js and Elm available now
Message-ID: <6B11A2D1-07C0-4013-983B-0E8FD524A57F@redhat.com>

Hi all,

The video of the talk Dyvia and I gave on building reactive apps with JBoss Data Grid 7, Node.js and Elm that we gave at DevNation is now available online: http://developers.redhat.com/video/youtube/ebUbzrpCuTA/

Enjoy!
--
Galder Zamarre?o
Infinispan, Red Hat



From galder at redhat.com  Wed Aug  3 03:49:07 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 3 Aug 2016 09:49:07 +0200
Subject: [infinispan-dev] Video for DevNation talk on auto-scaling
	Infinispan with Kubernetes
Message-ID: <DA32D5D5-02E1-49AA-B606-9C85D03A59DA@redhat.com>

Hi all,

More amazing videos coming out of DevNation 2016, this time with the amazing Ray Tsang showing how to auto-scale Infinispan with Kubernetes:
http://developers.redhat.com/video/youtube/E4ACZ6_WDyo/

Enjoy!
--
Galder Zamarre?o
Infinispan, Red Hat



From slaskawi at redhat.com  Wed Aug  3 05:08:43 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 3 Aug 2016 11:08:43 +0200
Subject: [infinispan-dev] Video for DevNation talk on auto-scaling
 Infinispan with Kubernetes
In-Reply-To: <DA32D5D5-02E1-49AA-B606-9C85D03A59DA@redhat.com>
References: <DA32D5D5-02E1-49AA-B606-9C85D03A59DA@redhat.com>
Message-ID: <CAA9R9O5UYyebdJOoT-sbT31J0BJeWCaSvRUbasGpt5wvyccwsw@mail.gmail.com>

Thanks Galder for this finding!

Just a small comment on things which are not obvious - Ray created his own
custom autoscaler implementation. Both OpenShift and Kubernetes have
support for so called Custom Metrics [1][2] which you can use for
autoscaling.

However, trying to scale based on the memory consumption is a tricky
business [3][4]. I've been talking with Andrew about this and he says that
settings `-Xms` to some low value and `-Xmx` reasonably high might do the
trick. When you add a node to the cluster (assuming only DIST caches) the
total amount of used memory in the Pods will (possibly) drop after some
time. This however might not work for CMS GC.

Anyway - those are the things which are on my plate for this and next week.
Once I have some results I would be glad to share them with you.

And as always - any help with this would be highly appreciated.

Thanks
Sebastian

[1]
https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/custom-metrics.md
[2] https://github.com/openshift/origin-metrics
[3]
https://developerblog.redhat.com/2014/07/15/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-1/
[4]
https://developerblog.redhat.com/2014/07/22/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-2/


On Wed, Aug 3, 2016 at 9:49 AM, Galder Zamarre?o <galder at redhat.com> wrote:

> Hi all,
>
> More amazing videos coming out of DevNation 2016, this time with the
> amazing Ray Tsang showing how to auto-scale Infinispan with Kubernetes:
> http://developers.redhat.com/video/youtube/E4ACZ6_WDyo/
>
> Enjoy!
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160803/31088984/attachment.html 

From slaskawi at redhat.com  Wed Aug  3 05:13:58 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 3 Aug 2016 11:13:58 +0200
Subject: [infinispan-dev] Video for DevNation talk on JBoss Data Grid,
 Node.js and Elm available now
In-Reply-To: <6B11A2D1-07C0-4013-983B-0E8FD524A57F@redhat.com>
References: <6B11A2D1-07C0-4013-983B-0E8FD524A57F@redhat.com>
Message-ID: <CAA9R9O5Kbb=B=ocPb=9_4fOjkJXQH4bdbzGsc67nj++vrRx2Cg@mail.gmail.com>

Awesome! Good stuff!

On Wed, Aug 3, 2016 at 9:22 AM, Galder Zamarre?o <galder at redhat.com> wrote:

> Hi all,
>
> The video of the talk Dyvia and I gave on building reactive apps with
> JBoss Data Grid 7, Node.js and Elm that we gave at DevNation is now
> available online: http://developers.redhat.com/video/youtube/ebUbzrpCuTA/
>
> Enjoy!
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160803/fe95688a/attachment.html 

From galder at redhat.com  Wed Aug  3 05:16:52 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 3 Aug 2016 11:16:52 +0200
Subject: [infinispan-dev] Video for DevNation talk on auto-scaling
	Infinispan with Kubernetes
In-Reply-To: <CAA9R9O5UYyebdJOoT-sbT31J0BJeWCaSvRUbasGpt5wvyccwsw@mail.gmail.com>
References: <DA32D5D5-02E1-49AA-B606-9C85D03A59DA@redhat.com>
	<CAA9R9O5UYyebdJOoT-sbT31J0BJeWCaSvRUbasGpt5wvyccwsw@mail.gmail.com>
Message-ID: <0A08602A-85C9-4CB6-862A-758B6464B101@redhat.com>

Hey Sebastian,

Yeah, as Ray rightly says in the video, his approach is based on old Kubernetes versions, and so the current status might be different.

What about you write up a blog with a link to the video and you explain the best practices or more recent approaches that we're using these days? A comparison might be handy for our users to better understand the concepts.

WDYT?

Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat

> On 3 Aug 2016, at 11:08, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> 
> Thanks Galder for this finding!
> 
> Just a small comment on things which are not obvious - Ray created his own custom autoscaler implementation. Both OpenShift and Kubernetes have support for so called Custom Metrics [1][2] which you can use for autoscaling. 
> 
> However, trying to scale based on the memory consumption is a tricky business [3][4]. I've been talking with Andrew about this and he says that settings `-Xms` to some low value and `-Xmx` reasonably high might do the trick. When you add a node to the cluster (assuming only DIST caches) the total amount of used memory in the Pods will (possibly) drop after some time. This however might not work for CMS GC. 
> 
> Anyway - those are the things which are on my plate for this and next week. Once I have some results I would be glad to share them with you. 
> 
> And as always - any help with this would be highly appreciated.
> 
> Thanks
> Sebastian
> 
> [1] https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/custom-metrics.md
> [2] https://github.com/openshift/origin-metrics
> [3] https://developerblog.redhat.com/2014/07/15/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-1/
> [4] https://developerblog.redhat.com/2014/07/22/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-2/
> 
> 
> On Wed, Aug 3, 2016 at 9:49 AM, Galder Zamarre?o <galder at redhat.com> wrote:
> Hi all,
> 
> More amazing videos coming out of DevNation 2016, this time with the amazing Ray Tsang showing how to auto-scale Infinispan with Kubernetes:
> http://developers.redhat.com/video/youtube/E4ACZ6_WDyo/
> 
> Enjoy!
> --
> Galder Zamarre?o
> Infinispan, Red Hat
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From slaskawi at redhat.com  Wed Aug  3 05:26:59 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 3 Aug 2016 11:26:59 +0200
Subject: [infinispan-dev] Video for DevNation talk on auto-scaling
 Infinispan with Kubernetes
In-Reply-To: <0A08602A-85C9-4CB6-862A-758B6464B101@redhat.com>
References: <DA32D5D5-02E1-49AA-B606-9C85D03A59DA@redhat.com>
	<CAA9R9O5UYyebdJOoT-sbT31J0BJeWCaSvRUbasGpt5wvyccwsw@mail.gmail.com>
	<0A08602A-85C9-4CB6-862A-758B6464B101@redhat.com>
Message-ID: <CAA9R9O5YR0pzy3_xV-bFbH3301hE2Nr=vtdJmZEsmcPuy9KVJw@mail.gmail.com>

That's a good idea, but let me first figure out how to do it properly (and
make sure it works as I expect)...

On Wed, Aug 3, 2016 at 11:16 AM, Galder Zamarre?o <galder at redhat.com> wrote:

> Hey Sebastian,
>
> Yeah, as Ray rightly says in the video, his approach is based on old
> Kubernetes versions, and so the current status might be different.
>
> What about you write up a blog with a link to the video and you explain
> the best practices or more recent approaches that we're using these days? A
> comparison might be handy for our users to better understand the concepts.
>
> WDYT?
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
> > On 3 Aug 2016, at 11:08, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
> >
> > Thanks Galder for this finding!
> >
> > Just a small comment on things which are not obvious - Ray created his
> own custom autoscaler implementation. Both OpenShift and Kubernetes have
> support for so called Custom Metrics [1][2] which you can use for
> autoscaling.
> >
> > However, trying to scale based on the memory consumption is a tricky
> business [3][4]. I've been talking with Andrew about this and he says that
> settings `-Xms` to some low value and `-Xmx` reasonably high might do the
> trick. When you add a node to the cluster (assuming only DIST caches) the
> total amount of used memory in the Pods will (possibly) drop after some
> time. This however might not work for CMS GC.
> >
> > Anyway - those are the things which are on my plate for this and next
> week. Once I have some results I would be glad to share them with you.
> >
> > And as always - any help with this would be highly appreciated.
> >
> > Thanks
> > Sebastian
> >
> > [1]
> https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/custom-metrics.md
> > [2] https://github.com/openshift/origin-metrics
> > [3]
> https://developerblog.redhat.com/2014/07/15/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-1/
> > [4]
> https://developerblog.redhat.com/2014/07/22/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-2/
> >
> >
> > On Wed, Aug 3, 2016 at 9:49 AM, Galder Zamarre?o <galder at redhat.com>
> wrote:
> > Hi all,
> >
> > More amazing videos coming out of DevNation 2016, this time with the
> amazing Ray Tsang showing how to auto-scale Infinispan with Kubernetes:
> > http://developers.redhat.com/video/youtube/E4ACZ6_WDyo/
> >
> > Enjoy!
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160803/13d4a7a2/attachment-0001.html 

From galder at redhat.com  Wed Aug  3 05:57:17 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 3 Aug 2016 11:57:17 +0200
Subject: [infinispan-dev] Sequential interceptors API
In-Reply-To: <576AB428.10203@redhat.com>
References: <57580E54.10703@redhat.com>
	<8BD560BF-0F3E-4CC8-843C-EB54F5410877@redhat.com>
	<576AB428.10203@redhat.com>
Message-ID: <79B0BF43-E05B-44CF-BC9C-2E534B55C5BB@redhat.com>

Hey Radim,

Sorry for the delay having a look at this (Summit/DevNation then vacation).

I agree with you that the way you're passing down the handler down the stack and accross lambdas is a bit counter-productive, because it means the lambda has to capture the handler, which in turn means that each lambda execution will result in a lambda instantiation.

Looking at AsyncInvocationContext, methods like this feel a bit of an anti-pattern:

CompletableFuture<Void> forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler)

The problem here is that it mixes the two main ways of dealing with async computation:

1. Callback handlers [1].
2. Promises (~ CompletableFuture) [2].

Normally, you either use one or the other, but here both are in use because the method returns a CompletableFuture (e.g. like a JS/Scala promise), and then is taking a callback handler as well.

If you go down the path of promises, the normal thing would be:

CompletableFuture<Void> forkInvocation(VisitableCommand newCommand)

And then you chain (e.g. thenCompose or similar) the return function when the fork has completed. I assume that going down this route causes unnecessary allocations?

If we're worried about allocations, I wonder whether a pure handler based approach might work better? It might not be as user-friendly but if we want to avoid CompletableFuture issues internally, then maybe that's the way? e.g.

void forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler);

While at Summit/DevNation I had a chat with Clement from Vert.x team who had some reservations about CompletableFutures themselves. I don't remember the details right now (that's what vacation does to you!) but I've pinged him to find out more.

In general though, there are multiple issues associated with the handler approach, including:

* Lack of central error handling logic.
* Callback readibility, when multiple handlers are specified in one go [3].

Cheers,

[1] http://www.servicedesignpatterns.com/WebServiceInfrastructures/AsyncResponseHandler
[2] http://exploringjs.com/es6/ch_promises.html
[3] http://callbackhell.com
--
Galder Zamarre?o
Infinispan, Red Hat

> On 22 Jun 2016, at 17:52, Radim Vansa <rvansa at redhat.com> wrote:
> 
> Yes [1]. The longest chaining of operations I had in [2], basically 
> during ST I have to load a value locally*, perform a unicast/broadcast 
> to read different value and then execute the original one.
> 
> * I shouldn't load it just from DC, as it could be in cache store, too; 
> though without persistence (which I don't deal with properly in 
> scattered cache yet) it would be more efficient to do the DC lookup 
> directly.
> 
> Radim
> 
> [1] https://github.com/rvansa/infinispan/tree/ISPN-6645
> [2] 
> https://github.com/rvansa/infinispan/blob/ISPN-6645/core/src/main/java/org/infinispan/interceptors/impl/PrefetchInvalidationInterceptor.java
> 
> On 06/17/2016 10:52 AM, Galder Zamarre?o wrote:
>> Radim, do you have a branch where you have been trying these things out? I'd like to play with what you're trying to do.
>> 
>> Cheers,
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>> 
>>> On 8 Jun 2016, at 14:23, Radim Vansa <rvansa at redhat.com> wrote:
>>> 
>>> Hi,
>>> 
>>> I would like to encourage you to play with the (relatively) new API for
>>> sequential interceptors, and voice your comments - especially you corish
>>> devs, and Galder who has much experience with async invocations and that
>>> kind of stuff from JS-world.
>>> 
>>> I am now trying to use the asynchronous methods only (the
>>> forkInvocationSync() is only temporary helper!); Dan has made it this
>>> way as he wanted to avoid unnecessary allocations, and I welcome this
>>> GC-awareness, but regrettably I find it rather hard to use, due to its
>>> handler-style nature. For the simplest style interceptors (do this,
>>> invoke next interceptor, and process result) it's fine, but when you
>>> want to do something like:
>>> 
>>> visitFoo(cmd) {
>>>   Object x = null;
>>>   if (/* ... */) {
>>>       x = invoke(new OtherCommand());
>>>   }
>>>   invoke(new DifferentCommand(x));
>>>   Object retval = invoke(cmd);
>>>   return wrap(retval);
>>> }
>>> 
>>> I find myself passing handlers deep down. There is allocation cost for
>>> closures, so API that does not allocate CompletableFutures does not pay off.
>>> 
>>> I don't say that I could improve it (I have directed my comments to Dan
>>> on IRC when I had something in particular), I just say that this is very
>>> important API for further Infinispan development and everyone should pay
>>> attention before it gets final.
>>> 
>>> So please, play with it, and show your opinion.
>>> 
>>> Radim
>>> 
>>> -- 
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> -- 
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From rvansa at redhat.com  Wed Aug  3 06:49:16 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 3 Aug 2016 12:49:16 +0200
Subject: [infinispan-dev] Sequential interceptors API
In-Reply-To: <79B0BF43-E05B-44CF-BC9C-2E534B55C5BB@redhat.com>
References: <57580E54.10703@redhat.com>
	<8BD560BF-0F3E-4CC8-843C-EB54F5410877@redhat.com>
	<576AB428.10203@redhat.com>
	<79B0BF43-E05B-44CF-BC9C-2E534B55C5BB@redhat.com>
Message-ID: <57A1CC2C.2010501@redhat.com>

I think that Dan had another branch with more FSM-like API, but I can't 
find it :-/

R.

On 08/03/2016 11:57 AM, Galder Zamarre?o wrote:
> Hey Radim,
>
> Sorry for the delay having a look at this (Summit/DevNation then vacation).
>
> I agree with you that the way you're passing down the handler down the stack and accross lambdas is a bit counter-productive, because it means the lambda has to capture the handler, which in turn means that each lambda execution will result in a lambda instantiation.
>
> Looking at AsyncInvocationContext, methods like this feel a bit of an anti-pattern:
>
> CompletableFuture<Void> forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler)
>
> The problem here is that it mixes the two main ways of dealing with async computation:
>
> 1. Callback handlers [1].
> 2. Promises (~ CompletableFuture) [2].
>
> Normally, you either use one or the other, but here both are in use because the method returns a CompletableFuture (e.g. like a JS/Scala promise), and then is taking a callback handler as well.
>
> If you go down the path of promises, the normal thing would be:
>
> CompletableFuture<Void> forkInvocation(VisitableCommand newCommand)
>
> And then you chain (e.g. thenCompose or similar) the return function when the fork has completed. I assume that going down this route causes unnecessary allocations?
>
> If we're worried about allocations, I wonder whether a pure handler based approach might work better? It might not be as user-friendly but if we want to avoid CompletableFuture issues internally, then maybe that's the way? e.g.
>
> void forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler);
>
> While at Summit/DevNation I had a chat with Clement from Vert.x team who had some reservations about CompletableFutures themselves. I don't remember the details right now (that's what vacation does to you!) but I've pinged him to find out more.
>
> In general though, there are multiple issues associated with the handler approach, including:
>
> * Lack of central error handling logic.
> * Callback readibility, when multiple handlers are specified in one go [3].
>
> Cheers,
>
> [1] http://www.servicedesignpatterns.com/WebServiceInfrastructures/AsyncResponseHandler
> [2] http://exploringjs.com/es6/ch_promises.html
> [3] http://callbackhell.com
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 22 Jun 2016, at 17:52, Radim Vansa <rvansa at redhat.com> wrote:
>>
>> Yes [1]. The longest chaining of operations I had in [2], basically
>> during ST I have to load a value locally*, perform a unicast/broadcast
>> to read different value and then execute the original one.
>>
>> * I shouldn't load it just from DC, as it could be in cache store, too;
>> though without persistence (which I don't deal with properly in
>> scattered cache yet) it would be more efficient to do the DC lookup
>> directly.
>>
>> Radim
>>
>> [1] https://github.com/rvansa/infinispan/tree/ISPN-6645
>> [2]
>> https://github.com/rvansa/infinispan/blob/ISPN-6645/core/src/main/java/org/infinispan/interceptors/impl/PrefetchInvalidationInterceptor.java
>>
>> On 06/17/2016 10:52 AM, Galder Zamarre?o wrote:
>>> Radim, do you have a branch where you have been trying these things out? I'd like to play with what you're trying to do.
>>>
>>> Cheers,
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>>> On 8 Jun 2016, at 14:23, Radim Vansa <rvansa at redhat.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I would like to encourage you to play with the (relatively) new API for
>>>> sequential interceptors, and voice your comments - especially you corish
>>>> devs, and Galder who has much experience with async invocations and that
>>>> kind of stuff from JS-world.
>>>>
>>>> I am now trying to use the asynchronous methods only (the
>>>> forkInvocationSync() is only temporary helper!); Dan has made it this
>>>> way as he wanted to avoid unnecessary allocations, and I welcome this
>>>> GC-awareness, but regrettably I find it rather hard to use, due to its
>>>> handler-style nature. For the simplest style interceptors (do this,
>>>> invoke next interceptor, and process result) it's fine, but when you
>>>> want to do something like:
>>>>
>>>> visitFoo(cmd) {
>>>>    Object x = null;
>>>>    if (/* ... */) {
>>>>        x = invoke(new OtherCommand());
>>>>    }
>>>>    invoke(new DifferentCommand(x));
>>>>    Object retval = invoke(cmd);
>>>>    return wrap(retval);
>>>> }
>>>>
>>>> I find myself passing handlers deep down. There is allocation cost for
>>>> closures, so API that does not allocate CompletableFutures does not pay off.
>>>>
>>>> I don't say that I could improve it (I have directed my comments to Dan
>>>> on IRC when I had something in particular), I just say that this is very
>>>> important API for further Infinispan development and everyone should pay
>>>> attention before it gets final.
>>>>
>>>> So please, play with it, and show your opinion.
>>>>
>>>> Radim
>>>>
>>>> -- 
>>>> Radim Vansa <rvansa at redhat.com>
>>>> JBoss Performance Team
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> -- 
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Wed Aug  3 07:54:41 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Wed, 3 Aug 2016 14:54:41 +0300
Subject: [infinispan-dev] Sequential interceptors API
In-Reply-To: <57A1CC2C.2010501@redhat.com>
References: <57580E54.10703@redhat.com>
	<8BD560BF-0F3E-4CC8-843C-EB54F5410877@redhat.com>
	<576AB428.10203@redhat.com>
	<79B0BF43-E05B-44CF-BC9C-2E534B55C5BB@redhat.com>
	<57A1CC2C.2010501@redhat.com>
Message-ID: <CA+nfvwSwb2=DAO7wbkGVv+5DTA4GjGo=afQenw3coMFHYpTOjA@mail.gmail.com>

Yeah, I just fixed (almost all of) the test failures this morning and
I pushed it here:

https://github.com/infinispan/infinispan/compare/master...danberindei:ISPN-5467_CompletableFuture-like_API?expand=1


Galder, I'm not sure I understand the part about mixing promises and
callbacks... almost all CompletableFuture methods take a callback
parameter, so I'm not sure how one could avoid mixing them. That being
said, the API as it is in master now tends to encourage nesting of
callbacks, and that's what I'm trying to change.

My new approach has a promise-like interface called
IntermediateInvocationStage (for now), which works like a promise on
the interceptors not yet executed. It has various methods you can then
use to add callbacks to be executed after those interceptors (and any
callbacks they installed).

To get an IntermediateInvocationStage, you need to call a method like
invokeNext(ctx, command) on BaseAsyncInterceptor/InterceptorStage. I'm
not yet sure whether exposing InterceptorStage to the interceptor is
really necessary.

Both the interceptor and the callbacks then return an InvocationStage,
which is implemented by the same classes that implement
IntermediateInvocationStage, but doesn't allow adding more callbacks.
This is mainly so that I can have returnWith(returnValue) instead of
returnWith(ctx, command, returnValue).

For the actual asynchronous work, I have 2 goAsync methods in
InterceptorStage, because I think having an explicit callback can be
slightly more efficient - I'm not 100% sure about this either. I used
to have a waitFor(CompletionStage<?>) method that returned an
InterceptorStage, but I decided it was too confusing to have
InterceptorStage behave like a promise as well.

IntermediateInvocationStage also allows "asynchronous" callbacks with
composeAsync, but it seems a bit weird to have a single callback
handle both on an InvocationStage and on a CompletionStage. So I'm
thinking of removing that, and forcing the interceptors to use
compose+goAsync instead. That probably means callback nesting again,
so I need to check how it would affect the L1 interceptors.

About performance: I only tested non-tx local reads so far, but in
that case I'm only getting one real allocation in CallInterceptor,
which gets reused by all the other interceptors. For this scenario all
the callbacks are cached in the interceptor fields, of course, so it's
not HotSpot doing scalar replacement for the callbacks - although in
theory that should work as well. I need to do a lot more testing on
this...


Cheers
Dan



On Wed, Aug 3, 2016 at 1:49 PM, Radim Vansa <rvansa at redhat.com> wrote:
> I think that Dan had another branch with more FSM-like API, but I can't
> find it :-/
>
> R.
>
> On 08/03/2016 11:57 AM, Galder Zamarre?o wrote:
>> Hey Radim,
>>
>> Sorry for the delay having a look at this (Summit/DevNation then vacation).
>>
>> I agree with you that the way you're passing down the handler down the stack and accross lambdas is a bit counter-productive, because it means the lambda has to capture the handler, which in turn means that each lambda execution will result in a lambda instantiation.
>>
>> Looking at AsyncInvocationContext, methods like this feel a bit of an anti-pattern:
>>
>> CompletableFuture<Void> forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler)
>>
>> The problem here is that it mixes the two main ways of dealing with async computation:
>>
>> 1. Callback handlers [1].
>> 2. Promises (~ CompletableFuture) [2].
>>
>> Normally, you either use one or the other, but here both are in use because the method returns a CompletableFuture (e.g. like a JS/Scala promise), and then is taking a callback handler as well.
>>
>> If you go down the path of promises, the normal thing would be:
>>
>> CompletableFuture<Void> forkInvocation(VisitableCommand newCommand)
>>
>> And then you chain (e.g. thenCompose or similar) the return function when the fork has completed. I assume that going down this route causes unnecessary allocations?
>>
>> If we're worried about allocations, I wonder whether a pure handler based approach might work better? It might not be as user-friendly but if we want to avoid CompletableFuture issues internally, then maybe that's the way? e.g.
>>
>> void forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler);
>>
>> While at Summit/DevNation I had a chat with Clement from Vert.x team who had some reservations about CompletableFutures themselves. I don't remember the details right now (that's what vacation does to you!) but I've pinged him to find out more.
>>
>> In general though, there are multiple issues associated with the handler approach, including:
>>
>> * Lack of central error handling logic.
>> * Callback readibility, when multiple handlers are specified in one go [3].
>>
>> Cheers,
>>
>> [1] http://www.servicedesignpatterns.com/WebServiceInfrastructures/AsyncResponseHandler
>> [2] http://exploringjs.com/es6/ch_promises.html
>> [3] http://callbackhell.com
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>> On 22 Jun 2016, at 17:52, Radim Vansa <rvansa at redhat.com> wrote:
>>>
>>> Yes [1]. The longest chaining of operations I had in [2], basically
>>> during ST I have to load a value locally*, perform a unicast/broadcast
>>> to read different value and then execute the original one.
>>>
>>> * I shouldn't load it just from DC, as it could be in cache store, too;
>>> though without persistence (which I don't deal with properly in
>>> scattered cache yet) it would be more efficient to do the DC lookup
>>> directly.
>>>
>>> Radim
>>>
>>> [1] https://github.com/rvansa/infinispan/tree/ISPN-6645
>>> [2]
>>> https://github.com/rvansa/infinispan/blob/ISPN-6645/core/src/main/java/org/infinispan/interceptors/impl/PrefetchInvalidationInterceptor.java
>>>
>>> On 06/17/2016 10:52 AM, Galder Zamarre?o wrote:
>>>> Radim, do you have a branch where you have been trying these things out? I'd like to play with what you're trying to do.
>>>>
>>>> Cheers,
>>>> --
>>>> Galder Zamarre?o
>>>> Infinispan, Red Hat
>>>>
>>>>> On 8 Jun 2016, at 14:23, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I would like to encourage you to play with the (relatively) new API for
>>>>> sequential interceptors, and voice your comments - especially you corish
>>>>> devs, and Galder who has much experience with async invocations and that
>>>>> kind of stuff from JS-world.
>>>>>
>>>>> I am now trying to use the asynchronous methods only (the
>>>>> forkInvocationSync() is only temporary helper!); Dan has made it this
>>>>> way as he wanted to avoid unnecessary allocations, and I welcome this
>>>>> GC-awareness, but regrettably I find it rather hard to use, due to its
>>>>> handler-style nature. For the simplest style interceptors (do this,
>>>>> invoke next interceptor, and process result) it's fine, but when you
>>>>> want to do something like:
>>>>>
>>>>> visitFoo(cmd) {
>>>>>    Object x = null;
>>>>>    if (/* ... */) {
>>>>>        x = invoke(new OtherCommand());
>>>>>    }
>>>>>    invoke(new DifferentCommand(x));
>>>>>    Object retval = invoke(cmd);
>>>>>    return wrap(retval);
>>>>> }
>>>>>
>>>>> I find myself passing handlers deep down. There is allocation cost for
>>>>> closures, so API that does not allocate CompletableFutures does not pay off.
>>>>>
>>>>> I don't say that I could improve it (I have directed my comments to Dan
>>>>> on IRC when I had something in particular), I just say that this is very
>>>>> important API for further Infinispan development and everyone should pay
>>>>> attention before it gets final.
>>>>>
>>>>> So please, play with it, and show your opinion.
>>>>>
>>>>> Radim
>>>>>
>>>>> --
>>>>> Radim Vansa <rvansa at redhat.com>
>>>>> JBoss Performance Team
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From mudokonman at gmail.com  Wed Aug  3 08:50:58 2016
From: mudokonman at gmail.com (William Burns)
Date: Wed, 03 Aug 2016 12:50:58 +0000
Subject: [infinispan-dev] Compatiibility 2.0 dump
In-Reply-To: <1DA0CA01-4903-4A7B-AC75-EBBC80A0FA63@redhat.com>
References: <5768F177.9060307@redhat.com> <576AB255.7060202@redhat.com>
	<1DA0CA01-4903-4A7B-AC75-EBBC80A0FA63@redhat.com>
Message-ID: <CA+YCuUNUu2zygFbqbVHsz5OjiH6MoD=JiZbbJ2AtHeVf9NPCeQ@mail.gmail.com>

On Mon, Jul 25, 2016 at 10:03 AM Galder Zamarre?o <galder at redhat.com> wrote:

>
> > On 22 Jun 2016, at 17:44, Radim Vansa <rvansa at redhat.com> wrote:
> >
> > I've spotted things like 'decorating cache' in the wiki page. I though
> > that the core architecture in Infinispan, modifying the behavior
> > according to configurations, is the interceptor stack. While we have
> > some doubts about its performance, and there are limitations - e.g. the
> > Flags don't allow to add custom parameters and we certainly don't want
> > to add Flag.JSON and Flag.XML - I would consider decorating a Cache<K,
> > V> vs. adding interceptors.
> >
> > I am thinking of adding the transcoder information to invocation context
> > and only pass different ICF to the CacheImpl. Though, this requires new
> > factory, new interceptor and a handful of specialized context classes
> > (or a wrapper to the existing ones). Whoo, just decorating Cache sounds
> > much simpler (and probably more performant). Or should we have forks in
> > interceptor stack? (as an alternative to different wrappers).
> >
> > The idea of interceptors is that these are common for all operations, if
> > we want to do things differently for different endpoints (incl.
> > embedded), decorating probably is the answer.
> >
> > My 2c (or rather just random thoughts and whining)
>
> Will and I had a good discussion on the problems that interceptors had
> with data conversion layers while at Summit/DevNation. I don't remember the
> details very well, that's what 3 weeks of holiday does to you ;), but Will
> will reply with some more details. From what I remember, doing conversion
> in interceptors made did not fully work with streams and custom filters.
>

Sorry for the delay.

The issue from what I remember is mostly what you said Galder.  Streams in
particular are interesting because you may have a Stream<byte[], byte[]>
but underneath it is really a Stream<Object, Object> so if you try to pass
an intermediate operation such as filter or map, you would pass a
Predicate<? super byte[]> but that can't be applied to the underlying
stream without first unboxing the key/value but then the value returned
from the stream may not know if the value is unboxed or not.  It is just
quite messy.  I don't even remember how this works when protobuf is also in
the mix :(

Filters are somewhat similar but they are mostly okay because they only
apply at most 1 filter/map method and then use an iterator where as a
stream could be any number.


>
> Cheers,
>
> >
> > Radim
> >
> > On 06/21/2016 09:49 AM, Tristan Tarrant wrote:
> >> Hi all,
> >>
> >> I've created a wiki [1] for the "compatibility 2.0" ideas we talked
> >> about recently at the query meeting.
> >>
> >> This is mostly a dump of the minutes, so the form is not complete, but
> >> initial comments are welcome.
> >>
> >>
> >> Tristan
> >>
> >> [1] https://github.com/infinispan/infinispan/wiki/Compatibility-2.0
> >
> >
> > --
> > Radim Vansa <rvansa at redhat.com>
> > JBoss Performance Team
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160803/9cf68986/attachment-0001.html 

From galder at redhat.com  Thu Aug  4 03:03:10 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Thu, 4 Aug 2016 09:03:10 +0200
Subject: [infinispan-dev] Sequential interceptors API
In-Reply-To: <CA+nfvwSwb2=DAO7wbkGVv+5DTA4GjGo=afQenw3coMFHYpTOjA@mail.gmail.com>
References: <57580E54.10703@redhat.com>
	<8BD560BF-0F3E-4CC8-843C-EB54F5410877@redhat.com>
	<576AB428.10203@redhat.com>
	<79B0BF43-E05B-44CF-BC9C-2E534B55C5BB@redhat.com>
	<57A1CC2C.2010501@redhat.com>
	<CA+nfvwSwb2=DAO7wbkGVv+5DTA4GjGo=afQenw3coMFHYpTOjA@mail.gmail.com>
Message-ID: <F8975DD6-B563-46EA-9B81-A5C28C2075AC@redhat.com>


--
Galder Zamarre?o
Infinispan, Red Hat

> On 3 Aug 2016, at 13:54, Dan Berindei <dan.berindei at gmail.com> wrote:
> 
> Yeah, I just fixed (almost all of) the test failures this morning and
> I pushed it here:
> 
> https://github.com/infinispan/infinispan/compare/master...danberindei:ISPN-5467_CompletableFuture-like_API?expand=1
> 
> 
> Galder, I'm not sure I understand the part about mixing promises and
> callbacks... almost all CompletableFuture methods take a callback
> parameter, so I'm not sure how one could avoid mixing them.

The question how these callbacks are passed:

When using Promises or CompletableFutures, the callbacks are passed to the promise/CF via its fluent API, e.g:

CompletableFuture cf = methodA();
cf.thenCompose().thenCombine().handle().whenComplete()... // Might not be realistic, just showing fluency in action

In your case, the forkHandler is passed in as parameter to methodA() instead of it being a continuation that you add to the CompletableFuture via its fluent API. 

Passing handlers as parameters to a method you created (and not as parameters to a continuation of CompletableFuture) is the old asynchronous call handling method that Node.js made so famous. The current API mixes both, by passing some callbacks to CompletableFuture methods and then some being passed around as method parameters, which makes it ackward.

So, instead of:

CompletableFuture<Void> forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler)

I'd expect:

CompletableFuture<Z> forkInvocation(VisitableCommand newCommand);

And use it like this:

CompletableFuture<Z> cf = forkInvocation(cmd);
cf.whenComplete(forkReturnHandler); // 

So, what is forkReturnHandler? forkReturnHandler would be a BiConsumer<Z, Throwable>.

Now, what is Z? Z is everything that the handler needs to deal with, except Throwable, which gets passed as 2nd element to the BiConsumer. So Z would a wrapper around InvocationContext, VisitableCommand and Object (return value).


> That being
> said, the API as it is in master now tends to encourage nesting of
> callbacks, and that's what I'm trying to change.
> 
> My new approach has a promise-like interface called
> IntermediateInvocationStage (for now), which works like a promise on
> the interceptors not yet executed. It has various methods you can then
> use to add callbacks to be executed after those interceptors (and any
> callbacks they installed).
> 
> To get an IntermediateInvocationStage, you need to call a method like
> invokeNext(ctx, command) on BaseAsyncInterceptor/InterceptorStage. I'm
> not yet sure whether exposing InterceptorStage to the interceptor is
> really necessary.
> 
> Both the interceptor and the callbacks then return an InvocationStage,
> which is implemented by the same classes that implement
> IntermediateInvocationStage, but doesn't allow adding more callbacks.
> This is mainly so that I can have returnWith(returnValue) instead of
> returnWith(ctx, command, returnValue).
> 
> For the actual asynchronous work, I have 2 goAsync methods in
> InterceptorStage, because I think having an explicit callback can be
> slightly more efficient - I'm not 100% sure about this either. I used
> to have a waitFor(CompletionStage<?>) method that returned an
> InterceptorStage, but I decided it was too confusing to have
> InterceptorStage behave like a promise as well.
> 
> IntermediateInvocationStage also allows "asynchronous" callbacks with
> composeAsync, but it seems a bit weird to have a single callback
> handle both on an InvocationStage and on a CompletionStage. So I'm
> thinking of removing that, and forcing the interceptors to use
> compose+goAsync instead. That probably means callback nesting again,
> so I need to check how it would affect the L1 interceptors.
> 
> About performance: I only tested non-tx local reads so far, but in
> that case I'm only getting one real allocation in CallInterceptor,
> which gets reused by all the other interceptors. For this scenario all
> the callbacks are cached in the interceptor fields, of course, so it's
> not HotSpot doing scalar replacement for the callbacks - although in
> theory that should work as well. I need to do a lot more testing on
> this...
> 
> 
> Cheers
> Dan
> 
> 
> 
> On Wed, Aug 3, 2016 at 1:49 PM, Radim Vansa <rvansa at redhat.com> wrote:
>> I think that Dan had another branch with more FSM-like API, but I can't
>> find it :-/
>> 
>> R.
>> 
>> On 08/03/2016 11:57 AM, Galder Zamarre?o wrote:
>>> Hey Radim,
>>> 
>>> Sorry for the delay having a look at this (Summit/DevNation then vacation).
>>> 
>>> I agree with you that the way you're passing down the handler down the stack and accross lambdas is a bit counter-productive, because it means the lambda has to capture the handler, which in turn means that each lambda execution will result in a lambda instantiation.
>>> 
>>> Looking at AsyncInvocationContext, methods like this feel a bit of an anti-pattern:
>>> 
>>> CompletableFuture<Void> forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler)
>>> 
>>> The problem here is that it mixes the two main ways of dealing with async computation:
>>> 
>>> 1. Callback handlers [1].
>>> 2. Promises (~ CompletableFuture) [2].
>>> 
>>> Normally, you either use one or the other, but here both are in use because the method returns a CompletableFuture (e.g. like a JS/Scala promise), and then is taking a callback handler as well.
>>> 
>>> If you go down the path of promises, the normal thing would be:
>>> 
>>> CompletableFuture<Void> forkInvocation(VisitableCommand newCommand)
>>> 
>>> And then you chain (e.g. thenCompose or similar) the return function when the fork has completed. I assume that going down this route causes unnecessary allocations?
>>> 
>>> If we're worried about allocations, I wonder whether a pure handler based approach might work better? It might not be as user-friendly but if we want to avoid CompletableFuture issues internally, then maybe that's the way? e.g.
>>> 
>>> void forkInvocation(VisitableCommand newCommand, AsyncInterceptor.ForkReturnHandler forkReturnHandler);
>>> 
>>> While at Summit/DevNation I had a chat with Clement from Vert.x team who had some reservations about CompletableFutures themselves. I don't remember the details right now (that's what vacation does to you!) but I've pinged him to find out more.
>>> 
>>> In general though, there are multiple issues associated with the handler approach, including:
>>> 
>>> * Lack of central error handling logic.
>>> * Callback readibility, when multiple handlers are specified in one go [3].
>>> 
>>> Cheers,
>>> 
>>> [1] http://www.servicedesignpatterns.com/WebServiceInfrastructures/AsyncResponseHandler
>>> [2] http://exploringjs.com/es6/ch_promises.html
>>> [3] http://callbackhell.com
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>> 
>>>> On 22 Jun 2016, at 17:52, Radim Vansa <rvansa at redhat.com> wrote:
>>>> 
>>>> Yes [1]. The longest chaining of operations I had in [2], basically
>>>> during ST I have to load a value locally*, perform a unicast/broadcast
>>>> to read different value and then execute the original one.
>>>> 
>>>> * I shouldn't load it just from DC, as it could be in cache store, too;
>>>> though without persistence (which I don't deal with properly in
>>>> scattered cache yet) it would be more efficient to do the DC lookup
>>>> directly.
>>>> 
>>>> Radim
>>>> 
>>>> [1] https://github.com/rvansa/infinispan/tree/ISPN-6645
>>>> [2]
>>>> https://github.com/rvansa/infinispan/blob/ISPN-6645/core/src/main/java/org/infinispan/interceptors/impl/PrefetchInvalidationInterceptor.java
>>>> 
>>>> On 06/17/2016 10:52 AM, Galder Zamarre?o wrote:
>>>>> Radim, do you have a branch where you have been trying these things out? I'd like to play with what you're trying to do.
>>>>> 
>>>>> Cheers,
>>>>> --
>>>>> Galder Zamarre?o
>>>>> Infinispan, Red Hat
>>>>> 
>>>>>> On 8 Jun 2016, at 14:23, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> I would like to encourage you to play with the (relatively) new API for
>>>>>> sequential interceptors, and voice your comments - especially you corish
>>>>>> devs, and Galder who has much experience with async invocations and that
>>>>>> kind of stuff from JS-world.
>>>>>> 
>>>>>> I am now trying to use the asynchronous methods only (the
>>>>>> forkInvocationSync() is only temporary helper!); Dan has made it this
>>>>>> way as he wanted to avoid unnecessary allocations, and I welcome this
>>>>>> GC-awareness, but regrettably I find it rather hard to use, due to its
>>>>>> handler-style nature. For the simplest style interceptors (do this,
>>>>>> invoke next interceptor, and process result) it's fine, but when you
>>>>>> want to do something like:
>>>>>> 
>>>>>> visitFoo(cmd) {
>>>>>>   Object x = null;
>>>>>>   if (/* ... */) {
>>>>>>       x = invoke(new OtherCommand());
>>>>>>   }
>>>>>>   invoke(new DifferentCommand(x));
>>>>>>   Object retval = invoke(cmd);
>>>>>>   return wrap(retval);
>>>>>> }
>>>>>> 
>>>>>> I find myself passing handlers deep down. There is allocation cost for
>>>>>> closures, so API that does not allocate CompletableFutures does not pay off.
>>>>>> 
>>>>>> I don't say that I could improve it (I have directed my comments to Dan
>>>>>> on IRC when I had something in particular), I just say that this is very
>>>>>> important API for further Infinispan development and everyone should pay
>>>>>> attention before it gets final.
>>>>>> 
>>>>>> So please, play with it, and show your opinion.
>>>>>> 
>>>>>> Radim
>>>>>> 
>>>>>> --
>>>>>> Radim Vansa <rvansa at redhat.com>
>>>>>> JBoss Performance Team
>>>>>> 
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> --
>>>> Radim Vansa <rvansa at redhat.com>
>>>> JBoss Performance Team
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From slaskawi at redhat.com  Fri Aug  5 09:30:42 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Fri, 5 Aug 2016 15:30:42 +0200
Subject: [infinispan-dev] Status 2015-08-05
Message-ID: <CAA9R9O7ChXc+JmiF33UZACVzGtQ2LhVmA+KbXYWG0qkvHqiKeg@mail.gmail.com>

Hey!

Unfortunately I won't be able to join you during the status meeting on
Monday, so here's what managed to do last week:

   - I mainly worked on HelthChack API:
      - https://issues.jboss.org/browse/ISPN-6679
      -
      https://github.com/slaskawi/infinispan/tree/ISPN-6679/Healthcheck_api
      - Interfaces and JMX are done, the only missing bit is the Wildfly
      integrations (currently implemented as metrics)
      - Tomaz Cerar mentioned there is no way to use Custom Handlers for
      Management API (we can not implement gathering statistics with a
custom URL
      like http://localhost:9990/health). We can use HTTP GET for obtaining
      resources or POST for operations and runtime resources.
      - I need to think it over and choose one of the solutions (using
      metrics seem to be the worst approach)
      - I talked with James about printing out last X lines of log - again
      it's not an easy task and James suggested that it can be using
the logging
      subsystem. I will look into that too.
   - I fixed a couple of Windows errors for the product (they were ok in
   the upstream).
   - I started implementing a CI job for our Docker images and OpenShift
   - I also prepared a blog post about Infinispan and OpenShift (it will be
   released next week).
   - There are also some good news from my home city - we're opening
   Bydgoszcz JUG (I'd love to see you guys pronouncing "Bydgoszcz" :D) in
   September/October and I reserved 2 talks about Open Source (maintaining,
   contributing etc) and Infinispan+OpenShift/Kubernetes. Wish me luck!

Thanks
Sebastian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160805/5489ed4d/attachment.html 

From rory.odonnell at oracle.com  Mon Aug  8 04:22:58 2016
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Mon, 8 Aug 2016 09:22:58 +0100
Subject: [infinispan-dev] Early Access builds of JDK 8u112 b04,
 JDK 9 b130 are available on java.net
Message-ID: <c959a80d-4bcb-fdad-c090-9c9d42595299@oracle.com>


Hi Galder,

Early Access b130 <https://jdk9.java.net/download/> for JDK 9 is 
available on java.net, summary of  changes are listed here 
<http://download.java.net/java/jdk9/changes/jdk-9+130.html>.

Early Access b129 <https://jdk9.java.net/jigsaw/> (#5332) for JDK 9 with 
Project Jigsaw is available on java.net, summary of changes are listed 
here 
<http://download.java.net/java/jigsaw/archive/129/binaries/jdk-9+129.html>

Early Access b04 <https://jdk8.java.net/download.html> for JDK 8u112 is 
available on java.net, summary of  changes are listed here 
<http://download.java.net/java/jdk8u112/changes/jdk8u112-b04.html>


Rgds,Rory

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160808/f39fb254/attachment.html 

From sanne at infinispan.org  Mon Aug  8 08:09:21 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Mon, 8 Aug 2016 13:09:21 +0100
Subject: [infinispan-dev] jdk8backported package
In-Reply-To: <CA+YCuUMZ-3_d7o9MNyofo3EoYkbJDXJRUU8svnEm3qupJbZ63Q@mail.gmail.com>
References: <56976B94.9010704@redhat.com>
	<CA+YCuUNV+BLiLKHuym+O8ftSVwE93bQoAwFiMQ1RAK9=P0CUJg@mail.gmail.com>
	<CAA9R9O5JVYyWgEWJi_h=SDpt=Cic6rDmQNGJGAZjqBK66abEig@mail.gmail.com>
	<CA+YCuUMZ-3_d7o9MNyofo3EoYkbJDXJRUU8svnEm3qupJbZ63Q@mail.gmail.com>
Message-ID: <CAFm4XO2Aw9HhdetNWXs5d5eoYjNR_fDZKbmdHvwRdcFoLafK-g@mail.gmail.com>

Hi,
sorry for reviving this old thread, but this came up again in the form
of doubts and design questions this morning from people in the
performance team profiling it.

Let's please change the name, I've opened a JIRA to not forget again:
 - https://issues.jboss.org/browse/ISPN-6933

Will, assigned to you as I assume it would take you 2 minutes :)

Thanks,
Sanne


On 25 January 2016 at 20:18, William Burns <mudokonman at gmail.com> wrote:
>
>
> On Mon, Jan 25, 2016 at 10:09 AM Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
>>
>> Would it be possible to decouple Map implementation from our custom code
>> (like eviction policy or setting max size)? Or the dependency between those
>> two is too strong?
>
>
> This is possible.  I had already thought of doing that at some point, but
> haven't been able to look into it or have a great reason yet (we lose some
> stuff like better size estimations - since we have access to the underlying
> array).
>
> Unfortunately the equivalent versions have to stay even if we remove the
> eviction stuff, since there is no pluggable way to do this with the JVM
> classes.
>
>>
>>
>> On Mon, Jan 25, 2016 at 3:15 PM, William Burns <mudokonman at gmail.com>
>> wrote:
>>>
>>> Unfortunately this isn't really possible to remove these classes.  They
>>> have additional changes over and above the jdk8 classes.  We could move it
>>> into a different package though if we wanted to.
>>>
>>> To be honest the only reason this is still really needed is for the
>>> BoundedConcurrentHashMapV8.  I have been talking with Dan on and off for a
>>> while about even possibly looking into using a different map implementation
>>> (such as caffeine or something else equivalent).  We haven't looked into it
>>> very seriously though.
>>>
>>> We could look into this more if people think we want to tackle that.
>>>
>>>  - Will
>>>
>>> On Thu, Jan 14, 2016 at 4:34 AM Radim Vansa <rvansa at redhat.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> although we're on Java 8, there's still the package
>>>> org.infinispan.*.jdk8backported in our codebase. Is there any plan (and
>>>> possibility) to remove these and use implementation provided by runtime?
>>>> Or have we tweaked them too much, so shall we rather rename them?
>>>>
>>>> Radim
>>>>
>>>> --
>>>> Radim Vansa <rvansa at redhat.com>
>>>> JBoss Performance Team
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From galder at redhat.com  Tue Aug  9 08:15:01 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 9 Aug 2016 14:15:01 +0200
Subject: [infinispan-dev] Weekly IRC meeting - 15th August 2016
Message-ID: <E5DFF795-1CAA-4264-B835-F0EE725D7240@redhat.com>

Hi all,

Please find below the summary of the weekly IRC meeting we had yesterday:
http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2016/infinispan.2016-01-04-15.08.html

Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat



From ttarrant at redhat.com  Tue Aug  9 10:51:33 2016
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Tue, 9 Aug 2016 16:51:33 +0200
Subject: [infinispan-dev] Weekly IRC meeting - 15th August 2016
In-Reply-To: <E5DFF795-1CAA-4264-B835-F0EE725D7240@redhat.com>
References: <E5DFF795-1CAA-4264-B835-F0EE725D7240@redhat.com>
Message-ID: <79be37cc-cbd6-0003-6fa6-6c477432f299@infinispan.org>

Galder, I believe you really wanted to share this instead:

http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2016/infinispan.2016-08-08-13.58.log.html

As for me,  I've been doing the following:

- releasing the MongoDB cachestore
- removing some TODOs from the docs (ISPN-4735)
- playing with the DynaLinker from Java 9 to find a solution for Nashorn 
clustered streams (ISPN-6173). The DynaLinker also exists in Java 8 but 
uses a non public API (although it should be possible to support it as 
well).
- continuing work on refreshing the DZone RefCard for Infinispan

This week I plan to continue the above as well as release an update to 
the Cloud Cache Store.

Tristan

On 09/08/16 14:15, Galder Zamarre?o wrote:
> Hi all,
>
> Please find below the summary of the weekly IRC meeting we had yesterday:
> http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2016/infinispan.2016-01-04-15.08.html
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From slaskawi at redhat.com  Thu Aug 11 04:59:46 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 11 Aug 2016 10:59:46 +0200
Subject: [infinispan-dev] Configuration management and
	Docker/Kubernetes/OpenShift
Message-ID: <CAA9R9O7Qc3WG=F-W0QtfgeitbWe4Y_45y_Y1nd33KOzeQn6V+w@mail.gmail.com>

Hey!

I'm working on configuration management in cloud environment [1]. I put
some finding in the ticket [1] but let me tell you more about the options I
investigated:

   - Kubernetes/OpenShift ConfigMaps [2][3]
      - Those structures are specially designed to hold configuration data
      in any format (yaml, json, properties, xml, doesn't matter).
      - The configuration can be mounted into a pod as a directory
      (technically a volume). We can not mount it as a single file. Having said
      that we would need to either store cloud.xml file separately
      (standalone/cloud/configuration?) or ask users to build
configuration from
      whole standalone/configuration directory. Both options are valid in my
      opinion.
   - OpenShift S2I [4] builder
      - S2I builder takes a git repository and a Docker image and combines
      those two together.
      - We could store a cloud.xml file inside a git repository and modify
      our infinispan Docker image to support S2I scripts (if cloud.xml is
      detected in a git repository - replace the default configuration).
      - Unfortunately it's OpenShift specific thing.
   - Extend the Infinispan Docker image
      - We could ask users to extend our Docker image and put their
      specific configuration there
      - The biggest advantage - it will work regardless to the environment
      (pure Docker, Kubernetes, OpenShift, doesn't matter)

All options require restarting pods to update configuration (remember, pods
were designed to be immutable).

I think we should support 2 options - ConfigMaps for Kubernetes and
OpenShift and extending our Docker image for all other use cases (because
this option gives the most flexibility).

What do you think?

Thanks
Sebastian

[1] https://issues.jboss.org/browse/ISPN-6675
[2] http://kubernetes.io/docs/user-guide/configmap/
[3] https://docs.openshift.org/latest/dev_guide/configmaps.html
[4] https://github.com/openshift/source-to-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160811/cd1e3c13/attachment.html 

From gustavo at infinispan.org  Thu Aug 11 05:17:14 2016
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Thu, 11 Aug 2016 10:17:14 +0100
Subject: [infinispan-dev] Configuration management and
	Docker/Kubernetes/OpenShift
In-Reply-To: <CAA9R9O7Qc3WG=F-W0QtfgeitbWe4Y_45y_Y1nd33KOzeQn6V+w@mail.gmail.com>
References: <CAA9R9O7Qc3WG=F-W0QtfgeitbWe4Y_45y_Y1nd33KOzeQn6V+w@mail.gmail.com>
Message-ID: <CAH8Ud1EgmhFrX6gvUsM_VamaEejiDAd56-vqDYX49Z8Uu+2oPA@mail.gmail.com>

On Thu, Aug 11, 2016 at 9:59 AM, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> Hey!
>
> I'm working on configuration management in cloud environment [1]. I put
> some finding in the ticket [1] but let me tell you more about the options I
> investigated:
>
>    - Kubernetes/OpenShift ConfigMaps [2][3]
>       - Those structures are specially designed to hold configuration
>       data in any format (yaml, json, properties, xml, doesn't matter).
>       - The configuration can be mounted into a pod as a directory
>       (technically a volume). We can not mount it as a single file. Having said
>       that we would need to either store cloud.xml file separately
>       (standalone/cloud/configuration?) or ask users to build
>       configuration from whole standalone/configuration directory. Both options
>       are valid in my opinion.
>    - OpenShift S2I [4] builder
>       - S2I builder takes a git repository and a Docker image and
>       combines those two together.
>       - We could store a cloud.xml file inside a git repository and
>       modify our infinispan Docker image to support S2I scripts (if cloud.xml is
>       detected in a git repository - replace the default configuration).
>       - Unfortunately it's OpenShift specific thing.
>    - Extend the Infinispan Docker image
>       - We could ask users to extend our Docker image and put their
>       specific configuration there
>       - The biggest advantage - it will work regardless to the
>       environment (pure Docker, Kubernetes, OpenShift, doesn't matter)
>
> All options require restarting pods to update configuration (remember,
> pods were designed to be immutable).
>
> I think we should support 2 options - ConfigMaps for Kubernetes and
> OpenShift and extending our Docker image for all other use cases (because
> this option gives the most flexibility).
>
> What do you think?
>


For static configuration use cases, I think they are all valid options,
it's up to the user to pick the right one depending on the environment/use
case.
With relation to the S2I specifically, it could be really interesting for
the cases when custom code (marshallers, filters, taks, etc) need to be
deployed along Infinispan:
the S2I machinery could build those jars, add custom configurations,
combine with the jboss/infinispan-server and have it running on Openshift.

When dynamic configuration changes need to be applied, I think [1] is best
option.

[1] https://issues.jboss.org/browse/ISPN-6000

Cheers,
Gustavo



>
> Thanks
> Sebastian
>
> [1] https://issues.jboss.org/browse/ISPN-6675
> [2] http://kubernetes.io/docs/user-guide/configmap/
> [3] https://docs.openshift.org/latest/dev_guide/configmaps.html
> [4] https://github.com/openshift/source-to-image
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160811/65affa0e/attachment.html 

From anistor at redhat.com  Fri Aug 12 07:36:39 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Fri, 12 Aug 2016 14:36:39 +0300
Subject: [infinispan-dev] Release of Infinispan 8.2.4.Final
Message-ID: <f3c6e537-ca02-0a82-1f95-4ffc39a8afe9@redhat.com>

Hi all,

yesterday we made a new maintenance release: 8.2.4.Final. Read about it 
here: http://goo.gl/d1DqSQ

Cheers!

Adrian


From sanne at infinispan.org  Tue Aug 16 05:34:21 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 16 Aug 2016 10:34:21 +0100
Subject: [infinispan-dev] Hot Rod Multi-get with versions?
Message-ID: <CAFm4XO35=0CN1MFToPOaP7OR6PqqkKgtDwC8-KP6JnOGuGD_Zw@mail.gmail.com>

Hi all,

I'm having lots of fun learning to use the Hot Rod clients.

One thing that is getting very clear to me is that getVersioned( k )
is much more useful than a regular get, as anything I'm doing turns
out to ultimately need a version when it goes beyond a trivial hello
world test.

In certain cases I can optimise requests by using the very useful
getAll( keyset ) method.

However, I don't see how I can use getAll while still reading versioned entries?

Thanks,
Sanne

From galder at redhat.com  Tue Aug 16 11:37:43 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 16 Aug 2016 17:37:43 +0200
Subject: [infinispan-dev] Style  errors :(
Message-ID: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>

Hi all,

After integrating [1] I'm getting build errors such as:

[INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @ infinispan-core ---
[INFO] Starting audit...
/home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/internal/InternalExternalizerTable.java:55: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
/home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/ExternalizerTable.java:76: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
/home/g/0/infinispan/git/core/src/test/java/org/infinispan/filter/CompositeKeyValueFilterConverter.java:11:1: error: Duplicate import to line 10 - org.infinispan.metadata.Metadata.

Even after installing the latest style for IntelliJ [2], reformatting InternalExternalizerTable.java won't fix those how errors.

So, what do we do? :(

Cheers,

[1] https://github.com/infinispan/infinispan/commit/313b19301055c6267c6f2ea9065a7ab1b68099fe
[2] https://github.com/infinispan/infinispan/blob/master/ide-settings/intellij/IntelliJ_IDEA_Code_Style.jar
--
Galder Zamarre?o
Infinispan, Red Hat



From sanne at infinispan.org  Tue Aug 16 14:21:04 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 16 Aug 2016 19:21:04 +0100
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
Message-ID: <CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>

Please don't forget Eclipse users. It's hard enough to contribute to
Infinispan for non-IDEA users; if you enforce specific style rules, at
the very least provide compatible formatters for the other IDEs or the
barrier for contributors becomes too high.

Galder: to answer your specific question I'd personally say you should
revert the offending commit. I don't know who did it nor who merged
it, but it's just not nice for others that such things get merged
carelessly. Errors can be forgiven, but should be fixed by those who
create the mess ;)

On 16 August 2016 at 16:37, Galder Zamarre?o <galder at redhat.com> wrote:
> Hi all,
>
> After integrating [1] I'm getting build errors such as:
>
> [INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @ infinispan-core ---
> [INFO] Starting audit...
> /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/internal/InternalExternalizerTable.java:55: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
> /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/ExternalizerTable.java:76: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
> /home/g/0/infinispan/git/core/src/test/java/org/infinispan/filter/CompositeKeyValueFilterConverter.java:11:1: error: Duplicate import to line 10 - org.infinispan.metadata.Metadata.
>
> Even after installing the latest style for IntelliJ [2], reformatting InternalExternalizerTable.java won't fix those how errors.
>
> So, what do we do? :(
>
> Cheers,
>
> [1] https://github.com/infinispan/infinispan/commit/313b19301055c6267c6f2ea9065a7ab1b68099fe
> [2] https://github.com/infinispan/infinispan/blob/master/ide-settings/intellij/IntelliJ_IDEA_Code_Style.jar
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From slaskawi at redhat.com  Tue Aug 16 16:16:36 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Tue, 16 Aug 2016 22:16:36 +0200
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
	<CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
Message-ID: <CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>

Hey Galder, Sanne!

The author is no longer anonymous! It was me [1]!

Jokes aside; Galder - may I ask you to double check your configuration? I
extracted formatter archive and it seems to be fine [2]. Perhaps you didn't
enable it (watch out, the formatter name was changed some time ago from
Horizon to Infinispan) or this is some tooling problem (which probably
should be reported to Jetbrains). Anyway - could you please double check
it? I'll try to catch you tomorrow on IRC and talk about it...

BTW - There are 2 different errors in your output - the first one is the
start import which should be avoided. The second one is duplicated import
(also should be avoided). Please don't confuse those two.

@Sanne - before you trow a 'git revert' axe on me - may I ask you to import
the latest formatter and check if the amount of imports before replacing
them with star looks correctly? It looks fine to me but I may have some old
installation, old configuration etc...

Thanks
Sebastian "the Troublemaker"

[1] https://www.youtube.com/watch?v=a43kowi2ncI
[2] unzip IntelliJ_IDEA_Code_Style.jar and look into attached XML. Those
two properties are responsible for star imports.
<option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
<option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />


On Tue, Aug 16, 2016 at 8:21 PM, Sanne Grinovero <sanne at infinispan.org>
wrote:

> Please don't forget Eclipse users. It's hard enough to contribute to
> Infinispan for non-IDEA users; if you enforce specific style rules, at
> the very least provide compatible formatters for the other IDEs or the
> barrier for contributors becomes too high.
>
> Galder: to answer your specific question I'd personally say you should
> revert the offending commit. I don't know who did it nor who merged
> it, but it's just not nice for others that such things get merged
> carelessly. Errors can be forgiven, but should be fixed by those who
> create the mess ;)
>
> On 16 August 2016 at 16:37, Galder Zamarre?o <galder at redhat.com> wrote:
> > Hi all,
> >
> > After integrating [1] I'm getting build errors such as:
> >
> > [INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @
> infinispan-core ---
> > [INFO] Starting audit...
> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/
> marshall/core/internal/InternalExternalizerTable.java:55: error: Using
> the '.*' form of import should be avoided - org.infinispan.marshall.exts.*
> .
> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/
> marshall/core/ExternalizerTable.java:76: error: Using the '.*' form of
> import should be avoided - org.infinispan.marshall.exts.*.
> > /home/g/0/infinispan/git/core/src/test/java/org/infinispan/filter/
> CompositeKeyValueFilterConverter.java:11:1: error: Duplicate import to
> line 10 - org.infinispan.metadata.Metadata.
> >
> > Even after installing the latest style for IntelliJ [2], reformatting
> InternalExternalizerTable.java won't fix those how errors.
> >
> > So, what do we do? :(
> >
> > Cheers,
> >
> > [1] https://github.com/infinispan/infinispan/commit/
> 313b19301055c6267c6f2ea9065a7ab1b68099fe
> > [2] https://github.com/infinispan/infinispan/blob/master/ide-
> settings/intellij/IntelliJ_IDEA_Code_Style.jar
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160816/ce26e7cd/attachment.html 

From slaskawi at redhat.com  Tue Aug 16 16:25:48 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Tue, 16 Aug 2016 22:25:48 +0200
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
	<CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
	<CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
Message-ID: <CAA9R9O4wpxBfaijsPECeB6HG59bBwMKoYQJLwPJmQDWnkgkPVg@mail.gmail.com>

I also integrated Adrian's PR with additional fixed (thank you Adrian!):
https://github.com/infinispan/infinispan/pull/4510

The master branch builds fine now.

Thanks
Sebastian

On Tue, Aug 16, 2016 at 10:16 PM, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> Hey Galder, Sanne!
>
> The author is no longer anonymous! It was me [1]!
>
> Jokes aside; Galder - may I ask you to double check your configuration? I
> extracted formatter archive and it seems to be fine [2]. Perhaps you didn't
> enable it (watch out, the formatter name was changed some time ago from
> Horizon to Infinispan) or this is some tooling problem (which probably
> should be reported to Jetbrains). Anyway - could you please double check
> it? I'll try to catch you tomorrow on IRC and talk about it...
>
> BTW - There are 2 different errors in your output - the first one is the
> start import which should be avoided. The second one is duplicated import
> (also should be avoided). Please don't confuse those two.
>
> @Sanne - before you trow a 'git revert' axe on me - may I ask you to
> import the latest formatter and check if the amount of imports before
> replacing them with star looks correctly? It looks fine to me but I may
> have some old installation, old configuration etc...
>
> Thanks
> Sebastian "the Troublemaker"
>
> [1] https://www.youtube.com/watch?v=a43kowi2ncI
> [2] unzip IntelliJ_IDEA_Code_Style.jar and look into attached XML. Those
> two properties are responsible for star imports.
> <option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
> <option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>
>
> On Tue, Aug 16, 2016 at 8:21 PM, Sanne Grinovero <sanne at infinispan.org>
> wrote:
>
>> Please don't forget Eclipse users. It's hard enough to contribute to
>> Infinispan for non-IDEA users; if you enforce specific style rules, at
>> the very least provide compatible formatters for the other IDEs or the
>> barrier for contributors becomes too high.
>>
>> Galder: to answer your specific question I'd personally say you should
>> revert the offending commit. I don't know who did it nor who merged
>> it, but it's just not nice for others that such things get merged
>> carelessly. Errors can be forgiven, but should be fixed by those who
>> create the mess ;)
>>
>> On 16 August 2016 at 16:37, Galder Zamarre?o <galder at redhat.com> wrote:
>> > Hi all,
>> >
>> > After integrating [1] I'm getting build errors such as:
>> >
>> > [INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @
>> infinispan-core ---
>> > [INFO] Starting audit...
>> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/m
>> arshall/core/internal/InternalExternalizerTable.java:55: error: Using
>> the '.*' form of import should be avoided - org.infinispan.marshall.exts.*
>> .
>> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/m
>> arshall/core/ExternalizerTable.java:76: error: Using the '.*' form of
>> import should be avoided - org.infinispan.marshall.exts.*.
>> > /home/g/0/infinispan/git/core/src/test/java/org/infinispan/f
>> ilter/CompositeKeyValueFilterConverter.java:11:1: error: Duplicate
>> import to line 10 - org.infinispan.metadata.Metadata.
>> >
>> > Even after installing the latest style for IntelliJ [2], reformatting
>> InternalExternalizerTable.java won't fix those how errors.
>> >
>> > So, what do we do? :(
>> >
>> > Cheers,
>> >
>> > [1] https://github.com/infinispan/infinispan/commit/313b19301055
>> c6267c6f2ea9065a7ab1b68099fe
>> > [2] https://github.com/infinispan/infinispan/blob/master/ide-set
>> tings/intellij/IntelliJ_IDEA_Code_Style.jar
>> > --
>> > Galder Zamarre?o
>> > Infinispan, Red Hat
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160816/fc7e1198/attachment.html 

From anistor at redhat.com  Tue Aug 16 19:01:15 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Wed, 17 Aug 2016 02:01:15 +0300
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <CAA9R9O4wpxBfaijsPECeB6HG59bBwMKoYQJLwPJmQDWnkgkPVg@mail.gmail.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
	<CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
	<CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
	<CAA9R9O4wpxBfaijsPECeB6HG59bBwMKoYQJLwPJmQDWnkgkPVg@mail.gmail.com>
Message-ID: <a9239364-f37a-b340-106f-221411b7f483@redhat.com>

The master branch was building fine anyway because checkstyle does not 
seem to be included in default lifecycle yet, so I'm not sure why this 
generates so much commotion.
But we should stop being lenient about these checkstyle violations and 
fail the build from now on. ATM there are no more violations for the 
currently enabled rules, so it is about time :).
This approach seems to work fine for hibernate-ogm and hibernate-search, 
and they have tons of style rules. We only have three.

On 08/16/2016 11:25 PM, Sebastian Laskawiec wrote:
> I also integrated Adrian's PR with additional fixed (thank you 
> Adrian!): https://github.com/infinispan/infinispan/pull/4510
>
> The master branch builds fine now.
>
> Thanks
> Sebastian
>
> On Tue, Aug 16, 2016 at 10:16 PM, Sebastian Laskawiec 
> <slaskawi at redhat.com <mailto:slaskawi at redhat.com>> wrote:
>
>     Hey Galder, Sanne!
>
>     The author is no longer anonymous! It was me [1]!
>
>     Jokes aside; Galder - may I ask you to double check your
>     configuration? I extracted formatter archive and it seems to be
>     fine [2]. Perhaps you didn't enable it (watch out, the formatter
>     name was changed some time ago from Horizon to Infinispan) or this
>     is some tooling problem (which probably should be reported to
>     Jetbrains). Anyway - could you please double check it? I'll try to
>     catch you tomorrow on IRC and talk about it...
>
>     BTW - There are 2 different errors in your output - the first one
>     is the start import which should be avoided. The second one is
>     duplicated import (also should be avoided). Please don't confuse
>     those two.
>
>     @Sanne - before you trow a 'git revert' axe on me - may I ask you
>     to import the latest formatter and check if the amount of imports
>     before replacing them with star looks correctly? It looks fine to
>     me but I may have some old installation, old configuration etc...
>
>     Thanks
>     Sebastian "the Troublemaker"
>
>     [1] https://www.youtube.com/watch?v=a43kowi2ncI
>     <https://www.youtube.com/watch?v=a43kowi2ncI>
>     [2] unzip IntelliJ_IDEA_Code_Style.jar and look into attached XML.
>     Those two properties are responsible for star imports.
>     <option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>     <option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>
>
>     On Tue, Aug 16, 2016 at 8:21 PM, Sanne Grinovero
>     <sanne at infinispan.org <mailto:sanne at infinispan.org>> wrote:
>
>         Please don't forget Eclipse users. It's hard enough to
>         contribute to
>         Infinispan for non-IDEA users; if you enforce specific style
>         rules, at
>         the very least provide compatible formatters for the other
>         IDEs or the
>         barrier for contributors becomes too high.
>
>         Galder: to answer your specific question I'd personally say
>         you should
>         revert the offending commit. I don't know who did it nor who
>         merged
>         it, but it's just not nice for others that such things get merged
>         carelessly. Errors can be forgiven, but should be fixed by
>         those who
>         create the mess ;)
>
>         On 16 August 2016 at 16:37, Galder Zamarre?o
>         <galder at redhat.com <mailto:galder at redhat.com>> wrote:
>         > Hi all,
>         >
>         > After integrating [1] I'm getting build errors such as:
>         >
>         > [INFO] --- maven-checkstyle-plugin:2.17:checkstyle
>         (checkstyle) @ infinispan-core ---
>         > [INFO] Starting audit...
>         >
>         /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/internal/InternalExternalizerTable.java:55:
>         error: Using the '.*' form of import should be avoided -
>         org.infinispan.marshall.exts.*.
>         >
>         /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/ExternalizerTable.java:76:
>         error: Using the '.*' form of import should be avoided -
>         org.infinispan.marshall.exts.*.
>         >
>         /home/g/0/infinispan/git/core/src/test/java/org/infinispan/filter/CompositeKeyValueFilterConverter.java:11:1:
>         error: Duplicate import to line 10 -
>         org.infinispan.metadata.Metadata.
>         >
>         > Even after installing the latest style for IntelliJ [2],
>         reformatting InternalExternalizerTable.java won't fix those
>         how errors.
>         >
>         > So, what do we do? :(
>         >
>         > Cheers,
>         >
>         > [1]
>         https://github.com/infinispan/infinispan/commit/313b19301055c6267c6f2ea9065a7ab1b68099fe
>         <https://github.com/infinispan/infinispan/commit/313b19301055c6267c6f2ea9065a7ab1b68099fe>
>         > [2]
>         https://github.com/infinispan/infinispan/blob/master/ide-settings/intellij/IntelliJ_IDEA_Code_Style.jar
>         <https://github.com/infinispan/infinispan/blob/master/ide-settings/intellij/IntelliJ_IDEA_Code_Style.jar>
>         > --
>         > Galder Zamarre?o
>         > Infinispan, Red Hat
>         >
>         >
>         > _______________________________________________
>         > infinispan-dev mailing list
>         > infinispan-dev at lists.jboss.org
>         <mailto:infinispan-dev at lists.jboss.org>
>         > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>         <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>
>         _______________________________________________
>         infinispan-dev mailing list
>         infinispan-dev at lists.jboss.org
>         <mailto:infinispan-dev at lists.jboss.org>
>         https://lists.jboss.org/mailman/listinfo/infinispan-dev
>         <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160817/73a1582c/attachment-0001.html 

From slaskawi at redhat.com  Wed Aug 17 01:22:27 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 17 Aug 2016 07:22:27 +0200
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <a9239364-f37a-b340-106f-221411b7f483@redhat.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
	<CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
	<CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
	<CAA9R9O4wpxBfaijsPECeB6HG59bBwMKoYQJLwPJmQDWnkgkPVg@mail.gmail.com>
	<a9239364-f37a-b340-106f-221411b7f483@redhat.com>
Message-ID: <CAA9R9O62W6Gnn3Fp+rHxQ1vcuKj+sUE4_ZLzfiZRXzC15iFOTQ@mail.gmail.com>

Hey Adrian!

+1. There are some more rules in our checkstyle module and I'm planning to
turn them on soon (however I agree, the import one introduces probably the
biggest amount of changes).

Checkstyle should be enabled by default [3] and it executes fine on my
local machine. However if you're not building the checkstyle module with
the same reactor as the rest of the project, it may happen that Maven
downloads it from Nexus (this is exactly what happened in case of [4]). The
situation should be fixed automatically as soon as TeamCity rebuilds and
redeploys new SNAPSHOT.

Thanks
Sebastian

[3]
https://github.com/infinispan/infinispan/blob/master/parent/pom.xml#L1196
[4]
https://gist.github.com/slaskawi/390e87f867218a62e867dab4c1395aee#file-gistfile1-txt-L60

On Wed, Aug 17, 2016 at 1:01 AM, Adrian Nistor <anistor at redhat.com> wrote:

> The master branch was building fine anyway because checkstyle does not
> seem to be included in default lifecycle yet, so I'm not sure why this
> generates so much commotion.
> But we should stop being lenient about these checkstyle violations and
> fail the build from now on. ATM there are no more violations for the
> currently enabled rules, so it is about time :).
> This approach seems to work fine for hibernate-ogm and hibernate-search,
> and they have tons of style rules. We only have three.
>
>
> On 08/16/2016 11:25 PM, Sebastian Laskawiec wrote:
>
> I also integrated Adrian's PR with additional fixed (thank you Adrian!):
> https://github.com/infinispan/infinispan/pull/4510
>
> The master branch builds fine now.
>
> Thanks
> Sebastian
>
> On Tue, Aug 16, 2016 at 10:16 PM, Sebastian Laskawiec <slaskawi at redhat.com
> > wrote:
>
>> Hey Galder, Sanne!
>>
>> The author is no longer anonymous! It was me [1]!
>>
>> Jokes aside; Galder - may I ask you to double check your configuration? I
>> extracted formatter archive and it seems to be fine [2]. Perhaps you didn't
>> enable it (watch out, the formatter name was changed some time ago from
>> Horizon to Infinispan) or this is some tooling problem (which probably
>> should be reported to Jetbrains). Anyway - could you please double check
>> it? I'll try to catch you tomorrow on IRC and talk about it...
>>
>> BTW - There are 2 different errors in your output - the first one is the
>> start import which should be avoided. The second one is duplicated import
>> (also should be avoided). Please don't confuse those two.
>>
>> @Sanne - before you trow a 'git revert' axe on me - may I ask you to
>> import the latest formatter and check if the amount of imports before
>> replacing them with star looks correctly? It looks fine to me but I may
>> have some old installation, old configuration etc...
>>
>> Thanks
>> Sebastian "the Troublemaker"
>>
>> [1] https://www.youtube.com/watch?v=a43kowi2ncI
>> [2] unzip IntelliJ_IDEA_Code_Style.jar and look into attached XML. Those
>> two properties are responsible for star imports.
>> <option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>> <option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>>
>>
>> On Tue, Aug 16, 2016 at 8:21 PM, Sanne Grinovero <sanne at infinispan.org>
>> wrote:
>>
>>> Please don't forget Eclipse users. It's hard enough to contribute to
>>> Infinispan for non-IDEA users; if you enforce specific style rules, at
>>> the very least provide compatible formatters for the other IDEs or the
>>> barrier for contributors becomes too high.
>>>
>>> Galder: to answer your specific question I'd personally say you should
>>> revert the offending commit. I don't know who did it nor who merged
>>> it, but it's just not nice for others that such things get merged
>>> carelessly. Errors can be forgiven, but should be fixed by those who
>>> create the mess ;)
>>>
>>> On 16 August 2016 at 16:37, Galder Zamarre?o <galder at redhat.com> wrote:
>>> > Hi all,
>>> >
>>> > After integrating [1] I'm getting build errors such as:
>>> >
>>> > [INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @
>>> infinispan-core ---
>>> > [INFO] Starting audit...
>>> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/m
>>> arshall/core/internal/InternalExternalizerTable.java:55: error: Using
>>> the '.*' form of import should be avoided - org.infinispan.marshall.exts.*
>>> .
>>> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/m
>>> arshall/core/ExternalizerTable.java:76: error: Using the '.*' form of
>>> import should be avoided - org.infinispan.marshall.exts.*.
>>> > /home/g/0/infinispan/git/core/src/test/java/org/infinispan/f
>>> ilter/CompositeKeyValueFilterConverter.java:11:1: error: Duplicate
>>> import to line 10 - org.infinispan.metadata.Metadata.
>>> >
>>> > Even after installing the latest style for IntelliJ [2], reformatting
>>> InternalExternalizerTable.java won't fix those how errors.
>>> >
>>> > So, what do we do? :(
>>> >
>>> > Cheers,
>>> >
>>> > [1] https://github.com/infinispan/infinispan/commit/313b19301055
>>> c6267c6f2ea9065a7ab1b68099fe
>>> > [2] https://github.com/infinispan/infinispan/blob/master/ide-set
>>> tings/intellij/IntelliJ_IDEA_Code_Style.jar
>>> > --
>>> > Galder Zamarre?o
>>> > Infinispan, Red Hat
>>> >
>>> >
>>> > _______________________________________________
>>> > infinispan-dev mailing list
>>> > infinispan-dev at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>>
>
>
> _______________________________________________
> infinispan-dev mailing listinfinispan-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160817/e890f4a6/attachment.html 

From galder at redhat.com  Wed Aug 17 02:36:31 2016
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Wed, 17 Aug 2016 08:36:31 +0200
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <a9239364-f37a-b340-106f-221411b7f483@redhat.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
	<CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
	<CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
	<CAA9R9O4wpxBfaijsPECeB6HG59bBwMKoYQJLwPJmQDWnkgkPVg@mail.gmail.com>
	<a9239364-f37a-b340-106f-221411b7f483@redhat.com>
Message-ID: <B99C94AF-B03A-475E-8370-DBDD0DA3D84D@redhat.com>


--
Galder Zamarre?o
Infinispan, Red Hat

> On 17 Aug 2016, at 01:01, Adrian Nistor <anistor at redhat.com> wrote:
> 
> The master branch was building fine anyway because checkstyle does not seem to be included in default lifecycle yet, so I'm not sure why this generates so much commotion.

Hmmm, when I execute "mvn -DskipTests=true clean install --projects core -am", it appears as part of the default lifecycle:

Here: https://gist.github.com/galderz/16b126afaf6adb801473ac58c1d5e504#file-gistfile1-txt-L96
Here: https://gist.github.com/galderz/16b126afaf6adb801473ac58c1d5e504#file-gistfile1-txt-L156
And here: https://gist.github.com/galderz/16b126afaf6adb801473ac58c1d5e504#file-gistfile1-txt-L356

> But we should stop being lenient about these checkstyle violations and fail the build from now on. ATM there are no more violations for the currently enabled rules, so it is about time :).

I'm happy for that, as long as the IDEs can quickly fix those. I'm not manually handrolling things to make checkstyle happy :|

> This approach seems to work fine for hibernate-ogm and hibernate-search, and they have tons of style rules. We only have three.
> 
> On 08/16/2016 11:25 PM, Sebastian Laskawiec wrote:
>> I also integrated Adrian's PR with additional fixed (thank you Adrian!): https://github.com/infinispan/infinispan/pull/4510
>> 
>> The master branch builds fine now.
>> 
>> Thanks
>> Sebastian
>> 
>> On Tue, Aug 16, 2016 at 10:16 PM, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>> Hey Galder, Sanne!
>> 
>> The author is no longer anonymous! It was me [1]!
>> 
>> Jokes aside; Galder - may I ask you to double check your configuration? I extracted formatter archive and it seems to be fine [2]. Perhaps you didn't enable it (watch out, the formatter name was changed some time ago from Horizon to Infinispan) or this is some tooling problem (which probably should be reported to Jetbrains). Anyway - could you please double check it? I'll try to catch you tomorrow on IRC and talk about it...
>> 
>> BTW - There are 2 different errors in your output - the first one is the start import which should be avoided. The second one is duplicated import (also should be avoided). Please don't confuse those two.
>> 
>> @Sanne - before you trow a 'git revert' axe on me - may I ask you to import the latest formatter and check if the amount of imports before replacing them with star looks correctly? It looks fine to me but I may have some old installation, old configuration etc...
>> 
>> Thanks
>> Sebastian "the Troublemaker"
>> 
>> [1] https://www.youtube.com/watch?v=a43kowi2ncI
>> [2] unzip IntelliJ_IDEA_Code_Style.jar and look into attached XML. Those two properties are responsible for star imports. 
>> <option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>> <option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>> 
>> 
>> On Tue, Aug 16, 2016 at 8:21 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
>> Please don't forget Eclipse users. It's hard enough to contribute to
>> Infinispan for non-IDEA users; if you enforce specific style rules, at
>> the very least provide compatible formatters for the other IDEs or the
>> barrier for contributors becomes too high.
>> 
>> Galder: to answer your specific question I'd personally say you should
>> revert the offending commit. I don't know who did it nor who merged
>> it, but it's just not nice for others that such things get merged
>> carelessly. Errors can be forgiven, but should be fixed by those who
>> create the mess ;)
>> 
>> On 16 August 2016 at 16:37, Galder Zamarre?o <galder at redhat.com> wrote:
>> > Hi all,
>> >
>> > After integrating [1] I'm getting build errors such as:
>> >
>> > [INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @ infinispan-core ---
>> > [INFO] Starting audit...
>> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/internal/InternalExternalizerTable.java:55: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
>> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/ExternalizerTable.java:76: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
>> > /home/g/0/infinispan/git/core/src/test/java/org/infinispan/filter/CompositeKeyValueFilterConverter.java:11:1: error: Duplicate import to line 10 - org.infinispan.metadata.Metadata.
>> >
>> > Even after installing the latest style for IntelliJ [2], reformatting InternalExternalizerTable.java won't fix those how errors.
>> >
>> > So, what do we do? :(
>> >
>> > Cheers,
>> >
>> > [1] https://github.com/infinispan/infinispan/commit/313b19301055c6267c6f2ea9065a7ab1b68099fe
>> > [2] https://github.com/infinispan/infinispan/blob/master/ide-settings/intellij/IntelliJ_IDEA_Code_Style.jar
>> > --
>> > Galder Zamarre?o
>> > Infinispan, Red Hat
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> 
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From galder at redhat.com  Wed Aug 17 02:40:35 2016
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 17 Aug 2016 08:40:35 +0200
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
	<CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
	<CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
Message-ID: <751CA085-9A83-4219-9C8F-6236950B4F60@redhat.com>

I've double checked the configuration and it looks as you mention. The IDE dialog also shows the values you indicate but reformatting still does not seem to unroll the * import :|

Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat

> On 16 Aug 2016, at 22:16, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> 
> Hey Galder, Sanne!
> 
> The author is no longer anonymous! It was me [1]!
> 
> Jokes aside; Galder - may I ask you to double check your configuration? I extracted formatter archive and it seems to be fine [2]. Perhaps you didn't enable it (watch out, the formatter name was changed some time ago from Horizon to Infinispan) or this is some tooling problem (which probably should be reported to Jetbrains). Anyway - could you please double check it? I'll try to catch you tomorrow on IRC and talk about it...
> 
> BTW - There are 2 different errors in your output - the first one is the start import which should be avoided. The second one is duplicated import (also should be avoided). Please don't confuse those two.
> 
> @Sanne - before you trow a 'git revert' axe on me - may I ask you to import the latest formatter and check if the amount of imports before replacing them with star looks correctly? It looks fine to me but I may have some old installation, old configuration etc...
> 
> Thanks
> Sebastian "the Troublemaker"
> 
> [1] https://www.youtube.com/watch?v=a43kowi2ncI
> [2] unzip IntelliJ_IDEA_Code_Style.jar and look into attached XML. Those two properties are responsible for star imports. 
> <option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
> <option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
> 
> 
> On Tue, Aug 16, 2016 at 8:21 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
> Please don't forget Eclipse users. It's hard enough to contribute to
> Infinispan for non-IDEA users; if you enforce specific style rules, at
> the very least provide compatible formatters for the other IDEs or the
> barrier for contributors becomes too high.
> 
> Galder: to answer your specific question I'd personally say you should
> revert the offending commit. I don't know who did it nor who merged
> it, but it's just not nice for others that such things get merged
> carelessly. Errors can be forgiven, but should be fixed by those who
> create the mess ;)
> 
> On 16 August 2016 at 16:37, Galder Zamarre?o <galder at redhat.com> wrote:
> > Hi all,
> >
> > After integrating [1] I'm getting build errors such as:
> >
> > [INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @ infinispan-core ---
> > [INFO] Starting audit...
> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/internal/InternalExternalizerTable.java:55: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
> > /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/ExternalizerTable.java:76: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
> > /home/g/0/infinispan/git/core/src/test/java/org/infinispan/filter/CompositeKeyValueFilterConverter.java:11:1: error: Duplicate import to line 10 - org.infinispan.metadata.Metadata.
> >
> > Even after installing the latest style for IntelliJ [2], reformatting InternalExternalizerTable.java won't fix those how errors.
> >
> > So, what do we do? :(
> >
> > Cheers,
> >
> > [1] https://github.com/infinispan/infinispan/commit/313b19301055c6267c6f2ea9065a7ab1b68099fe
> > [2] https://github.com/infinispan/infinispan/blob/master/ide-settings/intellij/IntelliJ_IDEA_Code_Style.jar
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From rvansa at redhat.com  Wed Aug 17 03:24:37 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 17 Aug 2016 09:24:37 +0200
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <751CA085-9A83-4219-9C8F-6236950B4F60@redhat.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
	<CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
	<CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
	<751CA085-9A83-4219-9C8F-6236950B4F60@redhat.com>
Message-ID: <57B41135.5060207@redhat.com>

Galder, in Editor/General/Auto import there's Optimize imports on the 
fly. Without that, IDEA won't mess with imports, so make sure it's 
checked if you want to enforce the imports order.

Anyway, I personally think that order of imports (which are hidden by 
IDEs anyway) is not important at all, at it shouldn't be enforced by 
checkstyle. The only trouble is that people have the switch above on 
(it's default in IDEA) - or similiar option in other IDEs - and then IDE 
changes the order and generates fuss in the PR.

Regrettably IDEA does not have per-module coding style setting [1], so 
if you have multiple projects (which use different coding style) in 
single workspace, it gets into the way.

Radim

[1] https://youtrack.jetbrains.com/issue/IDEA-69685

On 08/17/2016 08:40 AM, Galder Zamarre?o wrote:
> I've double checked the configuration and it looks as you mention. The IDE dialog also shows the values you indicate but reformatting still does not seem to unroll the * import :|
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 16 Aug 2016, at 22:16, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>>
>> Hey Galder, Sanne!
>>
>> The author is no longer anonymous! It was me [1]!
>>
>> Jokes aside; Galder - may I ask you to double check your configuration? I extracted formatter archive and it seems to be fine [2]. Perhaps you didn't enable it (watch out, the formatter name was changed some time ago from Horizon to Infinispan) or this is some tooling problem (which probably should be reported to Jetbrains). Anyway - could you please double check it? I'll try to catch you tomorrow on IRC and talk about it...
>>
>> BTW - There are 2 different errors in your output - the first one is the start import which should be avoided. The second one is duplicated import (also should be avoided). Please don't confuse those two.
>>
>> @Sanne - before you trow a 'git revert' axe on me - may I ask you to import the latest formatter and check if the amount of imports before replacing them with star looks correctly? It looks fine to me but I may have some old installation, old configuration etc...
>>
>> Thanks
>> Sebastian "the Troublemaker"
>>
>> [1] https://www.youtube.com/watch?v=a43kowi2ncI
>> [2] unzip IntelliJ_IDEA_Code_Style.jar and look into attached XML. Those two properties are responsible for star imports.
>> <option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>> <option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>>
>>
>> On Tue, Aug 16, 2016 at 8:21 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
>> Please don't forget Eclipse users. It's hard enough to contribute to
>> Infinispan for non-IDEA users; if you enforce specific style rules, at
>> the very least provide compatible formatters for the other IDEs or the
>> barrier for contributors becomes too high.
>>
>> Galder: to answer your specific question I'd personally say you should
>> revert the offending commit. I don't know who did it nor who merged
>> it, but it's just not nice for others that such things get merged
>> carelessly. Errors can be forgiven, but should be fixed by those who
>> create the mess ;)
>>
>> On 16 August 2016 at 16:37, Galder Zamarre?o <galder at redhat.com> wrote:
>>> Hi all,
>>>
>>> After integrating [1] I'm getting build errors such as:
>>>
>>> [INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @ infinispan-core ---
>>> [INFO] Starting audit...
>>> /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/internal/InternalExternalizerTable.java:55: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
>>> /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/ExternalizerTable.java:76: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
>>> /home/g/0/infinispan/git/core/src/test/java/org/infinispan/filter/CompositeKeyValueFilterConverter.java:11:1: error: Duplicate import to line 10 - org.infinispan.metadata.Metadata.
>>>
>>> Even after installing the latest style for IntelliJ [2], reformatting InternalExternalizerTable.java won't fix those how errors.
>>>
>>> So, what do we do? :(
>>>
>>> Cheers,
>>>
>>> [1] https://github.com/infinispan/infinispan/commit/313b19301055c6267c6f2ea9065a7ab1b68099fe
>>> [2] https://github.com/infinispan/infinispan/blob/master/ide-settings/intellij/IntelliJ_IDEA_Code_Style.jar
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From satishbtgsl at gmail.com  Wed Aug 17 04:02:21 2016
From: satishbtgsl at gmail.com (Sathish Kumarbt)
Date: Wed, 17 Aug 2016 08:02:21 +0000
Subject: [infinispan-dev] Replication is not happening with Infinispan 8.2.2
Message-ID: <CAKbqWyOZ8uegoorq90GCe1e8EivQtV5E66w5awfD-La5FpoVpA@mail.gmail.com>

Dear all,

       I am upgrading infinispan version from 6.0 to 8.2.2 version .

Replication between the nodes is not happening .Here is the infinispan
config

<infinispan>

    <jgroups>

        <stack-file name="configurationFile" path="config/jgroups.xml"/>

    </jgroups>

    <cache-container>

        <transport  cluster="x-cluster" stack="configurationFile" />

        <replicated-cache name="transactional-type" mode="SYNC">

            <transaction mode="NON_XA"  locking="OPTIMISTIC"
transaction-manager-lookup="org.infinispan.transaction.lookup.JBossStandaloneJTAManagerLookup"
auto-commit="true"  />

            <locking acquire-timeout="60000"/>

            <expiration lifespan="43200000"/>

        </replicated-cache>

    </cache-container>

</infinispan>



Jgroups configuration



<!--

    TCP based stack, with flow control and message bundling. This is
usually used when IP

    multicasting cannot be used in a network, e.g. because it is disabled
(routers discard multicast).

    Note that TCP.bind_addr and TCPPING.initial_hosts should be set,
possibly via system properties, e.g.

    -Djgroups.bind_addr=192.168.5.2 and
-Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".

    author: Bela Ban

-->

<config xmlns="urn:org:jgroups"

        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

        xsi:schemaLocation="urn:org:jgroups
http://www.jgroups.org/schema/jgroups-3.6.xsd">



  <TCP loopback="true"

       bind_addr="${jgroups.tcp.address:127.0.0.1}"

       bind_port="${jgroups.tcp.port:7800}"

       recv_buf_size="${tcp.recv_buf_size:20M}"

       send_buf_size="${tcp.send_buf_size:640K}"

       discard_incompatible_packets="true"

       max_bundle_size="64K"

       max_bundle_timeout="30"

       enable_bundling="true"

       use_send_queues="true"

       sock_conn_timeout="300"

       timer_type="new"

       timer.min_threads="4"

       timer.max_threads="10"

       timer.keep_alive_time="3000"

       timer.queue_max_size="500"

       thread_pool.enabled="true"

       thread_pool.min_threads="2"

       thread_pool.max_threads="30"

       thread_pool.keep_alive_time="60000"

       thread_pool.queue_enabled="false"

       thread_pool.queue_max_size="100"

       thread_pool.rejection_policy="discard"

       oob_thread_pool.enabled="true"

       oob_thread_pool.min_threads="2"

       oob_thread_pool.max_threads="30"

       oob_thread_pool.keep_alive_time="60000"

       oob_thread_pool.queue_enabled="false"

       oob_thread_pool.queue_max_size="100"

       oob_thread_pool.rejection_policy="discard"/>



    <!-- <TCP_NIO -->

    <!--         bind_port="7800" -->

    <!--         bind_interface="${jgroups.tcp_nio.bind_interface:bond0}"
-->

    <!--         use_send_queues="true" -->

    <!--         sock_conn_timeout="300" -->

    <!--         reader_threads="3" -->

    <!--         writer_threads="3" -->

    <!--         processor_threads="0" -->

    <!--         processor_minThreads="0" -->

    <!--         processor_maxThreads="0" -->

    <!--         processor_queueSize="100" -->

    <!--         processor_keepAliveTime="9223372036854775807"/> -->

    <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>

    <!--    <TCPPING async_discovery="true"
initial_hosts="${jgroups.tcpping.initial_hosts}"

            port_range="2"  timeout="3000" /> -->

    <MERGE2 max_interval="30000" min_interval="10000"/>

    <FD_SOCK/>

    <FD timeout="3000" max_tries="3"/>

    <VERIFY_SUSPECT timeout="1500"/>

    <pbcast.NAKACK

        use_mcast_xmit="false"

        retransmit_timeout="300,600,1200,2400,4800"

        discard_delivered_msgs="false"/>

    <UNICAST2 timeout="300,600,1200"

              stable_interval="5000"

              max_bytes="1m"/>

    <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000"
max_bytes="1m"/>

    <pbcast.GMS print_local_addr="false" join_timeout="3000"
view_bundling="true"/>

    <UFC max_credits="200k" min_threshold="0.20"/>

    <MFC max_credits="200k" min_threshold="0.20"/>

    <FRAG2 frag_size="60000"/>

    <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />

</config>



Regards,

              Sathish.b.t



??K?? ? ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160817/5f2748f9/attachment.html 

From rvansa at redhat.com  Wed Aug 17 04:09:54 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 17 Aug 2016 10:09:54 +0200
Subject: [infinispan-dev] Replication is not happening with Infinispan
 8.2.2
In-Reply-To: <CAKbqWyOZ8uegoorq90GCe1e8EivQtV5E66w5awfD-La5FpoVpA@mail.gmail.com>
References: <CAKbqWyOZ8uegoorq90GCe1e8EivQtV5E66w5awfD-La5FpoVpA@mail.gmail.com>
Message-ID: <57B41BD2.1060400@redhat.com>

Hi Satish,

this mailing list is for developer discussions. We have a nice forum for 
user questions [1], so I recommend posting the problem there, with more 
details (e.g. why do you think that the replication is not happening, 
and logs). Luckily even in Infinispan 8.2.2 we kept the replication 
working, so the problem may be elsewhere.

Radim

[1] https://developer.jboss.org/en/infinispan/


On 08/17/2016 10:02 AM, Sathish Kumarbt wrote:
>
> Dear all,
>
>        I am upgrading infinispan version from 6.0 to 8.2.2 version .
>
> Replication between the nodes is not happening .Here is the infinispan 
> config
>
> <infinispan>
>
>     <jgroups>
>
>         <stack-file name="configurationFile" path="config/jgroups.xml"/>
>
>     </jgroups>
>
>     <cache-container>
>
>         <transport cluster="x-cluster" stack="configurationFile" />
>
> <replicated-cache name="transactional-type" mode="SYNC">
>
>             <transaction mode="NON_XA"  locking="OPTIMISTIC" 
> transaction-manager-lookup="org.infinispan.transaction.lookup.JBossStandaloneJTAManagerLookup" 
> auto-commit="true" />
>
> <locking acquire-timeout="60000"/>
>
> <expiration lifespan="43200000"/>
>
>         </replicated-cache>
>
>     </cache-container>
>
> </infinispan>
>
> Jgroups configuration
>
> <!--
>
>     TCP based stack, with flow control and message bundling. This is 
> usually used when IP
>
>     multicasting cannot be used in a network, e.g. because it is 
> disabled (routers discard multicast).
>
>     Note that TCP.bind_addr and TCPPING.initial_hosts should be set, 
> possibly via system properties, e.g.
>
> -Djgroups.bind_addr=192.168.5.2 and 
> -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>
>     author: Bela Ban
>
> -->
>
> <config xmlns="urn:org:jgroups"
>
>         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>
> xsi:schemaLocation="urn:org:jgroups 
> http://www.jgroups.org/schema/jgroups-3.6.xsd">
>
>   <TCP loopback="true"
>
> bind_addr="${jgroups.tcp.address:127.0.0.1}"
>
> bind_port="${jgroups.tcp.port:7800}"
>
> recv_buf_size="${tcp.recv_buf_size:20M}"
>
> send_buf_size="${tcp.send_buf_size:640K}"
>
> discard_incompatible_packets="true"
>
> max_bundle_size="64K"
>
> max_bundle_timeout="30"
>
> enable_bundling="true"
>
> use_send_queues="true"
>
> sock_conn_timeout="300"
>
> timer_type="new"
>
>        timer.min_threads="4"
>
> timer.max_threads="10"
>
> timer.keep_alive_time="3000"
>
> timer.queue_max_size="500"
>
> thread_pool.enabled="true"
>
> thread_pool.min_threads="2"
>
> thread_pool.max_threads="30"
>
> thread_pool.keep_alive_time="60000"
>
> thread_pool.queue_enabled="false"
>
> thread_pool.queue_max_size="100"
>
> thread_pool.rejection_policy="discard"
>
>   oob_thread_pool.enabled="true"
>
> oob_thread_pool.min_threads="2"
>
> oob_thread_pool.max_threads="30"
>
> oob_thread_pool.keep_alive_time="60000"
>
> oob_thread_pool.queue_enabled="false"
>
> oob_thread_pool.queue_max_size="100"
>
> oob_thread_pool.rejection_policy="discard"/>
>
>     <!-- <TCP_NIO -->
>
> <!-- bind_port="7800" -->
>
>     <!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>
> <!-- use_send_queues="true" -->
>
> <!-- sock_conn_timeout="300" -->
>
> <!-- reader_threads="3" -->
>
> <!-- writer_threads="3" -->
>
> <!-- processor_threads="0" -->
>
> <!-- processor_minThreads="0" -->
>
> <!-- processor_maxThreads="0" -->
>
> <!-- processor_queueSize="100" -->
>
> <!-- processor_keepAliveTime="9223372036854775807"/> -->
>
>     <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>
>     <!--    <TCPPING async_discovery="true" 
> initial_hosts="${jgroups.tcpping.initial_hosts}"
>
>             port_range="2" timeout="3000" /> -->
>
>     <MERGE2 max_interval="30000" min_interval="10000"/>
>
>     <FD_SOCK/>
>
>     <FD timeout="3000" max_tries="3"/>
>
>     <VERIFY_SUSPECT timeout="1500"/>
>
>     <pbcast.NAKACK
>
> use_mcast_xmit="false"
>
>     retransmit_timeout="300,600,1200,2400,4800"
>
> discard_delivered_msgs="false"/>
>
>     <UNICAST2 timeout="300,600,1200"
>
> stable_interval="5000"
>
> max_bytes="1m"/>
>
>     <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" 
> max_bytes="1m"/>
>
>     <pbcast.GMS print_local_addr="false" join_timeout="3000" 
> view_bundling="true"/>
>
>     <UFC max_credits="200k" min_threshold="0.20"/>
>
>     <MFC max_credits="200k" min_threshold="0.20"/>
>
>     <FRAG2 frag_size="60000"/>
>
>     <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />
>
> </config>
>
> Regards,
>
> Sathish.b.t
>
> ??K?? ? ?
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From sanne at infinispan.org  Wed Aug 17 05:00:06 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 17 Aug 2016 10:00:06 +0100
Subject: [infinispan-dev] Style errors :(
In-Reply-To: <57B41135.5060207@redhat.com>
References: <69EB565C-3FEC-4F10-A3D6-9A044EF9240B@redhat.com>
	<CAFm4XO0enta7rCoFKi2G6ftKxf_d3hBV=XRBtv5szPXknPPMyQ@mail.gmail.com>
	<CAA9R9O6cXR-0eqDb0TvV-t+oMmQddMVr2mcq2Q29RGB7X=nsTQ@mail.gmail.com>
	<751CA085-9A83-4219-9C8F-6236950B4F60@redhat.com>
	<57B41135.5060207@redhat.com>
Message-ID: <CAFm4XO3UdYWHRQzj37LpYe2n=JL4wJ4YB9e_pvfAOpTTen6x1A@mail.gmail.com>

On 17 August 2016 at 08:24, Radim Vansa <rvansa at redhat.com> wrote:
> Galder, in Editor/General/Auto import there's Optimize imports on the
> fly. Without that, IDEA won't mess with imports, so make sure it's
> checked if you want to enforce the imports order.
>
> Anyway, I personally think that order of imports (which are hidden by
> IDEs anyway) is not important at all, at it shouldn't be enforced by
> checkstyle. The only trouble is that people have the switch above on
> (it's default in IDEA) - or similiar option in other IDEs - and then IDE
> changes the order and generates fuss in the PR.

+1 better tread carefully, I predict a lot of pain here.

And since you all mentioned Hibernate Search as an example of were it
works, let me share some hints:
some rules are important as for example importing "*" has consequences
on determinism, so there's a small subset of rules which I'd highly
recommend.
Another example is that we actively use checsstyle rules to ban usage
of types which should never be used, e.g. mistakenly use the JGroups
assertions instead of a JUnit assertion:
https://github.com/hibernate/hibernate-search/blob/master/build-config/src/main/resources/checkstyle.xml#L173-L175

Then we also enforce our "style", but be aware that these are
controversial and often seen as a PITA by contributors so I'd feel the
need to stress that the important value is not really about
maintaining our weird style but rather about making sure that changes
in PR "stick to the point" and don't change anything else, this is
important to ease porting and rebasing of patches across branches.
In short, it's designed to _prevent_ people from sending PRs with
massive, automated or not, source code reformatting.

In conclusion, messing with import order seems a lot of pain and will
make your future rebases and backports a huge pain. I'd advise against
changes to the status quo but rather enforce such rules on
new/rewritten modules, or other cases in which you already know that
backports are not going to be likely.

Finally, be aware that some style rules can't possibly be enforced in
the same way across IDEs. Import order is one of the tricky ones so
unless you provide rules which you've tested and working you're being
very annoying to contributors.

I'm not against introducing it at all, I did the initial contribution
;) But be careful on what you enable, on which module, and when, and
ultimately what's the effect you want to achieve.

>
> Regrettably IDEA does not have per-module coding style setting [1], so
> if you have multiple projects (which use different coding style) in
> single workspace, it gets into the way.
>
> Radim
>
> [1] https://youtrack.jetbrains.com/issue/IDEA-69685
>
> On 08/17/2016 08:40 AM, Galder Zamarre?o wrote:
>> I've double checked the configuration and it looks as you mention. The IDE dialog also shows the values you indicate but reformatting still does not seem to unroll the * import :|
>>
>> Cheers,
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>> On 16 Aug 2016, at 22:16, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>>>
>>> Hey Galder, Sanne!
>>>
>>> The author is no longer anonymous! It was me [1]!
>>>
>>> Jokes aside; Galder - may I ask you to double check your configuration? I extracted formatter archive and it seems to be fine [2]. Perhaps you didn't enable it (watch out, the formatter name was changed some time ago from Horizon to Infinispan) or this is some tooling problem (which probably should be reported to Jetbrains). Anyway - could you please double check it? I'll try to catch you tomorrow on IRC and talk about it...
>>>
>>> BTW - There are 2 different errors in your output - the first one is the start import which should be avoided. The second one is duplicated import (also should be avoided). Please don't confuse those two.
>>>
>>> @Sanne - before you trow a 'git revert' axe on me - may I ask you to import the latest formatter and check if the amount of imports before replacing them with star looks correctly? It looks fine to me but I may have some old installation, old configuration etc...
>>>
>>> Thanks
>>> Sebastian "the Troublemaker"
>>>
>>> [1] https://www.youtube.com/watch?v=a43kowi2ncI
>>> [2] unzip IntelliJ_IDEA_Code_Style.jar and look into attached XML. Those two properties are responsible for star imports.
>>> <option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>>> <option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="999" />
>>>
>>>
>>> On Tue, Aug 16, 2016 at 8:21 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
>>> Please don't forget Eclipse users. It's hard enough to contribute to
>>> Infinispan for non-IDEA users; if you enforce specific style rules, at
>>> the very least provide compatible formatters for the other IDEs or the
>>> barrier for contributors becomes too high.
>>>
>>> Galder: to answer your specific question I'd personally say you should
>>> revert the offending commit. I don't know who did it nor who merged
>>> it, but it's just not nice for others that such things get merged
>>> carelessly. Errors can be forgiven, but should be fixed by those who
>>> create the mess ;)
>>>
>>> On 16 August 2016 at 16:37, Galder Zamarre?o <galder at redhat.com> wrote:
>>>> Hi all,
>>>>
>>>> After integrating [1] I'm getting build errors such as:
>>>>
>>>> [INFO] --- maven-checkstyle-plugin:2.17:checkstyle (checkstyle) @ infinispan-core ---
>>>> [INFO] Starting audit...
>>>> /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/internal/InternalExternalizerTable.java:55: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
>>>> /home/g/0/infinispan/git/core/src/main/java/org/infinispan/marshall/core/ExternalizerTable.java:76: error: Using the '.*' form of import should be avoided - org.infinispan.marshall.exts.*.
>>>> /home/g/0/infinispan/git/core/src/test/java/org/infinispan/filter/CompositeKeyValueFilterConverter.java:11:1: error: Duplicate import to line 10 - org.infinispan.metadata.Metadata.
>>>>
>>>> Even after installing the latest style for IntelliJ [2], reformatting InternalExternalizerTable.java won't fix those how errors.
>>>>
>>>> So, what do we do? :(
>>>>
>>>> Cheers,
>>>>
>>>> [1] https://github.com/infinispan/infinispan/commit/313b19301055c6267c6f2ea9065a7ab1b68099fe
>>>> [2] https://github.com/infinispan/infinispan/blob/master/ide-settings/intellij/IntelliJ_IDEA_Code_Style.jar
>>>> --
>>>> Galder Zamarre?o
>>>> Infinispan, Red Hat
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From satishbtgsl at gmail.com  Wed Aug 17 05:25:16 2016
From: satishbtgsl at gmail.com (Sathish Kumarbt)
Date: Wed, 17 Aug 2016 09:25:16 +0000
Subject: [infinispan-dev] Replication is not happening with Infinispan
	8.2.2
In-Reply-To: <57B41BD2.1060400@redhat.com>
References: <CAKbqWyOZ8uegoorq90GCe1e8EivQtV5E66w5awfD-La5FpoVpA@mail.gmail.com>
	<57B41BD2.1060400@redhat.com>
Message-ID: <CAKbqWyNzpWf--8yGCHYPDgpHFCen8fQ65C-q5JuqmPTH4CzYVw@mail.gmail.com>

Hi Radim,
      I need below answer
1.Do we need to enable multicast in the network where nodes are deployed.
2.We are using TCPGOSSIP router for clustering communication
3.How can i create TCPGOSSIP router communication with my local desktop.

Any help is appreciated .

regards,
 sathish.b.t

On Wed, 17 Aug 2016 at 13:40 Radim Vansa <rvansa at redhat.com> wrote:

> Hi Satish,
>
> this mailing list is for developer discussions. We have a nice forum for
> user questions [1], so I recommend posting the problem there, with more
> details (e.g. why do you think that the replication is not happening,
> and logs). Luckily even in Infinispan 8.2.2 we kept the replication
> working, so the problem may be elsewhere.
>
> Radim
>
> [1] https://developer.jboss.org/en/infinispan/
>
>
> On 08/17/2016 10:02 AM, Sathish Kumarbt wrote:
> >
> > Dear all,
> >
> >        I am upgrading infinispan version from 6.0 to 8.2.2 version .
> >
> > Replication between the nodes is not happening .Here is the infinispan
> > config
> >
> > <infinispan>
> >
> >     <jgroups>
> >
> >         <stack-file name="configurationFile" path="config/jgroups.xml"/>
> >
> >     </jgroups>
> >
> >     <cache-container>
> >
> >         <transport cluster="x-cluster" stack="configurationFile" />
> >
> > <replicated-cache name="transactional-type" mode="SYNC">
> >
> >             <transaction mode="NON_XA"  locking="OPTIMISTIC"
> >
> transaction-manager-lookup="org.infinispan.transaction.lookup.JBossStandaloneJTAManagerLookup"
> > auto-commit="true" />
> >
> > <locking acquire-timeout="60000"/>
> >
> > <expiration lifespan="43200000"/>
> >
> >         </replicated-cache>
> >
> >     </cache-container>
> >
> > </infinispan>
> >
> > Jgroups configuration
> >
> > <!--
> >
> >     TCP based stack, with flow control and message bundling. This is
> > usually used when IP
> >
> >     multicasting cannot be used in a network, e.g. because it is
> > disabled (routers discard multicast).
> >
> >     Note that TCP.bind_addr and TCPPING.initial_hosts should be set,
> > possibly via system properties, e.g.
> >
> > -Djgroups.bind_addr=192.168.5.2 and
> > -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
> >
> >     author: Bela Ban
> >
> > -->
> >
> > <config xmlns="urn:org:jgroups"
> >
> >         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >
> > xsi:schemaLocation="urn:org:jgroups
> > http://www.jgroups.org/schema/jgroups-3.6.xsd">
> >
> >   <TCP loopback="true"
> >
> > bind_addr="${jgroups.tcp.address:127.0.0.1}"
> >
> > bind_port="${jgroups.tcp.port:7800}"
> >
> > recv_buf_size="${tcp.recv_buf_size:20M}"
> >
> > send_buf_size="${tcp.send_buf_size:640K}"
> >
> > discard_incompatible_packets="true"
> >
> > max_bundle_size="64K"
> >
> > max_bundle_timeout="30"
> >
> > enable_bundling="true"
> >
> > use_send_queues="true"
> >
> > sock_conn_timeout="300"
> >
> > timer_type="new"
> >
> >        timer.min_threads="4"
> >
> > timer.max_threads="10"
> >
> > timer.keep_alive_time="3000"
> >
> > timer.queue_max_size="500"
> >
> > thread_pool.enabled="true"
> >
> > thread_pool.min_threads="2"
> >
> > thread_pool.max_threads="30"
> >
> > thread_pool.keep_alive_time="60000"
> >
> > thread_pool.queue_enabled="false"
> >
> > thread_pool.queue_max_size="100"
> >
> > thread_pool.rejection_policy="discard"
> >
> >   oob_thread_pool.enabled="true"
> >
> > oob_thread_pool.min_threads="2"
> >
> > oob_thread_pool.max_threads="30"
> >
> > oob_thread_pool.keep_alive_time="60000"
> >
> > oob_thread_pool.queue_enabled="false"
> >
> > oob_thread_pool.queue_max_size="100"
> >
> > oob_thread_pool.rejection_policy="discard"/>
> >
> >     <!-- <TCP_NIO -->
> >
> > <!-- bind_port="7800" -->
> >
> >     <!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
> >
> > <!-- use_send_queues="true" -->
> >
> > <!-- sock_conn_timeout="300" -->
> >
> > <!-- reader_threads="3" -->
> >
> > <!-- writer_threads="3" -->
> >
> > <!-- processor_threads="0" -->
> >
> > <!-- processor_minThreads="0" -->
> >
> > <!-- processor_maxThreads="0" -->
> >
> > <!-- processor_queueSize="100" -->
> >
> > <!-- processor_keepAliveTime="9223372036854775807"/> -->
> >
> >     <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
> >
> >     <!--    <TCPPING async_discovery="true"
> > initial_hosts="${jgroups.tcpping.initial_hosts}"
> >
> >             port_range="2" timeout="3000" /> -->
> >
> >     <MERGE2 max_interval="30000" min_interval="10000"/>
> >
> >     <FD_SOCK/>
> >
> >     <FD timeout="3000" max_tries="3"/>
> >
> >     <VERIFY_SUSPECT timeout="1500"/>
> >
> >     <pbcast.NAKACK
> >
> > use_mcast_xmit="false"
> >
> >     retransmit_timeout="300,600,1200,2400,4800"
> >
> > discard_delivered_msgs="false"/>
> >
> >     <UNICAST2 timeout="300,600,1200"
> >
> > stable_interval="5000"
> >
> > max_bytes="1m"/>
> >
> >     <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000"
> > max_bytes="1m"/>
> >
> >     <pbcast.GMS print_local_addr="false" join_timeout="3000"
> > view_bundling="true"/>
> >
> >     <UFC max_credits="200k" min_threshold="0.20"/>
> >
> >     <MFC max_credits="200k" min_threshold="0.20"/>
> >
> >     <FRAG2 frag_size="60000"/>
> >
> >     <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false"
> />
> >
> > </config>
> >
> > Regards,
> >
> > Sathish.b.t
> >
> > ??K?? ? ?
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160817/23277eb8/attachment.html 

From slaskawi at redhat.com  Wed Aug 17 06:20:10 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 17 Aug 2016 12:20:10 +0200
Subject: [infinispan-dev] HTTP/2 Upgrade [0] ideas and thoughts
Message-ID: <CAA9R9O4ob+06G0N-E-URyRuPgm=78fH9-yQ6nQd8X-=Pv2qsPw@mail.gmail.com>

Hey!

I started sketching some ideas how HTTP/2 client and the upgrade procedure
could work in cloud environment. Before digging in, let me explain some
basic concepts and facts:

   - Kubernetes as well as OpenShift operate on a very simple architecture.
   We have a group of Pods (Docker Containers), a Service which acts as a load
   balancer and a Route (which acts as a proxy for serving requests to the
   outside world).
   - Communication between components deployed on the same
   Kubernetes/OpenShift cluster looks like this: MyApp -> Service (target app)
   -> one of the Pods
   - Communication from the outside world looks like this: MyApp -> the
   Internet -> Route (target app) -> Service (target app) -> one of the Pods
   [1]
   - Currently Kubernetes/OpenShift Services use round-robin strategy for
   load balancing, can use Client IP affinity or can use HTTP Cookies for
   session stickiness.
   - OpenShift Route (or Kubernetes Ingress) can support TLS. They can
   downgrade HTTPS to HTTP (in other words terminate it), pass through an
   encrypted request without inspecting the content or reencrypt it with
   different certificate [2].
   - HTTP/2 does not have upgrade header. It uses TLS with ALPN to
   negotiate which protocol should be used [3].
   - HTTP/2 can support custom protocols (which allows writing custom
   Frames, Settings and Error Codes) [4].

The initial idea of using HTTP 1.1/Upgrade (as I mentioned above HTTP/2
doesn't have this concept, it uses ALPN) is to support Hot Rod Clients from
the outside of the Kubernetes/OpenShift cluster. The client connects to a
random Pod (through w Route) using HTTP Protocol and upgrades the
connection to Hot Rod Protocol.

After thinking about it for a while, several things don't fit. A Hot Rod
client uses topology information to minimize the amount of hops. When we
access the data using a Route (or Ingress) and a Service we don't control
to which Pod we're connecting to. Moreover, if we switch from HTTP to Hot
Rod protocol (which is based on TCP), we lose HTTP Headers which could be
possible used for routing inside Kubernetes/OpenShift. Switching protocols
is also problematic since HTTP/2 does not support upgrade header (as I
mentioned above - it uses ALPN). ALPN support needs to be implemented in
Hot Rod Server (this is the only component which has enough data to say
which protocols are supported; OpenShift Routes or Kunernetes Ingresses
don't have this knowledge). This means that Routes and Services sees
encrypted traffic and won't be able to help us with Routing.

So what can we do about it? There are a couple of ideas how to solve those
problems.

The first one is to enhance the Hot Rod Client to initialize a connection
pool. The client could periodically initialize a new connection and send
PING operation. If the connection is already in the pool - close it.
Otherwise add it to the pool. We can call it brute-force-cluster-discovery
:) It should be good for any round-robin like load-balancers.

The second idea is to implement a fully fledged HTTP/2 client for Hot Rod
(@Anton - I think you're working on that aren't you?). We could use HTTP
Headers to control to which Pod we are connecting to (this would require
adding some code to Kubernetes/OpenShift but it shouldn't be very hard).
After TLS Handshake we could use topology information and HTTP Headers to
initialize connection to all cluster members. In this scenario ALPN won't
be needed (since we will implement separate server and client) which is
firewall friendly (operates on HTTP/HTTPS ports).

Having in mind the raise of stateful apps in the cloud we could also
propose PetSets enhancement and donate extra code for supporting them from
Ingress/Route perspective. The result should be similar to the previous
option (donating some code to Services/Ingresses and Routes).

The above are only some ideas how all the pieces could work together. I'll
also consult it with the OpenShift Team - maybe they will give us some more
hints.

Think about it for a while and let me know if you have any ideas...

Thanks
Sebastian

[0] https://issues.jboss.org/browse/ISPN-6676
[1]
http://www.slideshare.net/SamuelTerburg/open-shift-enterprise-31-paas-on-kubernetes
[2]
https://docs.openshift.org/latest/architecture/core_concepts/routes.html#secured-routes
[3] https://http2.github.io/http2-spec/#rfc.section.8.1.2.2
[4] https://http2.github.io/http2-spec/#rfc.section.5.5
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160817/08f86556/attachment-0001.html 

From sanne at infinispan.org  Thu Aug 18 08:43:32 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 18 Aug 2016 13:43:32 +0100
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
	or even enforce it
Message-ID: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>

Hi all,

I would like for Infinispan to use the AffinityPartitioner by default,
replacing the HashFunctionPartitioner.

This should have zero impact as AffinityPartitioner extends
HashFunctionPartitioner and only changes semantics when a key is
implementing AffinityTaggedKey.

So the difference would be that for those using AffinityTaggedKey,
these would work out of the box without having to change the
configuration as well.

WDYT?

As a further but separate improvement, I'd change the
AffinityPartitioner to use delegation to the HashFunctionPartitioner
instead of extending it, and always wrap any user configured
partitioner with the AffinityPartitioner.
This would ensure that AffinityTaggedKey work as expectedf even when
people experiment with other partitioners, and avoids some complexity
of configuring Infinispan:

"oh, I didn't know that changing hashing function would break feature [x]..."

For the record, we're using AffinityTaggedKey in our evil plans to
improve query performance, but it's also sparked interest as a very
useful function by the HACEP team.

I have patches ready...

Thanks,
Sanne

From dan.berindei at gmail.com  Thu Aug 18 09:44:16 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 18 Aug 2016 16:44:16 +0300
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
 or even enforce it
In-Reply-To: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
References: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
Message-ID: <CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>

First, a question: doesn't query already configure its internal caches
to use the AffinityPartitioner? Does it need the application caches to
have AffinityPartitioner enabled, too?

I'm ok with making AffinityPartitioner the default key partitioner:
the less configuration the user has to change to make things work, the
better. I'm also ok with changing it to use delegation, to make it
easy for user to compose it with another key partitioner. We should
just run some benchmarks first to check it doesn't affect performance
for repl mode reads.

But I feel users should be able to disable it if they want to, so I
wouldn't always wrap user partitioners. Also, usually less magic ==
better. Query can validate the configuration and warn the user if the
configuration doesn't have AffinityPartitioner in place.

Cheers
Dan


On Thu, Aug 18, 2016 at 3:43 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
> Hi all,
>
> I would like for Infinispan to use the AffinityPartitioner by default,
> replacing the HashFunctionPartitioner.
>
> This should have zero impact as AffinityPartitioner extends
> HashFunctionPartitioner and only changes semantics when a key is
> implementing AffinityTaggedKey.
>
> So the difference would be that for those using AffinityTaggedKey,
> these would work out of the box without having to change the
> configuration as well.
>
> WDYT?
>
> As a further but separate improvement, I'd change the
> AffinityPartitioner to use delegation to the HashFunctionPartitioner
> instead of extending it, and always wrap any user configured
> partitioner with the AffinityPartitioner.
> This would ensure that AffinityTaggedKey work as expectedf even when
> people experiment with other partitioners, and avoids some complexity
> of configuring Infinispan:
>
> "oh, I didn't know that changing hashing function would break feature [x]..."
>
> For the record, we're using AffinityTaggedKey in our evil plans to
> improve query performance, but it's also sparked interest as a very
> useful function by the HACEP team.
>
> I have patches ready...
>
> Thanks,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From sanne at infinispan.org  Thu Aug 18 11:12:22 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 18 Aug 2016 16:12:22 +0100
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
 or even enforce it
In-Reply-To: <CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>
References: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
	<CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>
Message-ID: <CAFm4XO1sfqg8J-4U+UCr8m3YznTWcdj7cY1wxFTPA1meMC6MrA@mail.gmail.com>

On 18 August 2016 at 14:44, Dan Berindei <dan.berindei at gmail.com> wrote:
> First, a question: doesn't query already configure its internal caches
> to use the AffinityPartitioner? Does it need the application caches to
> have AffinityPartitioner enabled, too?

No, we may provide reasonable defaults so that demos and tutorials
work out of the box but the general recommendation is that people make
their own configuration, as they will need to tune things, or add
CacheStores, etc..

> I'm ok with making AffinityPartitioner the default key partitioner:
> the less configuration the user has to change to make things work, the
> better. I'm also ok with changing it to use delegation, to make it
> easy for user to compose it with another key partitioner. We should
> just run some benchmarks first to check it doesn't affect performance
> for repl mode reads.

Ok!
https://issues.jboss.org/browse/ISPN-6956

I'll send a PR; regarding benchmarking I'll take the optimistic path:
we'll test it after it's integrated, worst case we can take it out.

> But I feel users should be able to disable it if they want to, so I
> wouldn't always wrap user partitioners. Also, usually less magic ==
> better. Query can validate the configuration and warn the user if the
> configuration doesn't have AffinityPartitioner in place.

I disagree on this as this would require more options, which make it
harder to use, and there's no good enough reason to give away the
option to disable this; especially as it kills other features.

BTW I never mentioned Query :)
We happen to use this functionality in Query but as I mentioned other
teams are looking forward to use this functionality, so I think we
should promote this as a public API, a new feature of Infinispan core.

So since you want to be able to disable it I won't wrap all user
configured implementations in my PR for ISPN-6956, but I'd prefer it
if as a second step you would reconsider and allow me to wrap them
all.

Thanks!
Sanne

>
> Cheers
> Dan
>
>
> On Thu, Aug 18, 2016 at 3:43 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
>> Hi all,
>>
>> I would like for Infinispan to use the AffinityPartitioner by default,
>> replacing the HashFunctionPartitioner.
>>
>> This should have zero impact as AffinityPartitioner extends
>> HashFunctionPartitioner and only changes semantics when a key is
>> implementing AffinityTaggedKey.
>>
>> So the difference would be that for those using AffinityTaggedKey,
>> these would work out of the box without having to change the
>> configuration as well.
>>
>> WDYT?
>>
>> As a further but separate improvement, I'd change the
>> AffinityPartitioner to use delegation to the HashFunctionPartitioner
>> instead of extending it, and always wrap any user configured
>> partitioner with the AffinityPartitioner.
>> This would ensure that AffinityTaggedKey work as expectedf even when
>> people experiment with other partitioners, and avoids some complexity
>> of configuring Infinispan:
>>
>> "oh, I didn't know that changing hashing function would break feature [x]..."
>>
>> For the record, we're using AffinityTaggedKey in our evil plans to
>> improve query performance, but it's also sparked interest as a very
>> useful function by the HACEP team.
>>
>> I have patches ready...
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From slaskawi at redhat.com  Fri Aug 19 05:00:12 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Fri, 19 Aug 2016 11:00:12 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
Message-ID: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>

Hey!

I've been playing with Kubernetes PetSets [1] for a while and I'd like to
share some thoughts. Before I dig in, let me give you some PetSets
highlights:

   - PetSets are alpha resources for managing stateful apps in Kubernetes
   1.3 (and OpenShift Origin 1.3).
   - Since this is an alpha resource, there are no guarantees about
   backwards compatibility. Alpha resources can also be disabled in some
   public cloud providers (you can control which API versions are accessible
   [2]).
   - PetSets allows starting pods in sequence (not relevant for us, but
   this is a killer feature for master-slave systems).
   - Each Pod has it's own unique entry in DNS, which makes discovery very
   simple (I'll dig into that a bit later)
   - Volumes are always mounted to the same Pods, which is very important
   in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades [3]).

Thoughts and ideas after spending some time playing with this feature:

   - PetSets make discovery a lot easier. It's a combination of two things
   - Headless Services [4] which create multiple A records in DNS and
   predictable host names. Each Pod has it's own unique DNS entry following
   pattern: {PetSetName}-{PodIndex}.{ServiceName} [5]. Here's an example of an
   Infinispan PetSet deployed on my local cluster [6]. As you can see we have
   all domain names and IPs from a single DNS query.
   - Maybe we could perform discovery using this mechanism? I'm aware of
   DNS discovery implemented in KUBE_PING [7][8] but the code looks trivial
   [9] so maybe it should be implement inside JGroups? @Bela - WDYT?
   - PetSets do not integrate well with OpenShift 'new-app' command. In
   other words, our users will need to use provided yaml (or json) files to
   create Infinispan cluster. It's not a show-stopper but it's a bit less
   convenient than 'oc new-app'.
   - Since PetSets are alpha resources they need to be considered as
   secondary way to deploy Infinispan on Kubernetes and OpenShift.
   - Finally, the persistent volumes - since a Pod always gets the same
   volume, it would be safe to use any file-based cache store.

If you'd like to play with PetSets on your local environment, here are
necessary yaml files [10].

Thanks
Sebastian


[1] http://kubernetes.io/docs/user-guide/petset/
[2] For checking which APIs are accessible, use 'kubectl api-versions'
[3]
http://infinispan.org/docs/stable/user_guide/user_guide.html#_Rolling_chapter
[4] http://kubernetes.io/docs/user-guide/services/#headless-services
[5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
[6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
[7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
[8] https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
[9] http://stackoverflow.com/a/12405896/562699
[10] You might need to adjust ImageStream.
https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160819/4b0cfef0/attachment.html 

From bban at redhat.com  Fri Aug 19 05:18:19 2016
From: bban at redhat.com (Bela Ban)
Date: Fri, 19 Aug 2016 11:18:19 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
Message-ID: <57B6CEDB.1090700@redhat.com>

Hi Sebastian

the usual restrictions apply: if DNS discovery depends on external libs, 
then it should be hosted in jgroups-extras, otherwise we can add it to 
JGroups itself.

On 19/08/16 11:00, Sebastian Laskawiec wrote:
> Hey!
>
> I've been playing with Kubernetes PetSets [1] for a while and I'd like
> to share some thoughts. Before I dig in, let me give you some PetSets
> highlights:
>
>   * PetSets are alpha resources for managing stateful apps in Kubernetes
>     1.3 (and OpenShift Origin 1.3).
>   * Since this is an alpha resource, there are no guarantees about
>     backwards compatibility. Alpha resources can also be disabled in
>     some public cloud providers (you can control which API versions are
>     accessible [2]).
>   * PetSets allows starting pods in sequence (not relevant for us, but
>     this is a killer feature for master-slave systems).
>   * Each Pod has it's own unique entry in DNS, which makes discovery
>     very simple (I'll dig into that a bit later)
>   * Volumes are always mounted to the same Pods, which is very important
>     in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades
>     [3]).
>
> Thoughts and ideas after spending some time playing with this feature:
>
>   * PetSets make discovery a lot easier. It's a combination of two
>     things - Headless Services [4] which create multiple A records in
>     DNS and predictable host names. Each Pod has it's own unique DNS
>     entry following pattern: {PetSetName}-{PodIndex}.{ServiceName} [5].
>     Here's an example of an Infinispan PetSet deployed on my local
>     cluster [6]. As you can see we have all domain names and IPs from a
>     single DNS query.
>   * Maybe we could perform discovery using this mechanism? I'm aware of
>     DNS discovery implemented in KUBE_PING [7][8] but the code looks
>     trivial [9] so maybe it should be implement inside JGroups? @Bela -
>     WDYT?
>   * PetSets do not integrate well with OpenShift 'new-app' command. In
>     other words, our users will need to use provided yaml (or json)
>     files to create Infinispan cluster. It's not a show-stopper but it's
>     a bit less convenient than 'oc new-app'.
>   * Since PetSets are alpha resources they need to be considered as
>     secondary way to deploy Infinispan on Kubernetes and OpenShift.
>   * Finally, the persistent volumes - since a Pod always gets the same
>     volume, it would be safe to use any file-based cache store.
>
> If you'd like to play with PetSets on your local environment, here are
> necessary yaml files [10].
>
> Thanks
> Sebastian
>
>
> [1] http://kubernetes.io/docs/user-guide/petset/
> [2] For checking which APIs are accessible, use 'kubectl api-versions'
> [3]
> http://infinispan.org/docs/stable/user_guide/user_guide.html#_Rolling_chapter
> [4] http://kubernetes.io/docs/user-guide/services/#headless-services
> [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
> [6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
> [7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
> [8] https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
> [9] http://stackoverflow.com/a/12405896/562699
> [10] You might need to adjust ImageStream.
> https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From slaskawi at redhat.com  Fri Aug 19 05:21:57 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Fri, 19 Aug 2016 11:21:57 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <57B6CEDB.1090700@redhat.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<57B6CEDB.1090700@redhat.com>
Message-ID: <CAA9R9O7cDnsx3PBnhxZ58ZQp0x6TOp+Orq2TUFsrPyyG5ZYWzQ@mail.gmail.com>

Hey Bela!

No no, the resolution can be done with pure JDK.

Thanks
Sebastian

On Fri, Aug 19, 2016 at 11:18 AM, Bela Ban <bban at redhat.com> wrote:

> Hi Sebastian
>
> the usual restrictions apply: if DNS discovery depends on external libs,
> then it should be hosted in jgroups-extras, otherwise we can add it to
> JGroups itself.
>
> On 19/08/16 11:00, Sebastian Laskawiec wrote:
>
>> Hey!
>>
>> I've been playing with Kubernetes PetSets [1] for a while and I'd like
>> to share some thoughts. Before I dig in, let me give you some PetSets
>> highlights:
>>
>>   * PetSets are alpha resources for managing stateful apps in Kubernetes
>>     1.3 (and OpenShift Origin 1.3).
>>   * Since this is an alpha resource, there are no guarantees about
>>     backwards compatibility. Alpha resources can also be disabled in
>>     some public cloud providers (you can control which API versions are
>>     accessible [2]).
>>   * PetSets allows starting pods in sequence (not relevant for us, but
>>     this is a killer feature for master-slave systems).
>>   * Each Pod has it's own unique entry in DNS, which makes discovery
>>     very simple (I'll dig into that a bit later)
>>   * Volumes are always mounted to the same Pods, which is very important
>>     in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades
>>     [3]).
>>
>> Thoughts and ideas after spending some time playing with this feature:
>>
>>   * PetSets make discovery a lot easier. It's a combination of two
>>     things - Headless Services [4] which create multiple A records in
>>     DNS and predictable host names. Each Pod has it's own unique DNS
>>     entry following pattern: {PetSetName}-{PodIndex}.{ServiceName} [5].
>>     Here's an example of an Infinispan PetSet deployed on my local
>>     cluster [6]. As you can see we have all domain names and IPs from a
>>     single DNS query.
>>   * Maybe we could perform discovery using this mechanism? I'm aware of
>>     DNS discovery implemented in KUBE_PING [7][8] but the code looks
>>     trivial [9] so maybe it should be implement inside JGroups? @Bela -
>>     WDYT?
>>   * PetSets do not integrate well with OpenShift 'new-app' command. In
>>     other words, our users will need to use provided yaml (or json)
>>     files to create Infinispan cluster. It's not a show-stopper but it's
>>     a bit less convenient than 'oc new-app'.
>>   * Since PetSets are alpha resources they need to be considered as
>>     secondary way to deploy Infinispan on Kubernetes and OpenShift.
>>   * Finally, the persistent volumes - since a Pod always gets the same
>>     volume, it would be safe to use any file-based cache store.
>>
>> If you'd like to play with PetSets on your local environment, here are
>> necessary yaml files [10].
>>
>> Thanks
>> Sebastian
>>
>>
>> [1] http://kubernetes.io/docs/user-guide/petset/
>> [2] For checking which APIs are accessible, use 'kubectl api-versions'
>> [3]
>> http://infinispan.org/docs/stable/user_guide/user_guide.html
>> #_Rolling_chapter
>> [4] http://kubernetes.io/docs/user-guide/services/#headless-services
>> [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
>> [6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
>> [7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
>> [8] https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
>> [9] http://stackoverflow.com/a/12405896/562699
>> [10] You might need to adjust ImageStream.
>> https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0
>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160819/44ebab24/attachment.html 

From gustavo at infinispan.org  Fri Aug 19 05:27:13 2016
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Fri, 19 Aug 2016 10:27:13 +0100
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <57B6CEDB.1090700@redhat.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<57B6CEDB.1090700@redhat.com>
Message-ID: <CAH8Ud1HtV-waeTSCLb7-t5mqOZHEOrv+R1fr42aam03D0hpb9Q@mail.gmail.com>

On Fri, Aug 19, 2016 at 10:18 AM, Bela Ban <bban at redhat.com> wrote:

> Hi Sebastian
>
> the usual restrictions apply: if DNS discovery depends on external libs,
> then it should be hosted in jgroups-extras, otherwise we can add it to
> JGroups itself.
>


Bela, you forgot to mention an extra restriction: even if it has no
external deps, it should
not be too big in terms of LOC (what is the threshold?)  ;)

Gustavo


>
> On 19/08/16 11:00, Sebastian Laskawiec wrote:
> > Hey!
> >
> > I've been playing with Kubernetes PetSets [1] for a while and I'd like
> > to share some thoughts. Before I dig in, let me give you some PetSets
> > highlights:
> >
> >   * PetSets are alpha resources for managing stateful apps in Kubernetes
> >     1.3 (and OpenShift Origin 1.3).
> >   * Since this is an alpha resource, there are no guarantees about
> >     backwards compatibility. Alpha resources can also be disabled in
> >     some public cloud providers (you can control which API versions are
> >     accessible [2]).
> >   * PetSets allows starting pods in sequence (not relevant for us, but
> >     this is a killer feature for master-slave systems).
> >   * Each Pod has it's own unique entry in DNS, which makes discovery
> >     very simple (I'll dig into that a bit later)
> >   * Volumes are always mounted to the same Pods, which is very important
> >     in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades
> >     [3]).
> >
> > Thoughts and ideas after spending some time playing with this feature:
> >
> >   * PetSets make discovery a lot easier. It's a combination of two
> >     things - Headless Services [4] which create multiple A records in
> >     DNS and predictable host names. Each Pod has it's own unique DNS
> >     entry following pattern: {PetSetName}-{PodIndex}.{ServiceName} [5].
> >     Here's an example of an Infinispan PetSet deployed on my local
> >     cluster [6]. As you can see we have all domain names and IPs from a
> >     single DNS query.
> >   * Maybe we could perform discovery using this mechanism? I'm aware of
> >     DNS discovery implemented in KUBE_PING [7][8] but the code looks
> >     trivial [9] so maybe it should be implement inside JGroups? @Bela -
> >     WDYT?
> >   * PetSets do not integrate well with OpenShift 'new-app' command. In
> >     other words, our users will need to use provided yaml (or json)
> >     files to create Infinispan cluster. It's not a show-stopper but it's
> >     a bit less convenient than 'oc new-app'.
> >   * Since PetSets are alpha resources they need to be considered as
> >     secondary way to deploy Infinispan on Kubernetes and OpenShift.
> >   * Finally, the persistent volumes - since a Pod always gets the same
> >     volume, it would be safe to use any file-based cache store.
> >
> > If you'd like to play with PetSets on your local environment, here are
> > necessary yaml files [10].
> >
> > Thanks
> > Sebastian
> >
> >
> > [1] http://kubernetes.io/docs/user-guide/petset/
> > [2] For checking which APIs are accessible, use 'kubectl api-versions'
> > [3]
> > http://infinispan.org/docs/stable/user_guide/user_guide.
> html#_Rolling_chapter
> > [4] http://kubernetes.io/docs/user-guide/services/#headless-services
> > [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
> > [6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
> > [7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
> > [8] https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
> > [9] http://stackoverflow.com/a/12405896/562699
> > [10] You might need to adjust ImageStream.
> > https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160819/e1104d88/attachment.html 

From bban at redhat.com  Fri Aug 19 06:07:44 2016
From: bban at redhat.com (Bela Ban)
Date: Fri, 19 Aug 2016 12:07:44 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <CAH8Ud1HtV-waeTSCLb7-t5mqOZHEOrv+R1fr42aam03D0hpb9Q@mail.gmail.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<57B6CEDB.1090700@redhat.com>
	<CAH8Ud1HtV-waeTSCLb7-t5mqOZHEOrv+R1fr42aam03D0hpb9Q@mail.gmail.com>
Message-ID: <57B6DA70.2060307@redhat.com>



On 19/08/16 11:27, Gustavo Fernandes wrote:
> On Fri, Aug 19, 2016 at 10:18 AM, Bela Ban <bban at redhat.com
> <mailto:bban at redhat.com>> wrote:
>
>     Hi Sebastian
>
>     the usual restrictions apply: if DNS discovery depends on external libs,
>     then it should be hosted in jgroups-extras, otherwise we can add it to
>     JGroups itself.
>
>
>
> Bela, you forgot to mention an extra restriction: even if it has no
> external deps, it should
> not be too big in terms of LOC (what is the threshold?)  ;)


You mean because people copy&paste the deps into one class? Yes, that 
happened before but since the resulting class was 17K, I rejected it. 
The largest class in JGroups (Util) has 4K, so that's a ballpark figure.

However, with jgroups-extras, we now have the possibility of adding deps.


> Gustavo
>
>
>     On 19/08/16 11:00, Sebastian Laskawiec wrote:
>     > Hey!
>     >
>     > I've been playing with Kubernetes PetSets [1] for a while and I'd like
>     > to share some thoughts. Before I dig in, let me give you some PetSets
>     > highlights:
>     >
>      >   * PetSets are alpha resources for managing stateful apps in
>     Kubernetes
>     >     1.3 (and OpenShift Origin 1.3).
>      >   * Since this is an alpha resource, there are no guarantees about
>     >     backwards compatibility. Alpha resources can also be disabled in
>     >     some public cloud providers (you can control which API versions are
>     >     accessible [2]).
>      >   * PetSets allows starting pods in sequence (not relevant for
>     us, but
>     >     this is a killer feature for master-slave systems).
>      >   * Each Pod has it's own unique entry in DNS, which makes discovery
>     >     very simple (I'll dig into that a bit later)
>      >   * Volumes are always mounted to the same Pods, which is very
>     important
>     >     in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades
>     >     [3]).
>     >
>     > Thoughts and ideas after spending some time playing with this feature:
>     >
>      >   * PetSets make discovery a lot easier. It's a combination of two
>     >     things - Headless Services [4] which create multiple A records in
>     >     DNS and predictable host names. Each Pod has it's own unique DNS
>     >     entry following pattern: {PetSetName}-{PodIndex}.{ServiceName} [5].
>     >     Here's an example of an Infinispan PetSet deployed on my local
>     >     cluster [6]. As you can see we have all domain names and IPs from a
>     >     single DNS query.
>      >   * Maybe we could perform discovery using this mechanism? I'm
>     aware of
>     >     DNS discovery implemented in KUBE_PING [7][8] but the code looks
>     >     trivial [9] so maybe it should be implement inside JGroups? @Bela -
>     >     WDYT?
>      >   * PetSets do not integrate well with OpenShift 'new-app'
>     command. In
>     >     other words, our users will need to use provided yaml (or json)
>     >     files to create Infinispan cluster. It's not a show-stopper but it's
>     >     a bit less convenient than 'oc new-app'.
>      >   * Since PetSets are alpha resources they need to be considered as
>     >     secondary way to deploy Infinispan on Kubernetes and OpenShift.
>      >   * Finally, the persistent volumes - since a Pod always gets the
>     same
>      >     volume, it would be safe to use any file-based cache store.
>      >
>      > If you'd like to play with PetSets on your local environment,
>     here are
>      > necessary yaml files [10].
>      >
>      > Thanks
>      > Sebastian
>      >
>      >
>      > [1] http://kubernetes.io/docs/user-guide/petset/
>     <http://kubernetes.io/docs/user-guide/petset/>
>      > [2] For checking which APIs are accessible, use 'kubectl
>     api-versions'
>      > [3]
>      >
>     http://infinispan.org/docs/stable/user_guide/user_guide.html#_Rolling_chapter
>     <http://infinispan.org/docs/stable/user_guide/user_guide.html#_Rolling_chapter>
>      > [4]
>     http://kubernetes.io/docs/user-guide/services/#headless-services
>     <http://kubernetes.io/docs/user-guide/services/#headless-services>
>      > [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
>     <http://kubernetes.io/docs/user-guide/petset/#peer-discovery>
>      > [6]
>     https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
>     <https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676>
>      > [7]
>     https://github.com/jboss-openshift/openshift-ping/tree/master/dns
>     <https://github.com/jboss-openshift/openshift-ping/tree/master/dns>
>      > [8]
>     https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
>     <https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns>
>      > [9] http://stackoverflow.com/a/12405896/562699
>     <http://stackoverflow.com/a/12405896/562699>
>      > [10] You might need to adjust ImageStream.
>      > https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0
>     <https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0>
>
>     --
>     Bela Ban, JGroups lead (http://www.jgroups.org)
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From dan.berindei at gmail.com  Fri Aug 19 10:20:21 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Fri, 19 Aug 2016 17:20:21 +0300
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
 or even enforce it
In-Reply-To: <CAFm4XO1sfqg8J-4U+UCr8m3YznTWcdj7cY1wxFTPA1meMC6MrA@mail.gmail.com>
References: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
	<CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>
	<CAFm4XO1sfqg8J-4U+UCr8m3YznTWcdj7cY1wxFTPA1meMC6MrA@mail.gmail.com>
Message-ID: <CA+nfvwT4SSr7g1mqBj8e7LtaENMQQ4NaiVvqYQ--3ru=OW8=tg@mail.gmail.com>

On Thu, Aug 18, 2016 at 6:12 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
> On 18 August 2016 at 14:44, Dan Berindei <dan.berindei at gmail.com> wrote:
>> First, a question: doesn't query already configure its internal caches
>> to use the AffinityPartitioner? Does it need the application caches to
>> have AffinityPartitioner enabled, too?
>
> No, we may provide reasonable defaults so that demos and tutorials
> work out of the box but the general recommendation is that people make
> their own configuration, as they will need to tune things, or add
> CacheStores, etc..
>

I see... I wonder if we could at least have some templates for the
user to extend, without having to copy a bunch of stuff from
tutorials?


>> I'm ok with making AffinityPartitioner the default key partitioner:
>> the less configuration the user has to change to make things work, the
>> better. I'm also ok with changing it to use delegation, to make it
>> easy for user to compose it with another key partitioner. We should
>> just run some benchmarks first to check it doesn't affect performance
>> for repl mode reads.
>
> Ok!
> https://issues.jboss.org/browse/ISPN-6956
>
> I'll send a PR; regarding benchmarking I'll take the optimistic path:
> we'll test it after it's integrated, worst case we can take it out.
>
>> But I feel users should be able to disable it if they want to, so I
>> wouldn't always wrap user partitioners. Also, usually less magic ==
>> better. Query can validate the configuration and warn the user if the
>> configuration doesn't have AffinityPartitioner in place.
>
> I disagree on this as this would require more options, which make it
> harder to use, and there's no good enough reason to give away the
> option to disable this; especially as it kills other features.
>

Well, grouping isn't enabled by default, even though that means
AdvancedCache.getGroup(String) doesn't work with the default
configuration. Sure, AffinityPartitioner's overhead is probably lower
than GroupingPartitioner's for non-targeted keys, but would you want
to wrap all key partitioners with AffinityPartitioner if we were
already wrapping them all with GroupingPartitioner?

TBH I don't know why a user would want to customize the key
partitioner when he could implement AffinityTaggedKey instead...


> BTW I never mentioned Query :)

You did mention your "evil" plans to improve query performance :D


> We happen to use this functionality in Query but as I mentioned other
> teams are looking forward to use this functionality, so I think we
> should promote this as a public API, a new feature of Infinispan core.
>

AffinityTaggedKey was already in the public API, even before
AffinityPartitioner was the default. So now we have these options for
influencing the key distribution:
* Implementing AffinityTaggedKey
* Implementing a custom KeyPartitioner
* Grouping: annotating a key method with @Group, or implementing Grouper
* Generating random keys and mapping them to nodes with KeyAffinityService


> So since you want to be able to disable it I won't wrap all user
> configured implementations in my PR for ISPN-6956, but I'd prefer it
> if as a second step you would reconsider and allow me to wrap them
> all.
>

Feel free to open a separate PR for this, I'm definitely not going to
block it from going in.

Cheers
Dan

From dan.berindei at gmail.com  Fri Aug 19 10:39:37 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Fri, 19 Aug 2016 17:39:37 +0300
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
 or even enforce it
In-Reply-To: <CA+nfvwT4SSr7g1mqBj8e7LtaENMQQ4NaiVvqYQ--3ru=OW8=tg@mail.gmail.com>
References: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
	<CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>
	<CAFm4XO1sfqg8J-4U+UCr8m3YznTWcdj7cY1wxFTPA1meMC6MrA@mail.gmail.com>
	<CA+nfvwT4SSr7g1mqBj8e7LtaENMQQ4NaiVvqYQ--3ru=OW8=tg@mail.gmail.com>
Message-ID: <CA+nfvwSHvyAYk0WS=WWSO59qtS6msjpAuEh7HPCqc-sS1B0OnQ@mail.gmail.com>

I just had a discussion with Adrian, and he reminded me that the key
partitioners don't yet work with any HotRod client.

Assuming we'll have for HotRod key partitioners soon,
AffinityPartitioner still won't work with remote caches, where we want
to keep the key as a byte[]. Instead, we would need a key partitioner
that knows protobuf and can parse a single field from the object.

So I'd like to withdraw my support for making AffinityPartitioner the default :)

Cheers
Dan


On Fri, Aug 19, 2016 at 5:20 PM, Dan Berindei <dan.berindei at gmail.com> wrote:
> On Thu, Aug 18, 2016 at 6:12 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
>> On 18 August 2016 at 14:44, Dan Berindei <dan.berindei at gmail.com> wrote:
>>> First, a question: doesn't query already configure its internal caches
>>> to use the AffinityPartitioner? Does it need the application caches to
>>> have AffinityPartitioner enabled, too?
>>
>> No, we may provide reasonable defaults so that demos and tutorials
>> work out of the box but the general recommendation is that people make
>> their own configuration, as they will need to tune things, or add
>> CacheStores, etc..
>>
>
> I see... I wonder if we could at least have some templates for the
> user to extend, without having to copy a bunch of stuff from
> tutorials?
>
>
>>> I'm ok with making AffinityPartitioner the default key partitioner:
>>> the less configuration the user has to change to make things work, the
>>> better. I'm also ok with changing it to use delegation, to make it
>>> easy for user to compose it with another key partitioner. We should
>>> just run some benchmarks first to check it doesn't affect performance
>>> for repl mode reads.
>>
>> Ok!
>> https://issues.jboss.org/browse/ISPN-6956
>>
>> I'll send a PR; regarding benchmarking I'll take the optimistic path:
>> we'll test it after it's integrated, worst case we can take it out.
>>
>>> But I feel users should be able to disable it if they want to, so I
>>> wouldn't always wrap user partitioners. Also, usually less magic ==
>>> better. Query can validate the configuration and warn the user if the
>>> configuration doesn't have AffinityPartitioner in place.
>>
>> I disagree on this as this would require more options, which make it
>> harder to use, and there's no good enough reason to give away the
>> option to disable this; especially as it kills other features.
>>
>
> Well, grouping isn't enabled by default, even though that means
> AdvancedCache.getGroup(String) doesn't work with the default
> configuration. Sure, AffinityPartitioner's overhead is probably lower
> than GroupingPartitioner's for non-targeted keys, but would you want
> to wrap all key partitioners with AffinityPartitioner if we were
> already wrapping them all with GroupingPartitioner?
>
> TBH I don't know why a user would want to customize the key
> partitioner when he could implement AffinityTaggedKey instead...
>
>
>> BTW I never mentioned Query :)
>
> You did mention your "evil" plans to improve query performance :D
>
>
>> We happen to use this functionality in Query but as I mentioned other
>> teams are looking forward to use this functionality, so I think we
>> should promote this as a public API, a new feature of Infinispan core.
>>
>
> AffinityTaggedKey was already in the public API, even before
> AffinityPartitioner was the default. So now we have these options for
> influencing the key distribution:
> * Implementing AffinityTaggedKey
> * Implementing a custom KeyPartitioner
> * Grouping: annotating a key method with @Group, or implementing Grouper
> * Generating random keys and mapping them to nodes with KeyAffinityService
>
>
>> So since you want to be able to disable it I won't wrap all user
>> configured implementations in my PR for ISPN-6956, but I'd prefer it
>> if as a second step you would reconsider and allow me to wrap them
>> all.
>>
>
> Feel free to open a separate PR for this, I'm definitely not going to
> block it from going in.
>
> Cheers
> Dan

From sanne at infinispan.org  Fri Aug 19 10:44:56 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 19 Aug 2016 15:44:56 +0100
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
 or even enforce it
In-Reply-To: <CA+nfvwSHvyAYk0WS=WWSO59qtS6msjpAuEh7HPCqc-sS1B0OnQ@mail.gmail.com>
References: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
	<CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>
	<CAFm4XO1sfqg8J-4U+UCr8m3YznTWcdj7cY1wxFTPA1meMC6MrA@mail.gmail.com>
	<CA+nfvwT4SSr7g1mqBj8e7LtaENMQQ4NaiVvqYQ--3ru=OW8=tg@mail.gmail.com>
	<CA+nfvwSHvyAYk0WS=WWSO59qtS6msjpAuEh7HPCqc-sS1B0OnQ@mail.gmail.com>
Message-ID: <CAFm4XO3d6_s-Q0LRuerA7Sv=VJKfwdDUmObzg8UWkvW=QkYLOg@mail.gmail.com>

On 19 August 2016 at 15:39, Dan Berindei <dan.berindei at gmail.com> wrote:
> I just had a discussion with Adrian, and he reminded me that the key
> partitioners don't yet work with any HotRod client.

In Hot Rod "out of the box" absolutely nothing works, as you have to set
key-equivalence="org.infinispan.commons.equivalence.ByteArrayEquivalence"
at the very least just to be able to get the "Get" operation working.

Which is another thing I don't like, but probably unrelated as it
looks like we epxect people to use our "server configurations" which
will have different settings.

>
> Assuming we'll have for HotRod key partitioners soon,
> AffinityPartitioner still won't work with remote caches, where we want
> to keep the key as a byte[]. Instead, we would need a key partitioner
> that knows protobuf and can parse a single field from the object.
>
> So I'd like to withdraw my support for making AffinityPartitioner the default :)
>
> Cheers
> Dan
>
>
> On Fri, Aug 19, 2016 at 5:20 PM, Dan Berindei <dan.berindei at gmail.com> wrote:
>> On Thu, Aug 18, 2016 at 6:12 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
>>> On 18 August 2016 at 14:44, Dan Berindei <dan.berindei at gmail.com> wrote:
>>>> First, a question: doesn't query already configure its internal caches
>>>> to use the AffinityPartitioner? Does it need the application caches to
>>>> have AffinityPartitioner enabled, too?
>>>
>>> No, we may provide reasonable defaults so that demos and tutorials
>>> work out of the box but the general recommendation is that people make
>>> their own configuration, as they will need to tune things, or add
>>> CacheStores, etc..
>>>
>>
>> I see... I wonder if we could at least have some templates for the
>> user to extend, without having to copy a bunch of stuff from
>> tutorials?
>>
>>
>>>> I'm ok with making AffinityPartitioner the default key partitioner:
>>>> the less configuration the user has to change to make things work, the
>>>> better. I'm also ok with changing it to use delegation, to make it
>>>> easy for user to compose it with another key partitioner. We should
>>>> just run some benchmarks first to check it doesn't affect performance
>>>> for repl mode reads.
>>>
>>> Ok!
>>> https://issues.jboss.org/browse/ISPN-6956
>>>
>>> I'll send a PR; regarding benchmarking I'll take the optimistic path:
>>> we'll test it after it's integrated, worst case we can take it out.
>>>
>>>> But I feel users should be able to disable it if they want to, so I
>>>> wouldn't always wrap user partitioners. Also, usually less magic ==
>>>> better. Query can validate the configuration and warn the user if the
>>>> configuration doesn't have AffinityPartitioner in place.
>>>
>>> I disagree on this as this would require more options, which make it
>>> harder to use, and there's no good enough reason to give away the
>>> option to disable this; especially as it kills other features.
>>>
>>
>> Well, grouping isn't enabled by default, even though that means
>> AdvancedCache.getGroup(String) doesn't work with the default
>> configuration. Sure, AffinityPartitioner's overhead is probably lower
>> than GroupingPartitioner's for non-targeted keys, but would you want
>> to wrap all key partitioners with AffinityPartitioner if we were
>> already wrapping them all with GroupingPartitioner?
>>
>> TBH I don't know why a user would want to customize the key
>> partitioner when he could implement AffinityTaggedKey instead...
>>
>>
>>> BTW I never mentioned Query :)
>>
>> You did mention your "evil" plans to improve query performance :D
>>
>>
>>> We happen to use this functionality in Query but as I mentioned other
>>> teams are looking forward to use this functionality, so I think we
>>> should promote this as a public API, a new feature of Infinispan core.
>>>
>>
>> AffinityTaggedKey was already in the public API, even before
>> AffinityPartitioner was the default. So now we have these options for
>> influencing the key distribution:
>> * Implementing AffinityTaggedKey
>> * Implementing a custom KeyPartitioner
>> * Grouping: annotating a key method with @Group, or implementing Grouper
>> * Generating random keys and mapping them to nodes with KeyAffinityService
>>
>>
>>> So since you want to be able to disable it I won't wrap all user
>>> configured implementations in my PR for ISPN-6956, but I'd prefer it
>>> if as a second step you would reconsider and allow me to wrap them
>>> all.
>>>
>>
>> Feel free to open a separate PR for this, I'm definitely not going to
>> block it from going in.
>>
>> Cheers
>> Dan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From gustavo at infinispan.org  Fri Aug 19 10:54:15 2016
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Fri, 19 Aug 2016 15:54:15 +0100
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
 or even enforce it
In-Reply-To: <CAFm4XO3d6_s-Q0LRuerA7Sv=VJKfwdDUmObzg8UWkvW=QkYLOg@mail.gmail.com>
References: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
	<CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>
	<CAFm4XO1sfqg8J-4U+UCr8m3YznTWcdj7cY1wxFTPA1meMC6MrA@mail.gmail.com>
	<CA+nfvwT4SSr7g1mqBj8e7LtaENMQQ4NaiVvqYQ--3ru=OW8=tg@mail.gmail.com>
	<CA+nfvwSHvyAYk0WS=WWSO59qtS6msjpAuEh7HPCqc-sS1B0OnQ@mail.gmail.com>
	<CAFm4XO3d6_s-Q0LRuerA7Sv=VJKfwdDUmObzg8UWkvW=QkYLOg@mail.gmail.com>
Message-ID: <CAH8Ud1FPmTZ0Vfj0W5dnaMz36aM6SqoyMaHZy-fvkKYoWwbBKw@mail.gmail.com>

On Fri, Aug 19, 2016 at 3:44 PM, Sanne Grinovero <sanne at infinispan.org>
wrote:

> On 19 August 2016 at 15:39, Dan Berindei <dan.berindei at gmail.com> wrote:
> > I just had a discussion with Adrian, and he reminded me that the key
> > partitioners don't yet work with any HotRod client.
>
> In Hot Rod "out of the box" absolutely nothing works, as you have to set
> key-equivalence="org.infinispan.commons.equivalence.ByteArrayEquivalence"
> at the very least just to be able to get the "Get" operation working.
>


This is set for you via inheritance at [1], it won't work if using the
HotServer class to start the server in unit tests for example.


[1]
https://github.com/infinispan/infinispan/blob/master/server/integration/infinispan/src/main/resources/infinispan-defaults.xml

Gustavo


>
> Which is another thing I don't like, but probably unrelated as it
> looks like we epxect people to use our "server configurations" which
> will have different settings.
>
> >
> > Assuming we'll have for HotRod key partitioners soon,
> > AffinityPartitioner still won't work with remote caches, where we want
> > to keep the key as a byte[]. Instead, we would need a key partitioner
> > that knows protobuf and can parse a single field from the object.
> >
> > So I'd like to withdraw my support for making AffinityPartitioner the
> default :)
> >
> > Cheers
> > Dan
> >
> >
> > On Fri, Aug 19, 2016 at 5:20 PM, Dan Berindei <dan.berindei at gmail.com>
> wrote:
> >> On Thu, Aug 18, 2016 at 6:12 PM, Sanne Grinovero <sanne at infinispan.org>
> wrote:
> >>> On 18 August 2016 at 14:44, Dan Berindei <dan.berindei at gmail.com>
> wrote:
> >>>> First, a question: doesn't query already configure its internal caches
> >>>> to use the AffinityPartitioner? Does it need the application caches to
> >>>> have AffinityPartitioner enabled, too?
> >>>
> >>> No, we may provide reasonable defaults so that demos and tutorials
> >>> work out of the box but the general recommendation is that people make
> >>> their own configuration, as they will need to tune things, or add
> >>> CacheStores, etc..
> >>>
> >>
> >> I see... I wonder if we could at least have some templates for the
> >> user to extend, without having to copy a bunch of stuff from
> >> tutorials?
> >>
> >>
> >>>> I'm ok with making AffinityPartitioner the default key partitioner:
> >>>> the less configuration the user has to change to make things work, the
> >>>> better. I'm also ok with changing it to use delegation, to make it
> >>>> easy for user to compose it with another key partitioner. We should
> >>>> just run some benchmarks first to check it doesn't affect performance
> >>>> for repl mode reads.
> >>>
> >>> Ok!
> >>> https://issues.jboss.org/browse/ISPN-6956
> >>>
> >>> I'll send a PR; regarding benchmarking I'll take the optimistic path:
> >>> we'll test it after it's integrated, worst case we can take it out.
> >>>
> >>>> But I feel users should be able to disable it if they want to, so I
> >>>> wouldn't always wrap user partitioners. Also, usually less magic ==
> >>>> better. Query can validate the configuration and warn the user if the
> >>>> configuration doesn't have AffinityPartitioner in place.
> >>>
> >>> I disagree on this as this would require more options, which make it
> >>> harder to use, and there's no good enough reason to give away the
> >>> option to disable this; especially as it kills other features.
> >>>
> >>
> >> Well, grouping isn't enabled by default, even though that means
> >> AdvancedCache.getGroup(String) doesn't work with the default
> >> configuration. Sure, AffinityPartitioner's overhead is probably lower
> >> than GroupingPartitioner's for non-targeted keys, but would you want
> >> to wrap all key partitioners with AffinityPartitioner if we were
> >> already wrapping them all with GroupingPartitioner?
> >>
> >> TBH I don't know why a user would want to customize the key
> >> partitioner when he could implement AffinityTaggedKey instead...
> >>
> >>
> >>> BTW I never mentioned Query :)
> >>
> >> You did mention your "evil" plans to improve query performance :D
> >>
> >>
> >>> We happen to use this functionality in Query but as I mentioned other
> >>> teams are looking forward to use this functionality, so I think we
> >>> should promote this as a public API, a new feature of Infinispan core.
> >>>
> >>
> >> AffinityTaggedKey was already in the public API, even before
> >> AffinityPartitioner was the default. So now we have these options for
> >> influencing the key distribution:
> >> * Implementing AffinityTaggedKey
> >> * Implementing a custom KeyPartitioner
> >> * Grouping: annotating a key method with @Group, or implementing Grouper
> >> * Generating random keys and mapping them to nodes with
> KeyAffinityService
> >>
> >>
> >>> So since you want to be able to disable it I won't wrap all user
> >>> configured implementations in my PR for ISPN-6956, but I'd prefer it
> >>> if as a second step you would reconsider and allow me to wrap them
> >>> all.
> >>>
> >>
> >> Feel free to open a separate PR for this, I'm definitely not going to
> >> block it from going in.
> >>
> >> Cheers
> >> Dan
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160819/d619565a/attachment-0001.html 

From dan.berindei at gmail.com  Fri Aug 19 11:04:10 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Fri, 19 Aug 2016 18:04:10 +0300
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
 or even enforce it
In-Reply-To: <CAFm4XO3d6_s-Q0LRuerA7Sv=VJKfwdDUmObzg8UWkvW=QkYLOg@mail.gmail.com>
References: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
	<CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>
	<CAFm4XO1sfqg8J-4U+UCr8m3YznTWcdj7cY1wxFTPA1meMC6MrA@mail.gmail.com>
	<CA+nfvwT4SSr7g1mqBj8e7LtaENMQQ4NaiVvqYQ--3ru=OW8=tg@mail.gmail.com>
	<CA+nfvwSHvyAYk0WS=WWSO59qtS6msjpAuEh7HPCqc-sS1B0OnQ@mail.gmail.com>
	<CAFm4XO3d6_s-Q0LRuerA7Sv=VJKfwdDUmObzg8UWkvW=QkYLOg@mail.gmail.com>
Message-ID: <CA+nfvwRXxs0daaxuRcxc94JUac3R3DjVArRyz+dOEYkjEfSx+A@mail.gmail.com>

Is there any way that AffinityPartitioner could work with HotRod, though?


On Fri, Aug 19, 2016 at 5:44 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
> On 19 August 2016 at 15:39, Dan Berindei <dan.berindei at gmail.com> wrote:
>> I just had a discussion with Adrian, and he reminded me that the key
>> partitioners don't yet work with any HotRod client.
>
> In Hot Rod "out of the box" absolutely nothing works, as you have to set
> key-equivalence="org.infinispan.commons.equivalence.ByteArrayEquivalence"
> at the very least just to be able to get the "Get" operation working.
>
> Which is another thing I don't like, but probably unrelated as it
> looks like we epxect people to use our "server configurations" which
> will have different settings.
>
>>
>> Assuming we'll have for HotRod key partitioners soon,
>> AffinityPartitioner still won't work with remote caches, where we want
>> to keep the key as a byte[]. Instead, we would need a key partitioner
>> that knows protobuf and can parse a single field from the object.
>>
>> So I'd like to withdraw my support for making AffinityPartitioner the default :)
>>
>> Cheers
>> Dan
>>
>>
>> On Fri, Aug 19, 2016 at 5:20 PM, Dan Berindei <dan.berindei at gmail.com> wrote:
>>> On Thu, Aug 18, 2016 at 6:12 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
>>>> On 18 August 2016 at 14:44, Dan Berindei <dan.berindei at gmail.com> wrote:
>>>>> First, a question: doesn't query already configure its internal caches
>>>>> to use the AffinityPartitioner? Does it need the application caches to
>>>>> have AffinityPartitioner enabled, too?
>>>>
>>>> No, we may provide reasonable defaults so that demos and tutorials
>>>> work out of the box but the general recommendation is that people make
>>>> their own configuration, as they will need to tune things, or add
>>>> CacheStores, etc..
>>>>
>>>
>>> I see... I wonder if we could at least have some templates for the
>>> user to extend, without having to copy a bunch of stuff from
>>> tutorials?
>>>
>>>
>>>>> I'm ok with making AffinityPartitioner the default key partitioner:
>>>>> the less configuration the user has to change to make things work, the
>>>>> better. I'm also ok with changing it to use delegation, to make it
>>>>> easy for user to compose it with another key partitioner. We should
>>>>> just run some benchmarks first to check it doesn't affect performance
>>>>> for repl mode reads.
>>>>
>>>> Ok!
>>>> https://issues.jboss.org/browse/ISPN-6956
>>>>
>>>> I'll send a PR; regarding benchmarking I'll take the optimistic path:
>>>> we'll test it after it's integrated, worst case we can take it out.
>>>>
>>>>> But I feel users should be able to disable it if they want to, so I
>>>>> wouldn't always wrap user partitioners. Also, usually less magic ==
>>>>> better. Query can validate the configuration and warn the user if the
>>>>> configuration doesn't have AffinityPartitioner in place.
>>>>
>>>> I disagree on this as this would require more options, which make it
>>>> harder to use, and there's no good enough reason to give away the
>>>> option to disable this; especially as it kills other features.
>>>>
>>>
>>> Well, grouping isn't enabled by default, even though that means
>>> AdvancedCache.getGroup(String) doesn't work with the default
>>> configuration. Sure, AffinityPartitioner's overhead is probably lower
>>> than GroupingPartitioner's for non-targeted keys, but would you want
>>> to wrap all key partitioners with AffinityPartitioner if we were
>>> already wrapping them all with GroupingPartitioner?
>>>
>>> TBH I don't know why a user would want to customize the key
>>> partitioner when he could implement AffinityTaggedKey instead...
>>>
>>>
>>>> BTW I never mentioned Query :)
>>>
>>> You did mention your "evil" plans to improve query performance :D
>>>
>>>
>>>> We happen to use this functionality in Query but as I mentioned other
>>>> teams are looking forward to use this functionality, so I think we
>>>> should promote this as a public API, a new feature of Infinispan core.
>>>>
>>>
>>> AffinityTaggedKey was already in the public API, even before
>>> AffinityPartitioner was the default. So now we have these options for
>>> influencing the key distribution:
>>> * Implementing AffinityTaggedKey
>>> * Implementing a custom KeyPartitioner
>>> * Grouping: annotating a key method with @Group, or implementing Grouper
>>> * Generating random keys and mapping them to nodes with KeyAffinityService
>>>
>>>
>>>> So since you want to be able to disable it I won't wrap all user
>>>> configured implementations in my PR for ISPN-6956, but I'd prefer it
>>>> if as a second step you would reconsider and allow me to wrap them
>>>> all.
>>>>
>>>
>>> Feel free to open a separate PR for this, I'm definitely not going to
>>> block it from going in.
>>>
>>> Cheers
>>> Dan
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From rvansa at redhat.com  Fri Aug 19 16:34:15 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Fri, 19 Aug 2016 22:34:15 +0200
Subject: [infinispan-dev] Optional in listener events
Message-ID: <57B76D47.10907@redhat.com>

Hi,

as I am trying to simplify current entry wrapping and distribution code, 
I often find that listeners can get wrong previous value in the event, 
and it sometimes forces the command to load the value even if it is not 
needed for the command.

I am wondering if we should change the previous value in events to 
Optional - we can usually at least detect that we cannot provide a 
reliable value (e.g. after retry due to topology change, or because the 
command did not bothered to load the previous value from cache loader) 
and return empty Optional.

WDYT?

Radim

-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From bban at redhat.com  Sat Aug 20 03:19:12 2016
From: bban at redhat.com (Bela Ban)
Date: Sat, 20 Aug 2016 09:19:12 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <447733132.4160093.1471644286616.JavaMail.zimbra@redhat.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<57B6CEDB.1090700@redhat.com>
	<CAA9R9O7cDnsx3PBnhxZ58ZQp0x6TOp+Orq2TUFsrPyyG5ZYWzQ@mail.gmail.com>
	<447733132.4160093.1471644286616.JavaMail.zimbra@redhat.com>
Message-ID: <57B80470.5060508@redhat.com>

If we add a DNS discovery protocol, it would only be another discovery 
protocol among many, and customers can choose which one to use.
I'm also thinking of adding the ability to JGroups to use multiple 
discovery protocols in the same stack and combine their result sets into 
one. Not sure though if it makes sense to use KUBE_PING and DNS in the 
same stack...

On 20/08/16 00:04, Rob Cernich wrote:
> A couple of things...
>
> re. volumes:
> We also need to consider the mounting behavior for scale down scenarios
> and for overage scenarios when doing upgrades.  For the latter,
> OpenShift can spin up pods of the new version before the older version
> pods have terminated.  This may mean that some volumes from the old pods
> are orphaned.  We did see this when testing A-MQ during upgrades.  With
> a single pod, the upgrade process caused the new version to have a new
> mount and the original mount was left orphaned (another upgrade would
> cause the newer pod to pick up the orphaned mount, leaving the new mount
> orphaned).  I believe we worked around this by specifying an overage of
> 0% during upgrades.  This ensured the new pods would pick up the volumes
> left behind by the old pods.  (Actually, we were using subdirectories in
> the mount, since all pods shared the same volume.)
>
> re. dns:
> DNS should work fine as-is, but there are a couple things that you need
> to consider.
> 1. Service endpoints are only available in DNS after the pod becomes
> ready (SVC records on the service name).  Because infinispan attaches
> itself to the cluster, this meant pods were all started as cluster of
> one, then merged once they noticed the other pods.  This had a
> significant impact on startup.  Since then, OpenShift has added the
> ability to query the endpoints associated with a service as soon as the
> pod is created, which would allow initialization to work correctly.  To
> make this work, we'd have to change the form of the DNS query to pick up
> the service endpoints (I forget the naming scheme).
>
> Another thing to keep in mind is that looking up pods by labels allows
> any pod with the specified label to be added to the cluster.  I'm not
> sure of a use case for this, but it would allow other deployments to be
> included in the cluster.  (You could also argue that the service is the
> authority for this and any pod with said label would be added as a
> service endpoint, thus achieving the same behavior...probably more
> simply too.)
>
> Lastly, DNS was a little flaky when we first implemented this, which was
> part of the reason we went straight to kubernetes.  Users were using
> dnsmasq with wildcards that worked well for routes, but ended up routing
> services to the router ip instead of pod ip.  Needless to say, there
> were a lot of complications trying to use DNS and debug user problems
> with service resolution.
>
> Hope that helps,
> Rob
>
> ------------------------------------------------------------------------
>
>     Hey Bela!
>
>     No no, the resolution can be done with pure JDK.
>
>     Thanks
>     Sebastian
>
>     On Fri, Aug 19, 2016 at 11:18 AM, Bela Ban <bban at redhat.com
>     <mailto:bban at redhat.com>> wrote:
>
>         Hi Sebastian
>
>         the usual restrictions apply: if DNS discovery depends on
>         external libs, then it should be hosted in jgroups-extras,
>         otherwise we can add it to JGroups itself.
>
>         On 19/08/16 11:00, Sebastian Laskawiec wrote:
>
>             Hey!
>
>             I've been playing with Kubernetes PetSets [1] for a while
>             and I'd like
>             to share some thoughts. Before I dig in, let me give you
>             some PetSets
>             highlights:
>
>                * PetSets are alpha resources for managing stateful apps
>             in Kubernetes
>                  1.3 (and OpenShift Origin 1.3).
>                * Since this is an alpha resource, there are no
>             guarantees about
>                  backwards compatibility. Alpha resources can also be
>             disabled in
>                  some public cloud providers (you can control which API
>             versions are
>                  accessible [2]).
>                * PetSets allows starting pods in sequence (not relevant
>             for us, but
>                  this is a killer feature for master-slave systems).
>                * Each Pod has it's own unique entry in DNS, which makes
>             discovery
>                  very simple (I'll dig into that a bit later)
>                * Volumes are always mounted to the same Pods, which is
>             very important
>                  in Cache Store scenarios when we restart pods (e.g.
>             Rolling Upgrades
>                  [3]).
>
>             Thoughts and ideas after spending some time playing with
>             this feature:
>
>                * PetSets make discovery a lot easier. It's a combination
>             of two
>                  things - Headless Services [4] which create multiple A
>             records in
>                  DNS and predictable host names. Each Pod has it's own
>             unique DNS
>                  entry following pattern:
>             {PetSetName}-{PodIndex}.{ServiceName} [5].
>                  Here's an example of an Infinispan PetSet deployed on
>             my local
>                  cluster [6]. As you can see we have all domain names
>             and IPs from a
>                  single DNS query.
>                * Maybe we could perform discovery using this mechanism?
>             I'm aware of
>                  DNS discovery implemented in KUBE_PING [7][8] but the
>             code looks
>                  trivial [9] so maybe it should be implement inside
>             JGroups? @Bela -
>                  WDYT?
>                * PetSets do not integrate well with OpenShift 'new-app'
>             command. In
>                  other words, our users will need to use provided yaml
>             (or json)
>                  files to create Infinispan cluster. It's not a
>             show-stopper but it's
>                  a bit less convenient than 'oc new-app'.
>                * Since PetSets are alpha resources they need to be
>             considered as
>                  secondary way to deploy Infinispan on Kubernetes and
>             OpenShift.
>                * Finally, the persistent volumes - since a Pod always
>             gets the same
>                  volume, it would be safe to use any file-based cache store.
>
>             If you'd like to play with PetSets on your local
>             environment, here are
>             necessary yaml files [10].
>
>             Thanks
>             Sebastian
>
>
>             [1] http://kubernetes.io/docs/user-guide/petset/
>             [2] For checking which APIs are accessible, use 'kubectl
>             api-versions'
>             [3]
>             http://infinispan.org/docs/stable/user_guide/user_guide.html#_Rolling_chapter
>             [4]
>             http://kubernetes.io/docs/user-guide/services/#headless-services
>             [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
>             [6]
>             https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
>             [7]
>             https://github.com/jboss-openshift/openshift-ping/tree/master/dns
>             [8]
>             https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
>             [9] http://stackoverflow.com/a/12405896/562699
>             [10] You might need to adjust ImageStream.
>             https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0
>
>
>         --
>         Bela Ban, JGroups lead (http://www.jgroups.org)
>
>
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From manohar.sl at ericsson.com  Sat Aug 20 03:38:25 2016
From: manohar.sl at ericsson.com (Manohar SL)
Date: Sat, 20 Aug 2016 07:38:25 +0000
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with "JGroups
 3.6.8 + Infinispan 8.2.2":
Message-ID: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>

Hi Bela Ban,

We have been trying to use JGroups 3.6.8 with Infinispan 8.2.2., in this context we are observing an issue with the Replication Mode of Infinispan usage.
We see the below exception from Jgroups:
JGRP000029: failed sending message to <>:7800 (100 bytes): java.lang.NullPointerException, headers: TCPGOSSIP: [type=GET_MBRS_REQ, cluster=x-cluster], TP: [cluster_name=x-cluster]

The configurations used are highlighted below:

Infinispan config

<infinispan>

    <jgroups>

        <stack-file name="configurationFile" path="config/jgroups.xml"/>

    </jgroups>

    <cache-container>

        <transport cluster="x-cluster" stack="configurationFile" />

<replicated-cache name="transactional-type" mode="SYNC">

            <transaction mode="NON_XA"  locking="OPTIMISTIC" transaction-manager-lookup="org.infinispan.transaction.lookup.JBossStandaloneJTAManagerLookup" auto-commit="true"  />

<locking acquire-timeout="60000"/>

<expiration lifespan="43200000"/>

        </replicated-cache>

    </cache-container>

</infinispan>

 

Jgroups configuration

<!--

    TCP based stack, with flow control and message bundling. This is usually used when IP

    multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).

    Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.

    -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".

    author: Bela Ban

-->

<config xmlns="urn:org:jgroups"

        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="urn:org:jgroups  http://www.jgroups.org/schema/jgroups-3.6.xsd">

 

  <TCP loopback="true"

bind_addr="${jgroups.tcp.address:127.0.0.1}"

bind_port="${jgroups.tcp.port:7800}"

recv_buf_size="${tcp.recv_buf_size:20M}"

send_buf_size="${tcp.send_buf_size:640K}"

discard_incompatible_packets="true"

max_bundle_size="64K"

max_bundle_timeout="30"

enable_bundling="true"

       use_send_queues="true"

sock_conn_timeout="300"

timer_type="new"

timer.min_threads="4"

timer.max_threads="10"

timer.keep_alive_time="3000"

timer.queue_max_size="500"

thread_pool.enabled="true"

thread_pool.min_threads="2"

thread_pool.max_threads="30"

thread_pool.keep_alive_time="60000"

thread_pool.queue_enabled="false"

thread_pool.queue_max_size="100"

thread_pool.rejection_policy="discard"

oob_thread_pool.enabled="true"

       oob_thread_pool.min_threads="2"

oob_thread_pool.max_threads="30"

oob_thread_pool.keep_alive_time="60000"

oob_thread_pool.queue_enabled="false"

oob_thread_pool.queue_max_size="100"

oob_thread_pool.rejection_policy="discard"/>

 

    <!-- <TCP_NIO -->

<!-- bind_port="7800" -->

<!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->

<!-- use_send_queues="true" -->

<!-- sock_conn_timeout="300" -->

<!--         reader_threads="3" -->

<!-- writer_threads="3" -->

<!-- processor_threads="0" -->

<!-- processor_minThreads="0" -->

<!-- processor_maxThreads="0" -->

<!-- processor_queueSize="100" -->

<!--         processor_keepAliveTime="9223372036854775807"/> -->

    <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>

    <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"

            port_range="2" timeout="3000" /> -->

    <MERGE2 max_interval="30000" min_interval="10000"/>

    <FD_SOCK/>

    <FD timeout="3000" max_tries="3"/>

    <VERIFY_SUSPECT timeout="1500"/>

    <pbcast.NAKACK

use_mcast_xmit="false"

retransmit_timeout="300,600,1200,2400,4800"

discard_delivered_msgs="false"/>

    <UNICAST2 timeout="300,600,1200"

stable_interval="5000"

max_bytes="1m"/>

    <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="1m"/>

    <pbcast.GMS print_local_addr="false" join_timeout="3000" view_bundling="true"/>

    <UFC max_credits="200k" min_threshold="0.20"/>

    <MFC max_credits="200k" min_threshold="0.20"/>

    <FRAG2 frag_size="60000"/>

    <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />

</config>

Any help on this would be really great.
Kindly let us know if you would need any further information on this.

Regs,
Manohar.


From slaskawi at redhat.com  Mon Aug 22 01:56:51 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Mon, 22 Aug 2016 07:56:51 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <447733132.4160093.1471644286616.JavaMail.zimbra@redhat.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<57B6CEDB.1090700@redhat.com>
	<CAA9R9O7cDnsx3PBnhxZ58ZQp0x6TOp+Orq2TUFsrPyyG5ZYWzQ@mail.gmail.com>
	<447733132.4160093.1471644286616.JavaMail.zimbra@redhat.com>
Message-ID: <CAA9R9O4jdfc30aVb+fRLPRkGkX3Rd-U2W9wFnxQEPWnNcNjU5w@mail.gmail.com>

Hey Rob!

Thanks a lot for clarification!

More comments inlined.

Thanks
Sebastian

On Sat, Aug 20, 2016 at 12:04 AM, Rob Cernich <rcernich at redhat.com> wrote:

> A couple of things...
>
> re. volumes:
> We also need to consider the mounting behavior for scale down scenarios
> and for overage scenarios when doing upgrades.  For the latter, OpenShift
> can spin up pods of the new version before the older version pods have
> terminated.  This may mean that some volumes from the old pods are
> orphaned.  We did see this when testing A-MQ during upgrades.  With a
> single pod, the upgrade process caused the new version to have a new mount
> and the original mount was left orphaned (another upgrade would cause the
> newer pod to pick up the orphaned mount, leaving the new mount orphaned).
> I believe we worked around this by specifying an overage of 0% during
> upgrades.  This ensured the new pods would pick up the volumes left behind
> by the old pods.  (Actually, we were using subdirectories in the mount,
> since all pods shared the same volume.)
>
>
I think PetSets try to address this kind of problems. According to the
manual page [11], the storage is linked to the Pod ordinal and hostaname
and should be stable.

[11] http://kubernetes.io/docs/user-guide/petset/#when-to-use-pet-set


> re. dns:
> DNS should work fine as-is, but there are a couple things that you need to
> consider.
> 1. Service endpoints are only available in DNS after the pod becomes ready
> (SVC records on the service name).  Because infinispan attaches itself to
> the cluster, this meant pods were all started as cluster of one, then
> merged once they noticed the other pods.  This had a significant impact on
> startup.  Since then, OpenShift has added the ability to query the
> endpoints associated with a service as soon as the pod is created, which
> would allow initialization to work correctly.  To make this work, we'd have
> to change the form of the DNS query to pick up the service endpoints (I
> forget the naming scheme).
>

Yes, I agree. Adding nodes one after another will have significant impact
on cluster startup time. However it should be safe to query the cluster
(and even put data) during rebalance. So I would say, if a node is up, and
cluster is not damaged - we should treat it as ready.

NB - I proposed a HealthCheck API to Infinispan 9 (currently under
development) [12][13]. The overall cluster health can be in one of 3
statuses - GREEN (everything is fine), YELLOW (rebalance in progress), RED
(cluster not healthy). Kubernetes/OpenShift readiness probe should check if
the status is GREEN or YELLOW. The HealthCheck API is attached to the WF
management API so you can query it with CURL or using ispn_cli.sh script.

[12] https://github.com/infinispan/infinispan/wiki/Health-check-API
[13] https://github.com/infinispan/infinispan/pull/4499


> Another thing to keep in mind is that looking up pods by labels allows any
> pod with the specified label to be added to the cluster.  I'm not sure of a
> use case for this, but it would allow other deployments to be included in
> the cluster.  (You could also argue that the service is the authority for
> this and any pod with said label would be added as a service endpoint, thus
> achieving the same behavior...probably more simply too.)
>

I think this is a scenario when someone might try to attach Infinispan in
library mode (a dependency in WAR file for example) to the Hot Rod cluster.
Gustavo answered question like this a while ago [14].

[14] https://developer.jboss.org/message/961568


> Lastly, DNS was a little flaky when we first implemented this, which was
> part of the reason we went straight to kubernetes.  Users were using
> dnsmasq with wildcards that worked well for routes, but ended up routing
> services to the router ip instead of pod ip.  Needless to say, there were a
> lot of complications trying to use DNS and debug user problems with service
> resolution.
>

I think a governing headless service [15] is required here (PetSets require
a service but considering how Infinispan works, it should be a headless
service in my opinion).

[15] http://kubernetes.io/docs/user-guide/services/#headless-services


>
>
> Hope that helps,
> Rob
>
> ------------------------------
>
> Hey Bela!
>
> No no, the resolution can be done with pure JDK.
>
> Thanks
> Sebastian
>
> On Fri, Aug 19, 2016 at 11:18 AM, Bela Ban <bban at redhat.com> wrote:
>
>> Hi Sebastian
>>
>> the usual restrictions apply: if DNS discovery depends on external libs,
>> then it should be hosted in jgroups-extras, otherwise we can add it to
>> JGroups itself.
>>
>> On 19/08/16 11:00, Sebastian Laskawiec wrote:
>>
>>> Hey!
>>>
>>> I've been playing with Kubernetes PetSets [1] for a while and I'd like
>>> to share some thoughts. Before I dig in, let me give you some PetSets
>>> highlights:
>>>
>>>   * PetSets are alpha resources for managing stateful apps in Kubernetes
>>>     1.3 (and OpenShift Origin 1.3).
>>>   * Since this is an alpha resource, there are no guarantees about
>>>     backwards compatibility. Alpha resources can also be disabled in
>>>     some public cloud providers (you can control which API versions are
>>>     accessible [2]).
>>>   * PetSets allows starting pods in sequence (not relevant for us, but
>>>     this is a killer feature for master-slave systems).
>>>   * Each Pod has it's own unique entry in DNS, which makes discovery
>>>     very simple (I'll dig into that a bit later)
>>>   * Volumes are always mounted to the same Pods, which is very important
>>>     in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades
>>>     [3]).
>>>
>>> Thoughts and ideas after spending some time playing with this feature:
>>>
>>>   * PetSets make discovery a lot easier. It's a combination of two
>>>     things - Headless Services [4] which create multiple A records in
>>>     DNS and predictable host names. Each Pod has it's own unique DNS
>>>     entry following pattern: {PetSetName}-{PodIndex}.{ServiceName} [5].
>>>     Here's an example of an Infinispan PetSet deployed on my local
>>>     cluster [6]. As you can see we have all domain names and IPs from a
>>>     single DNS query.
>>>   * Maybe we could perform discovery using this mechanism? I'm aware of
>>>     DNS discovery implemented in KUBE_PING [7][8] but the code looks
>>>     trivial [9] so maybe it should be implement inside JGroups? @Bela -
>>>     WDYT?
>>>   * PetSets do not integrate well with OpenShift 'new-app' command. In
>>>     other words, our users will need to use provided yaml (or json)
>>>     files to create Infinispan cluster. It's not a show-stopper but it's
>>>     a bit less convenient than 'oc new-app'.
>>>   * Since PetSets are alpha resources they need to be considered as
>>>     secondary way to deploy Infinispan on Kubernetes and OpenShift.
>>>   * Finally, the persistent volumes - since a Pod always gets the same
>>>     volume, it would be safe to use any file-based cache store.
>>>
>>> If you'd like to play with PetSets on your local environment, here are
>>> necessary yaml files [10].
>>>
>>> Thanks
>>> Sebastian
>>>
>>>
>>> [1] http://kubernetes.io/docs/user-guide/petset/
>>> [2] For checking which APIs are accessible, use 'kubectl api-versions'
>>> [3]
>>> http://infinispan.org/docs/stable/user_guide/user_guide.
>>> html#_Rolling_chapter
>>> [4] http://kubernetes.io/docs/user-guide/services/#headless-services
>>> [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
>>> [6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
>>> [7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
>>> [8] https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
>>> [9] http://stackoverflow.com/a/12405896/562699
>>> [10] You might need to adjust ImageStream.
>>> https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0
>>>
>>
>> --
>> Bela Ban, JGroups lead (http://www.jgroups.org)
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160822/6c3887aa/attachment.html 

From slaskawi at redhat.com  Mon Aug 22 02:02:23 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Mon, 22 Aug 2016 08:02:23 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <57B80470.5060508@redhat.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<57B6CEDB.1090700@redhat.com>
	<CAA9R9O7cDnsx3PBnhxZ58ZQp0x6TOp+Orq2TUFsrPyyG5ZYWzQ@mail.gmail.com>
	<447733132.4160093.1471644286616.JavaMail.zimbra@redhat.com>
	<57B80470.5060508@redhat.com>
Message-ID: <CAA9R9O5-0aDLs4LqbwAt4DVssw+=EFH+XQ3FMTCLUG8JPo4aFw@mail.gmail.com>

+1, I think using multiple discovery protocols could be a good idea.

I think it makes sense to use KUBE_PING and DNS together. The same
configuration could have been used for both PetSets (which use DNS) and
normal Kubernetes/OpenShift deployments (which use KUBE_PING). But that is
something that should be tested carefully.

Thanks
Sebastian

On Sat, Aug 20, 2016 at 9:19 AM, Bela Ban <bban at redhat.com> wrote:

> If we add a DNS discovery protocol, it would only be another discovery
> protocol among many, and customers can choose which one to use.
> I'm also thinking of adding the ability to JGroups to use multiple
> discovery protocols in the same stack and combine their result sets into
> one. Not sure though if it makes sense to use KUBE_PING and DNS in the same
> stack...
>
>
> On 20/08/16 00:04, Rob Cernich wrote:
>
>> A couple of things...
>>
>> re. volumes:
>> We also need to consider the mounting behavior for scale down scenarios
>> and for overage scenarios when doing upgrades.  For the latter,
>> OpenShift can spin up pods of the new version before the older version
>> pods have terminated.  This may mean that some volumes from the old pods
>> are orphaned.  We did see this when testing A-MQ during upgrades.  With
>> a single pod, the upgrade process caused the new version to have a new
>> mount and the original mount was left orphaned (another upgrade would
>> cause the newer pod to pick up the orphaned mount, leaving the new mount
>> orphaned).  I believe we worked around this by specifying an overage of
>> 0% during upgrades.  This ensured the new pods would pick up the volumes
>> left behind by the old pods.  (Actually, we were using subdirectories in
>> the mount, since all pods shared the same volume.)
>>
>> re. dns:
>> DNS should work fine as-is, but there are a couple things that you need
>> to consider.
>> 1. Service endpoints are only available in DNS after the pod becomes
>> ready (SVC records on the service name).  Because infinispan attaches
>> itself to the cluster, this meant pods were all started as cluster of
>> one, then merged once they noticed the other pods.  This had a
>> significant impact on startup.  Since then, OpenShift has added the
>> ability to query the endpoints associated with a service as soon as the
>> pod is created, which would allow initialization to work correctly.  To
>> make this work, we'd have to change the form of the DNS query to pick up
>> the service endpoints (I forget the naming scheme).
>>
>> Another thing to keep in mind is that looking up pods by labels allows
>> any pod with the specified label to be added to the cluster.  I'm not
>> sure of a use case for this, but it would allow other deployments to be
>> included in the cluster.  (You could also argue that the service is the
>> authority for this and any pod with said label would be added as a
>> service endpoint, thus achieving the same behavior...probably more
>> simply too.)
>>
>> Lastly, DNS was a little flaky when we first implemented this, which was
>> part of the reason we went straight to kubernetes.  Users were using
>> dnsmasq with wildcards that worked well for routes, but ended up routing
>> services to the router ip instead of pod ip.  Needless to say, there
>> were a lot of complications trying to use DNS and debug user problems
>> with service resolution.
>>
>> Hope that helps,
>> Rob
>>
>> ------------------------------------------------------------------------
>>
>>     Hey Bela!
>>
>>     No no, the resolution can be done with pure JDK.
>>
>>     Thanks
>>     Sebastian
>>
>>     On Fri, Aug 19, 2016 at 11:18 AM, Bela Ban <bban at redhat.com
>>     <mailto:bban at redhat.com>> wrote:
>>
>>         Hi Sebastian
>>
>>         the usual restrictions apply: if DNS discovery depends on
>>         external libs, then it should be hosted in jgroups-extras,
>>         otherwise we can add it to JGroups itself.
>>
>>         On 19/08/16 11:00, Sebastian Laskawiec wrote:
>>
>>             Hey!
>>
>>             I've been playing with Kubernetes PetSets [1] for a while
>>             and I'd like
>>             to share some thoughts. Before I dig in, let me give you
>>             some PetSets
>>             highlights:
>>
>>                * PetSets are alpha resources for managing stateful apps
>>             in Kubernetes
>>                  1.3 (and OpenShift Origin 1.3).
>>                * Since this is an alpha resource, there are no
>>             guarantees about
>>                  backwards compatibility. Alpha resources can also be
>>             disabled in
>>                  some public cloud providers (you can control which API
>>             versions are
>>                  accessible [2]).
>>                * PetSets allows starting pods in sequence (not relevant
>>             for us, but
>>                  this is a killer feature for master-slave systems).
>>                * Each Pod has it's own unique entry in DNS, which makes
>>             discovery
>>                  very simple (I'll dig into that a bit later)
>>                * Volumes are always mounted to the same Pods, which is
>>             very important
>>                  in Cache Store scenarios when we restart pods (e.g.
>>             Rolling Upgrades
>>                  [3]).
>>
>>             Thoughts and ideas after spending some time playing with
>>             this feature:
>>
>>                * PetSets make discovery a lot easier. It's a combination
>>             of two
>>                  things - Headless Services [4] which create multiple A
>>             records in
>>                  DNS and predictable host names. Each Pod has it's own
>>             unique DNS
>>                  entry following pattern:
>>             {PetSetName}-{PodIndex}.{ServiceName} [5].
>>                  Here's an example of an Infinispan PetSet deployed on
>>             my local
>>                  cluster [6]. As you can see we have all domain names
>>             and IPs from a
>>                  single DNS query.
>>                * Maybe we could perform discovery using this mechanism?
>>             I'm aware of
>>                  DNS discovery implemented in KUBE_PING [7][8] but the
>>             code looks
>>                  trivial [9] so maybe it should be implement inside
>>             JGroups? @Bela -
>>                  WDYT?
>>                * PetSets do not integrate well with OpenShift 'new-app'
>>             command. In
>>                  other words, our users will need to use provided yaml
>>             (or json)
>>                  files to create Infinispan cluster. It's not a
>>             show-stopper but it's
>>                  a bit less convenient than 'oc new-app'.
>>                * Since PetSets are alpha resources they need to be
>>             considered as
>>                  secondary way to deploy Infinispan on Kubernetes and
>>             OpenShift.
>>                * Finally, the persistent volumes - since a Pod always
>>             gets the same
>>                  volume, it would be safe to use any file-based cache
>> store.
>>
>>             If you'd like to play with PetSets on your local
>>             environment, here are
>>             necessary yaml files [10].
>>
>>             Thanks
>>             Sebastian
>>
>>
>>             [1] http://kubernetes.io/docs/user-guide/petset/
>>             [2] For checking which APIs are accessible, use 'kubectl
>>             api-versions'
>>             [3]
>>             http://infinispan.org/docs/stable/user_guide/user_guide.html
>> #_Rolling_chapter
>>             [4]
>>             http://kubernetes.io/docs/user-guide/services/#headless-serv
>> ices
>>             [5] http://kubernetes.io/docs/user
>> -guide/petset/#peer-discovery
>>             [6]
>>             https://gist.github.com/slaskawi/0866e63a39276f8ab6637622971
>> 6a676
>>             [7]
>>             https://github.com/jboss-openshift/openshift-ping/tree/maste
>> r/dns
>>             [8]
>>             https://github.com/jgroups-extras/jgroups-kubernetes/tree/
>> master/dns
>>             [9] http://stackoverflow.com/a/12405896/562699
>>             [10] You might need to adjust ImageStream.
>>             https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c
>> 5f2d0
>>
>>
>>         --
>>         Bela Ban, JGroups lead (http://www.jgroups.org)
>>
>>
>>
>>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160822/53c0e94e/attachment-0001.html 

From slaskawi at redhat.com  Mon Aug 22 02:13:19 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Mon, 22 Aug 2016 08:13:19 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <994F7890-498F-49F4-9014-CFF6253AFDE5@redhat.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<994F7890-498F-49F4-9014-CFF6253AFDE5@redhat.com>
Message-ID: <CAA9R9O5VDV8N4S7WLQ5u=ZZdktTQ+=dufw=5iwMpyeBWegz+Fw@mail.gmail.com>

Hey Kevin!

The timing for looking into PetSets is perfect I think. Kubernetes upstream
folks are thinking about Sticky IPs [16] (which are not essential for
Infinispan but other projects might be really interested in this) and I
asked some time ago about exposing PetSets to the outside world [17] (which
is essential for accessing the cluster using Hot Rod client).

Please keep me in the loop. I'm really interested in this.

Thanks
Sebastian


[16] https://github.com/kubernetes/kubernetes/issues/28969
[17] https://groups.google.com/forum/#!topic/kubernetes-dev/K-9KA_wMbmk

On Sun, Aug 21, 2016 at 7:05 PM, Kevin Conner <kconner at redhat.com> wrote:

> Apologies, I?ve been travelling to London for the JDG meeting.
>
> On 19 Aug 2016, at 10:00, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> > I've been playing with Kubernetes PetSets [1] for a while and I'd like
> to share some thoughts. Before I dig in, let me give you some PetSets
> highlights:
> >       ? PetSets are alpha resources for managing stateful apps in
> Kubernetes 1.3 (and OpenShift Origin 1.3).
> >       ? Since this is an alpha resource, there are no guarantees about
> backwards compatibility. Alpha resources can also be disabled in some
> public cloud providers (you can control which API versions are accessible
> [2]).
> >       ? PetSets allows starting pods in sequence (not relevant for us,
> but this is a killer feature for master-slave systems).
> >       ? Each Pod has it's own unique entry in DNS, which makes discovery
> very simple (I'll dig into that a bit later)
> >       ? Volumes are always mounted to the same Pods, which is very
> important in Cache Store scenarios when we restart pods (e.g. Rolling
> Upgrades [3]).
> > Thoughts and ideas after spending some time playing with this feature:
> >       ? PetSets make discovery a lot easier. It's a combination of two
> things - Headless Services [4] which create multiple A records in DNS and
> predictable host names. Each Pod has it's own unique DNS entry following
> pattern: {PetSetName}-{PodIndex}.{ServiceName} [5]. Here's an example of
> an Infinispan PetSet deployed on my local cluster [6]. As you can see we
> have all domain names and IPs from a single DNS query.
> >       ? Maybe we could perform discovery using this mechanism? I'm aware
> of DNS discovery implemented in KUBE_PING [7][8] but the code looks trivial
> [9] so maybe it should be implement inside JGroups? @Bela - WDYT?
> >       ? PetSets do not integrate well with OpenShift 'new-app' command.
> In other words, our users will need to use provided yaml (or json) files to
> create Infinispan cluster. It's not a show-stopper but it's a bit less
> convenient than 'oc new-app'.
> >       ? Since PetSets are alpha resources they need to be considered as
> secondary way to deploy Infinispan on Kubernetes and OpenShift.
> >       ? Finally, the persistent volumes - since a Pod always gets the
> same volume, it would be safe to use any file-based cache store.
> > If you'd like to play with PetSets on your local environment, here are
> necessary yaml files [10].
>
> PetSets are still in extremely early stages and I am part of a working
> group that has recently formed to cover the introduction of this within the
> products, including driving any necessary changes back upstream.  There
> will be Cloud Enablement involvement in this effort and any adoption of
> petsets will likely go through the existing project.
>
>         Kev
>
> --
> JBoss by Red Hat
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160822/60d070db/attachment.html 

From slaskawi at redhat.com  Mon Aug 22 02:23:27 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Mon, 22 Aug 2016 08:23:27 +0200
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <57B76D47.10907@redhat.com>
References: <57B76D47.10907@redhat.com>
Message-ID: <CAA9R9O59TULzWZjDReocuZCYSsRTUWmQeF93JuXksgxj_ChWWw@mail.gmail.com>

+1. I like the idea.

Thanks
Sebastian

On Fri, Aug 19, 2016 at 10:34 PM, Radim Vansa <rvansa at redhat.com> wrote:

> Hi,
>
> as I am trying to simplify current entry wrapping and distribution code,
> I often find that listeners can get wrong previous value in the event,
> and it sometimes forces the command to load the value even if it is not
> needed for the command.
>
> I am wondering if we should change the previous value in events to
> Optional - we can usually at least detect that we cannot provide a
> reliable value (e.g. after retry due to topology change, or because the
> command did not bothered to load the previous value from cache loader)
> and return empty Optional.
>
> WDYT?
>
> Radim
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160822/d5cdb9e6/attachment.html 

From manohar.sl at ericsson.com  Mon Aug 22 02:27:13 2016
From: manohar.sl at ericsson.com (Manohar SL)
Date: Mon, 22 Aug 2016 06:27:13 +0000
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
Message-ID: <DB5PR0701MB20720E402F49143C120B4ABEEEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>

Hi Sebastian/Radim,

Any help on this would be really great :-)
Kindly let us know if you would need any further information on this.

Regs,
Manohar.

-----Original Message-----
From: infinispan-dev-bounces at lists.jboss.org [mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Manohar SL
Sent: Saturday, August 20, 2016 1:08 PM
To: Bela Ban <bban at redhat.com>; infinispan -Dev List <infinispan-dev at lists.jboss.org>
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":

Hi Bela Ban,

We have been trying to use JGroups 3.6.8 with Infinispan 8.2.2., in this context we are observing an issue with the Replication Mode of Infinispan usage.
We see the below exception from Jgroups:
JGRP000029: failed sending message to <>:7800 (100 bytes): java.lang.NullPointerException, headers: TCPGOSSIP: [type=GET_MBRS_REQ, cluster=x-cluster], TP: [cluster_name=x-cluster]

The configurations used are highlighted below:

Infinispan config

<infinispan>

    <jgroups>

        <stack-file name="configurationFile" path="config/jgroups.xml"/>

    </jgroups>

    <cache-container>

        <transport cluster="x-cluster" stack="configurationFile" />

<replicated-cache name="transactional-type" mode="SYNC">

            <transaction mode="NON_XA"  locking="OPTIMISTIC" transaction-manager-lookup="org.infinispan.transaction.lookup.JBossStandaloneJTAManagerLookup" auto-commit="true"  />

<locking acquire-timeout="60000"/>

<expiration lifespan="43200000"/>

        </replicated-cache>

    </cache-container>

</infinispan>

 

Jgroups configuration

<!--

    TCP based stack, with flow control and message bundling. This is usually used when IP

    multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).

    Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.

    -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".

    author: Bela Ban

-->

<config xmlns="urn:org:jgroups"

        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="urn:org:jgroups  http://www.jgroups.org/schema/jgroups-3.6.xsd">

 

  <TCP loopback="true"

bind_addr="${jgroups.tcp.address:127.0.0.1}"

bind_port="${jgroups.tcp.port:7800}"

recv_buf_size="${tcp.recv_buf_size:20M}"

send_buf_size="${tcp.send_buf_size:640K}"

discard_incompatible_packets="true"

max_bundle_size="64K"

max_bundle_timeout="30"

enable_bundling="true"

       use_send_queues="true"

sock_conn_timeout="300"

timer_type="new"

timer.min_threads="4"

timer.max_threads="10"

timer.keep_alive_time="3000"

timer.queue_max_size="500"

thread_pool.enabled="true"

thread_pool.min_threads="2"

thread_pool.max_threads="30"

thread_pool.keep_alive_time="60000"

thread_pool.queue_enabled="false"

thread_pool.queue_max_size="100"

thread_pool.rejection_policy="discard"

oob_thread_pool.enabled="true"

       oob_thread_pool.min_threads="2"

oob_thread_pool.max_threads="30"

oob_thread_pool.keep_alive_time="60000"

oob_thread_pool.queue_enabled="false"

oob_thread_pool.queue_max_size="100"

oob_thread_pool.rejection_policy="discard"/>

 

    <!-- <TCP_NIO -->

<!-- bind_port="7800" -->

<!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->

<!-- use_send_queues="true" -->

<!-- sock_conn_timeout="300" -->

<!--         reader_threads="3" -->

<!-- writer_threads="3" -->

<!-- processor_threads="0" -->

<!-- processor_minThreads="0" -->

<!-- processor_maxThreads="0" -->

<!-- processor_queueSize="100" -->

<!--         processor_keepAliveTime="9223372036854775807"/> -->

    <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>

    <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"

            port_range="2" timeout="3000" /> -->

    <MERGE2 max_interval="30000" min_interval="10000"/>

    <FD_SOCK/>

    <FD timeout="3000" max_tries="3"/>

    <VERIFY_SUSPECT timeout="1500"/>

    <pbcast.NAKACK

use_mcast_xmit="false"

retransmit_timeout="300,600,1200,2400,4800"

discard_delivered_msgs="false"/>

    <UNICAST2 timeout="300,600,1200"

stable_interval="5000"

max_bytes="1m"/>

    <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="1m"/>

    <pbcast.GMS print_local_addr="false" join_timeout="3000" view_bundling="true"/>

    <UFC max_credits="200k" min_threshold="0.20"/>

    <MFC max_credits="200k" min_threshold="0.20"/>

    <FRAG2 frag_size="60000"/>

    <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />

</config>

Any help on this would be really great.
Kindly let us know if you would need any further information on this.

Regs,
Manohar.

_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev


From bban at redhat.com  Mon Aug 22 04:01:24 2016
From: bban at redhat.com (Bela Ban)
Date: Mon, 22 Aug 2016 10:01:24 +0200
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
Message-ID: <57BAB154.9000601@redhat.com>

1: What does the NPE cause? Incorrect behavior? This is in the discovery 
protocol, so I don't think it is a severe error.

2: If you change the JGroups code to print the exception (or set a 
breakpoint and debug) we'd know what the cause is.

3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?

4: Is this reproduceable? If yes, goto step #2



On 20/08/16 09:38, Manohar SL wrote:
> Hi Bela Ban,
>
> We have been trying to use JGroups 3.6.8 with Infinispan 8.2.2., in this context we are observing an issue with the Replication Mode of Infinispan usage.
> We see the below exception from Jgroups:
> JGRP000029: failed sending message to <>:7800 (100 bytes): java.lang.NullPointerException, headers: TCPGOSSIP: [type=GET_MBRS_REQ, cluster=x-cluster], TP: [cluster_name=x-cluster]
>
> The configurations used are highlighted below:
>
> Infinispan config
>
> <infinispan>
>
>      <jgroups>
>
>          <stack-file name="configurationFile" path="config/jgroups.xml"/>
>
>      </jgroups>
>
>      <cache-container>
>
>          <transport cluster="x-cluster" stack="configurationFile" />
>
> <replicated-cache name="transactional-type" mode="SYNC">
>
>              <transaction mode="NON_XA"  locking="OPTIMISTIC" transaction-manager-lookup="org.infinispan.transaction.lookup.JBossStandaloneJTAManagerLookup" auto-commit="true"  />
>
> <locking acquire-timeout="60000"/>
>
> <expiration lifespan="43200000"/>
>
>          </replicated-cache>
>
>      </cache-container>
>
> </infinispan>
>
>
>
> Jgroups configuration
>
> <!--
>
>      TCP based stack, with flow control and message bundling. This is usually used when IP
>
>      multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
>
>      Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
>
>      -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>
>      author: Bela Ban
>
> -->
>
> <config xmlns="urn:org:jgroups"
>
>          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>
> xsi:schemaLocation="urn:org:jgroups  http://www.jgroups.org/schema/jgroups-3.6.xsd">
>
>
>
>    <TCP loopback="true"
>
> bind_addr="${jgroups.tcp.address:127.0.0.1}"
>
> bind_port="${jgroups.tcp.port:7800}"
>
> recv_buf_size="${tcp.recv_buf_size:20M}"
>
> send_buf_size="${tcp.send_buf_size:640K}"
>
> discard_incompatible_packets="true"
>
> max_bundle_size="64K"
>
> max_bundle_timeout="30"
>
> enable_bundling="true"
>
>         use_send_queues="true"
>
> sock_conn_timeout="300"
>
> timer_type="new"
>
> timer.min_threads="4"
>
> timer.max_threads="10"
>
> timer.keep_alive_time="3000"
>
> timer.queue_max_size="500"
>
> thread_pool.enabled="true"
>
> thread_pool.min_threads="2"
>
> thread_pool.max_threads="30"
>
> thread_pool.keep_alive_time="60000"
>
> thread_pool.queue_enabled="false"
>
> thread_pool.queue_max_size="100"
>
> thread_pool.rejection_policy="discard"
>
> oob_thread_pool.enabled="true"
>
>         oob_thread_pool.min_threads="2"
>
> oob_thread_pool.max_threads="30"
>
> oob_thread_pool.keep_alive_time="60000"
>
> oob_thread_pool.queue_enabled="false"
>
> oob_thread_pool.queue_max_size="100"
>
> oob_thread_pool.rejection_policy="discard"/>
>
>
>
>      <!-- <TCP_NIO -->
>
> <!-- bind_port="7800" -->
>
> <!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>
> <!-- use_send_queues="true" -->
>
> <!-- sock_conn_timeout="300" -->
>
> <!--         reader_threads="3" -->
>
> <!-- writer_threads="3" -->
>
> <!-- processor_threads="0" -->
>
> <!-- processor_minThreads="0" -->
>
> <!-- processor_maxThreads="0" -->
>
> <!-- processor_queueSize="100" -->
>
> <!--         processor_keepAliveTime="9223372036854775807"/> -->
>
>      <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>
>      <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"
>
>              port_range="2" timeout="3000" /> -->
>
>      <MERGE2 max_interval="30000" min_interval="10000"/>
>
>      <FD_SOCK/>
>
>      <FD timeout="3000" max_tries="3"/>
>
>      <VERIFY_SUSPECT timeout="1500"/>
>
>      <pbcast.NAKACK
>
> use_mcast_xmit="false"
>
> retransmit_timeout="300,600,1200,2400,4800"
>
> discard_delivered_msgs="false"/>
>
>      <UNICAST2 timeout="300,600,1200"
>
> stable_interval="5000"
>
> max_bytes="1m"/>
>
>      <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="1m"/>
>
>      <pbcast.GMS print_local_addr="false" join_timeout="3000" view_bundling="true"/>
>
>      <UFC max_credits="200k" min_threshold="0.20"/>
>
>      <MFC max_credits="200k" min_threshold="0.20"/>
>
>      <FRAG2 frag_size="60000"/>
>
>      <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />
>
> </config>
>
> Any help on this would be really great.
> Kindly let us know if you would need any further information on this.
>
> Regs,
> Manohar.
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From gustavo at infinispan.org  Mon Aug 22 04:12:28 2016
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Mon, 22 Aug 2016 09:12:28 +0100
Subject: [infinispan-dev] Switch to AffinityPartitioner by default,
 or even enforce it
In-Reply-To: <CA+nfvwSHvyAYk0WS=WWSO59qtS6msjpAuEh7HPCqc-sS1B0OnQ@mail.gmail.com>
References: <CAFm4XO0NF4kzPwpo=WzbVEawnjXwc0Ai9V03wZcHdn-Buv_w-A@mail.gmail.com>
	<CA+nfvwQQGP8ZwdbsW4Qv1Uox_1i+OVr5aB2UxRYqx_zcZzcnpw@mail.gmail.com>
	<CAFm4XO1sfqg8J-4U+UCr8m3YznTWcdj7cY1wxFTPA1meMC6MrA@mail.gmail.com>
	<CA+nfvwT4SSr7g1mqBj8e7LtaENMQQ4NaiVvqYQ--3ru=OW8=tg@mail.gmail.com>
	<CA+nfvwSHvyAYk0WS=WWSO59qtS6msjpAuEh7HPCqc-sS1B0OnQ@mail.gmail.com>
Message-ID: <CAH8Ud1FXRyE6w9CBHnvda_Ug8WAZca59-APX0=LszoFUHbO=4g@mail.gmail.com>

On Fri, Aug 19, 2016 at 3:39 PM, Dan Berindei <dan.berindei at gmail.com>
wrote:

> I just had a discussion with Adrian, and he reminded me that the key
> partitioners don't yet work with any HotRod client.
>
> Assuming we'll have for HotRod key partitioners soon,
> AffinityPartitioner still won't work with remote caches, where we want
> to keep the key as a byte[]. Instead, we would need a key partitioner
> that knows protobuf and can parse a single field from the object.
>
> So I'd like to withdraw my support for making AffinityPartitioner the
> default :)
>
>

+1 for me, I don't think it should be default. Currently the Hot Rod
protocol
is hardcoded to use a hash function to map keys to segments (byte field
in the header to indicate hash function version to use), so C#, C++, JS
clients would not be able to take advantage of the AffinityPartitioner.

Another thing to consider is for query usage, mapping 1 segment to 1
index shard uses lots of resources, and I've been exploring ways of
making this configurable [1], which potentially could require a different
partitioner.

That does not mean it should never be default, we can revisit this later.


Gustavo


[1] https://issues.jboss.org/browse/ISPN-6225



> Cheers
> Dan
>
>
> On Fri, Aug 19, 2016 at 5:20 PM, Dan Berindei <dan.berindei at gmail.com>
> wrote:
> > On Thu, Aug 18, 2016 at 6:12 PM, Sanne Grinovero <sanne at infinispan.org>
> wrote:
> >> On 18 August 2016 at 14:44, Dan Berindei <dan.berindei at gmail.com>
> wrote:
> >>> First, a question: doesn't query already configure its internal caches
> >>> to use the AffinityPartitioner? Does it need the application caches to
> >>> have AffinityPartitioner enabled, too?
> >>
> >> No, we may provide reasonable defaults so that demos and tutorials
> >> work out of the box but the general recommendation is that people make
> >> their own configuration, as they will need to tune things, or add
> >> CacheStores, etc..
> >>
> >
> > I see... I wonder if we could at least have some templates for the
> > user to extend, without having to copy a bunch of stuff from
> > tutorials?
> >
> >
> >>> I'm ok with making AffinityPartitioner the default key partitioner:
> >>> the less configuration the user has to change to make things work, the
> >>> better. I'm also ok with changing it to use delegation, to make it
> >>> easy for user to compose it with another key partitioner. We should
> >>> just run some benchmarks first to check it doesn't affect performance
> >>> for repl mode reads.
> >>
> >> Ok!
> >> https://issues.jboss.org/browse/ISPN-6956
> >>
> >> I'll send a PR; regarding benchmarking I'll take the optimistic path:
> >> we'll test it after it's integrated, worst case we can take it out.
> >>
> >>> But I feel users should be able to disable it if they want to, so I
> >>> wouldn't always wrap user partitioners. Also, usually less magic ==
> >>> better. Query can validate the configuration and warn the user if the
> >>> configuration doesn't have AffinityPartitioner in place.
> >>
> >> I disagree on this as this would require more options, which make it
> >> harder to use, and there's no good enough reason to give away the
> >> option to disable this; especially as it kills other features.
> >>
> >
> > Well, grouping isn't enabled by default, even though that means
> > AdvancedCache.getGroup(String) doesn't work with the default
> > configuration. Sure, AffinityPartitioner's overhead is probably lower
> > than GroupingPartitioner's for non-targeted keys, but would you want
> > to wrap all key partitioners with AffinityPartitioner if we were
> > already wrapping them all with GroupingPartitioner?
> >
> > TBH I don't know why a user would want to customize the key
> > partitioner when he could implement AffinityTaggedKey instead...
> >
> >
> >> BTW I never mentioned Query :)
> >
> > You did mention your "evil" plans to improve query performance :D
> >
> >
> >> We happen to use this functionality in Query but as I mentioned other
> >> teams are looking forward to use this functionality, so I think we
> >> should promote this as a public API, a new feature of Infinispan core.
> >>
> >
> > AffinityTaggedKey was already in the public API, even before
> > AffinityPartitioner was the default. So now we have these options for
> > influencing the key distribution:
> > * Implementing AffinityTaggedKey
> > * Implementing a custom KeyPartitioner
> > * Grouping: annotating a key method with @Group, or implementing Grouper
> > * Generating random keys and mapping them to nodes with
> KeyAffinityService
> >
> >
> >> So since you want to be able to disable it I won't wrap all user
> >> configured implementations in my PR for ISPN-6956, but I'd prefer it
> >> if as a second step you would reconsider and allow me to wrap them
> >> all.
> >>
> >
> > Feel free to open a separate PR for this, I'm definitely not going to
> > block it from going in.
> >
> > Cheers
> > Dan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160822/db44b197/attachment.html 

From manohar.sl at ericsson.com  Mon Aug 22 04:23:35 2016
From: manohar.sl at ericsson.com (Manohar SL)
Date: Mon, 22 Aug 2016 08:23:35 +0000
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <57BAB154.9000601@redhat.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
Message-ID: <DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>

Hi Bela Ban,

Thanks for the input points.

Answers are highlighted below:

1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
	Actually, we need the Replication Mode of Infinispan, but after Upgrading to Infinispan 8.2.2, the replication mode is not working.
	The data updated in Infinispan is getting written to only the local node and not replicated to other nodes in the Cluster (we have deployed 3 Nodes).
	Since we saw the NPE in JGroups, we suspected this could be the reason for the Replication not functioning as expected, as it also complains of not
	being able to send data to the other nodes in the Cluster.

2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
	Our understating based on the JGroups code analysis, in the context of this issue, is:
		=> In the BaseServer.java file, the below method tries to send data to the other Nodes:
			public void send(Address dest, byte[] data, int offset, int length) throws Exception
		      The below logic is not able to retrieve the Connection Object:
------------------------------------------------------------------------------------------------------------------------
// Get a connection (or create one if not yet existent) and send the data
        Connection conn=null;
        try {
            conn=getConnection(dest);
            conn.send(data, offset, length);
        }
        catch(Exception ex) {
            removeConnectionIfPresent(dest, conn);
            throw ex;
        }
------------------------------------------------------------------------------------------------------------------------
		This eventually leads to the NPE.

3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
	Infinispan 8.2.2 by default uses JGroups 3.6.7, but we tried upgrading to JGroups 3.6.8.
		
4: Is this reproduceable? If yes, goto step #2
	Yes, this is consistently reproducible.

It would be really helpful if you could kindly mail us working JGroups configuration with Infinispan 8.2.2 version.
Also, if you see any issues with our current set of configurations, highlighted in the below mail,  please let us know the needed changes to this.

Thanks again for all the help.

Regs,
Manohar


-----Original Message-----
From: Bela Ban [mailto:bban at redhat.com] 
Sent: Monday, August 22, 2016 1:31 PM
To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List <infinispan-dev at lists.jboss.org>
Cc: Sathish Kumar Tippeshappa <sathish.kumar.tippeshappa at ericsson.com>
Subject: Re: JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":

1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.

2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.

3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?

4: Is this reproduceable? If yes, goto step #2



On 20/08/16 09:38, Manohar SL wrote:
> Hi Bela Ban,
>
> We have been trying to use JGroups 3.6.8 with Infinispan 8.2.2., in this context we are observing an issue with the Replication Mode of Infinispan usage.
> We see the below exception from Jgroups:
> JGRP000029: failed sending message to <>:7800 (100 bytes): 
> java.lang.NullPointerException, headers: TCPGOSSIP: 
> [type=GET_MBRS_REQ, cluster=x-cluster], TP: [cluster_name=x-cluster]
>
> The configurations used are highlighted below:
>
> Infinispan config
>
> <infinispan>
>
>      <jgroups>
>
>          <stack-file name="configurationFile" 
> path="config/jgroups.xml"/>
>
>      </jgroups>
>
>      <cache-container>
>
>          <transport cluster="x-cluster" stack="configurationFile" />
>
> <replicated-cache name="transactional-type" mode="SYNC">
>
>              <transaction mode="NON_XA"  locking="OPTIMISTIC" 
> transaction-manager-lookup="org.infinispan.transaction.lookup.JBossSta
> ndaloneJTAManagerLookup" auto-commit="true"  />
>
> <locking acquire-timeout="60000"/>
>
> <expiration lifespan="43200000"/>
>
>          </replicated-cache>
>
>      </cache-container>
>
> </infinispan>
>
>
>
> Jgroups configuration
>
> <!--
>
>      TCP based stack, with flow control and message bundling. This is 
> usually used when IP
>
>      multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
>
>      Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
>
>      -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>
>      author: Bela Ban
>
> -->
>
> <config xmlns="urn:org:jgroups"
>
>          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>
> xsi:schemaLocation="urn:org:jgroups  
> http://www.jgroups.org/schema/jgroups-3.6.xsd">
>
>
>
>    <TCP loopback="true"
>
> bind_addr="${jgroups.tcp.address:127.0.0.1}"
>
> bind_port="${jgroups.tcp.port:7800}"
>
> recv_buf_size="${tcp.recv_buf_size:20M}"
>
> send_buf_size="${tcp.send_buf_size:640K}"
>
> discard_incompatible_packets="true"
>
> max_bundle_size="64K"
>
> max_bundle_timeout="30"
>
> enable_bundling="true"
>
>         use_send_queues="true"
>
> sock_conn_timeout="300"
>
> timer_type="new"
>
> timer.min_threads="4"
>
> timer.max_threads="10"
>
> timer.keep_alive_time="3000"
>
> timer.queue_max_size="500"
>
> thread_pool.enabled="true"
>
> thread_pool.min_threads="2"
>
> thread_pool.max_threads="30"
>
> thread_pool.keep_alive_time="60000"
>
> thread_pool.queue_enabled="false"
>
> thread_pool.queue_max_size="100"
>
> thread_pool.rejection_policy="discard"
>
> oob_thread_pool.enabled="true"
>
>         oob_thread_pool.min_threads="2"
>
> oob_thread_pool.max_threads="30"
>
> oob_thread_pool.keep_alive_time="60000"
>
> oob_thread_pool.queue_enabled="false"
>
> oob_thread_pool.queue_max_size="100"
>
> oob_thread_pool.rejection_policy="discard"/>
>
>
>
>      <!-- <TCP_NIO -->
>
> <!-- bind_port="7800" -->
>
> <!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>
> <!-- use_send_queues="true" -->
>
> <!-- sock_conn_timeout="300" -->
>
> <!--         reader_threads="3" -->
>
> <!-- writer_threads="3" -->
>
> <!-- processor_threads="0" -->
>
> <!-- processor_minThreads="0" -->
>
> <!-- processor_maxThreads="0" -->
>
> <!-- processor_queueSize="100" -->
>
> <!--         processor_keepAliveTime="9223372036854775807"/> -->
>
>      <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>
>      <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"
>
>              port_range="2" timeout="3000" /> -->
>
>      <MERGE2 max_interval="30000" min_interval="10000"/>
>
>      <FD_SOCK/>
>
>      <FD timeout="3000" max_tries="3"/>
>
>      <VERIFY_SUSPECT timeout="1500"/>
>
>      <pbcast.NAKACK
>
> use_mcast_xmit="false"
>
> retransmit_timeout="300,600,1200,2400,4800"
>
> discard_delivered_msgs="false"/>
>
>      <UNICAST2 timeout="300,600,1200"
>
> stable_interval="5000"
>
> max_bytes="1m"/>
>
>      <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" 
> max_bytes="1m"/>
>
>      <pbcast.GMS print_local_addr="false" join_timeout="3000" 
> view_bundling="true"/>
>
>      <UFC max_credits="200k" min_threshold="0.20"/>
>
>      <MFC max_credits="200k" min_threshold="0.20"/>
>
>      <FRAG2 frag_size="60000"/>
>
>      <RSVP timeout="60000" resend_interval="500" 
> ack_on_delivery="false" />
>
> </config>
>
> Any help on this would be really great.
> Kindly let us know if you would need any further information on this.
>
> Regs,
> Manohar.
>

--
Bela Ban, JGroups lead (http://www.jgroups.org)



From bban at redhat.com  Mon Aug 22 04:28:16 2016
From: bban at redhat.com (Bela Ban)
Date: Mon, 22 Aug 2016 10:28:16 +0200
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
	<DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
Message-ID: <57BAB7A0.4070204@redhat.com>



On 22/08/16 10:23, Manohar SL wrote:
> Hi Bela Ban,
>
> Thanks for the input points.
>
> Answers are highlighted below:
>
> 1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
> 	Actually, we need the Replication Mode of Infinispan, but after Upgrading to Infinispan 8.2.2, the replication mode is not working.
> 	The data updated in Infinispan is getting written to only the local node and not replicated to other nodes in the Cluster (we have deployed 3 Nodes).
> 	Since we saw the NPE in JGroups, we suspected this could be the reason for the Replication not functioning as expected, as it also complains of not
> 	being able to send data to the other nodes in the Cluster.

OK

> 2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
> 	Our understating based on the JGroups code analysis, in the context of this issue, is:
> 		=> In the BaseServer.java file, the below method tries to send data to the other Nodes:
> 			public void send(Address dest, byte[] data, int offset, int length) throws Exception
> 		      The below logic is not able to retrieve the Connection Object:
> ------------------------------------------------------------------------------------------------------------------------
> // Get a connection (or create one if not yet existent) and send the data
>          Connection conn=null;
>          try {
>              conn=getConnection(dest);
>              conn.send(data, offset, length);
>          }
>          catch(Exception ex) {
>              removeConnectionIfPresent(dest, conn);
>              throw ex;
>          }
> ------------------------------------------------------------------------------------------------------------------------
> 		This eventually leads to the NPE.


Insert a ex.printStackTrace() statement in the catch clause.


> 3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
> 	Infinispan 8.2.2 by default uses JGroups 3.6.7, but we tried upgrading to JGroups 3.6.8.

Then 3.6.8 may not be supported. I'm sure REPL mode wa tested before 
releasing 8.2.2, so perhaps 3.6.8 doesn't work with 8.2.2...
  		
> 4: Is this reproduceable? If yes, goto step #2
> 	Yes, this is consistently reproducible.
>
> It would be really helpful if you could kindly mail us working JGroups configuration with Infinispan 8.2.2 version.

Well, I suggest take the TCP based config shipped with 8.2.2 and use it 
as base, ie. make changes to it.

> Also, if you see any issues with our current set of configurations, highlighted in the below mail,  please let us know the needed changes to this.
>
> Thanks again for all the help.
>
> Regs,
> Manohar
>
>
> -----Original Message-----
> From: Bela Ban [mailto:bban at redhat.com]
> Sent: Monday, August 22, 2016 1:31 PM
> To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List <infinispan-dev at lists.jboss.org>
> Cc: Sathish Kumar Tippeshappa <sathish.kumar.tippeshappa at ericsson.com>
> Subject: Re: JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":
>
> 1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
>
> 2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
>
> 3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
>
> 4: Is this reproduceable? If yes, goto step #2
>
>
>
> On 20/08/16 09:38, Manohar SL wrote:
>> Hi Bela Ban,
>>
>> We have been trying to use JGroups 3.6.8 with Infinispan 8.2.2., in this context we are observing an issue with the Replication Mode of Infinispan usage.
>> We see the below exception from Jgroups:
>> JGRP000029: failed sending message to <>:7800 (100 bytes):
>> java.lang.NullPointerException, headers: TCPGOSSIP:
>> [type=GET_MBRS_REQ, cluster=x-cluster], TP: [cluster_name=x-cluster]
>>
>> The configurations used are highlighted below:
>>
>> Infinispan config
>>
>> <infinispan>
>>
>>       <jgroups>
>>
>>           <stack-file name="configurationFile"
>> path="config/jgroups.xml"/>
>>
>>       </jgroups>
>>
>>       <cache-container>
>>
>>           <transport cluster="x-cluster" stack="configurationFile" />
>>
>> <replicated-cache name="transactional-type" mode="SYNC">
>>
>>               <transaction mode="NON_XA"  locking="OPTIMISTIC"
>> transaction-manager-lookup="org.infinispan.transaction.lookup.JBossSta
>> ndaloneJTAManagerLookup" auto-commit="true"  />
>>
>> <locking acquire-timeout="60000"/>
>>
>> <expiration lifespan="43200000"/>
>>
>>           </replicated-cache>
>>
>>       </cache-container>
>>
>> </infinispan>
>>
>>
>>
>> Jgroups configuration
>>
>> <!--
>>
>>       TCP based stack, with flow control and message bundling. This is
>> usually used when IP
>>
>>       multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
>>
>>       Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
>>
>>       -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>>
>>       author: Bela Ban
>>
>> -->
>>
>> <config xmlns="urn:org:jgroups"
>>
>>           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>
>> xsi:schemaLocation="urn:org:jgroups
>> http://www.jgroups.org/schema/jgroups-3.6.xsd">
>>
>>
>>
>>     <TCP loopback="true"
>>
>> bind_addr="${jgroups.tcp.address:127.0.0.1}"
>>
>> bind_port="${jgroups.tcp.port:7800}"
>>
>> recv_buf_size="${tcp.recv_buf_size:20M}"
>>
>> send_buf_size="${tcp.send_buf_size:640K}"
>>
>> discard_incompatible_packets="true"
>>
>> max_bundle_size="64K"
>>
>> max_bundle_timeout="30"
>>
>> enable_bundling="true"
>>
>>          use_send_queues="true"
>>
>> sock_conn_timeout="300"
>>
>> timer_type="new"
>>
>> timer.min_threads="4"
>>
>> timer.max_threads="10"
>>
>> timer.keep_alive_time="3000"
>>
>> timer.queue_max_size="500"
>>
>> thread_pool.enabled="true"
>>
>> thread_pool.min_threads="2"
>>
>> thread_pool.max_threads="30"
>>
>> thread_pool.keep_alive_time="60000"
>>
>> thread_pool.queue_enabled="false"
>>
>> thread_pool.queue_max_size="100"
>>
>> thread_pool.rejection_policy="discard"
>>
>> oob_thread_pool.enabled="true"
>>
>>          oob_thread_pool.min_threads="2"
>>
>> oob_thread_pool.max_threads="30"
>>
>> oob_thread_pool.keep_alive_time="60000"
>>
>> oob_thread_pool.queue_enabled="false"
>>
>> oob_thread_pool.queue_max_size="100"
>>
>> oob_thread_pool.rejection_policy="discard"/>
>>
>>
>>
>>       <!-- <TCP_NIO -->
>>
>> <!-- bind_port="7800" -->
>>
>> <!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>>
>> <!-- use_send_queues="true" -->
>>
>> <!-- sock_conn_timeout="300" -->
>>
>> <!--         reader_threads="3" -->
>>
>> <!-- writer_threads="3" -->
>>
>> <!-- processor_threads="0" -->
>>
>> <!-- processor_minThreads="0" -->
>>
>> <!-- processor_maxThreads="0" -->
>>
>> <!-- processor_queueSize="100" -->
>>
>> <!--         processor_keepAliveTime="9223372036854775807"/> -->
>>
>>       <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>>
>>       <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"
>>
>>               port_range="2" timeout="3000" /> -->
>>
>>       <MERGE2 max_interval="30000" min_interval="10000"/>
>>
>>       <FD_SOCK/>
>>
>>       <FD timeout="3000" max_tries="3"/>
>>
>>       <VERIFY_SUSPECT timeout="1500"/>
>>
>>       <pbcast.NAKACK
>>
>> use_mcast_xmit="false"
>>
>> retransmit_timeout="300,600,1200,2400,4800"
>>
>> discard_delivered_msgs="false"/>
>>
>>       <UNICAST2 timeout="300,600,1200"
>>
>> stable_interval="5000"
>>
>> max_bytes="1m"/>
>>
>>       <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000"
>> max_bytes="1m"/>
>>
>>       <pbcast.GMS print_local_addr="false" join_timeout="3000"
>> view_bundling="true"/>
>>
>>       <UFC max_credits="200k" min_threshold="0.20"/>
>>
>>       <MFC max_credits="200k" min_threshold="0.20"/>
>>
>>       <FRAG2 frag_size="60000"/>
>>
>>       <RSVP timeout="60000" resend_interval="500"
>> ack_on_delivery="false" />
>>
>> </config>
>>
>> Any help on this would be really great.
>> Kindly let us know if you would need any further information on this.
>>
>> Regs,
>> Manohar.
>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From anistor at redhat.com  Mon Aug 22 04:31:00 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Mon, 22 Aug 2016 11:31:00 +0300
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <57B76D47.10907@redhat.com>
References: <57B76D47.10907@redhat.com>
Message-ID: <c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>

Hi Radim,

Continuous query is built on top of these listeners. CQ _always_ needs 
the previous value and it is very convenient in this case that the 
command is forced to load the previous value. I imagine there may be 
other use cases where we cannot live without the prev value.

I think the listener should be able to state if it needs the prev value 
at registration time. Maybe add a new attribute in the Listener 
annotation? Similar to how we handled Observation.

Adrian

On 08/19/2016 11:34 PM, Radim Vansa wrote:
> Hi,
>
> as I am trying to simplify current entry wrapping and distribution code,
> I often find that listeners can get wrong previous value in the event,
> and it sometimes forces the command to load the value even if it is not
> needed for the command.
>
> I am wondering if we should change the previous value in events to
> Optional - we can usually at least detect that we cannot provide a
> reliable value (e.g. after retry due to topology change, or because the
> command did not bothered to load the previous value from cache loader)
> and return empty Optional.
>
> WDYT?
>
> Radim
>


From manohar.sl at ericsson.com  Mon Aug 22 06:56:38 2016
From: manohar.sl at ericsson.com (Manohar SL)
Date: Mon, 22 Aug 2016 10:56:38 +0000
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <57BAB7A0.4070204@redhat.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
	<DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB7A0.4070204@redhat.com>
Message-ID: <DB5PR0701MB2072E8D44B7633509FC775E9EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>

Hi Bela Ben,

We just now retested the combination of "Infinispan 8.2.2 + JGroups 3.6.7", we face the same issue,
data replication across Nodes does not go through.

Highlighted below are configurations used (derived from the Base configurations):
Infinispan-config.xml:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
<infinispan>
    <jgroups>
        <stack-file name="configurationFile" path="config/jgroups.xml"/>
    </jgroups>
    <cache-container> 
        <transport  cluster="x-cluster" stack="configurationFile" />
        <replicated-cache name="transactional-type" mode="SYNC"> 
            <transaction mode="NON_XA"  locking="OPTIMISTIC" transaction-manager-lookup="org.infinispan.transaction.lookup.GenericTransactionManagerLookup"  auto-commit="true"  />
              <locking acquire-timeout="60000"/>
              <expiration lifespan="43200000"/>  
	<state-transfer enabled="true" timeout="240000" chunk-size="10000" /> 
	<locking isolation="READ_COMMITTED"  acquire-timeout="20000"  write-skew="false"  concurrency-level="5000"  striping="false"  /> 
        </replicated-cache>
    </cache-container>
</infinispan>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

JGroups.xml
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

<!--
    TCP based stack, with flow control and message bundling. This is usually used when IP
    multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
    Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
    -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
    author: Bela Ban
-->
<config xmlns="urn:org:jgroups"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:org:jgroups  http://www.jgroups.org/schema/jgroups-3.6.xsd">

  <TCP loopback="true"
       bind_addr="${jgroups.tcp.address:127.0.0.1}"
       bind_port="${jgroups.tcp.port:7800}"
       recv_buf_size="${tcp.recv_buf_size:130k}"
       send_buf_size="${tcp.send_buf_size:130k}"
       discard_incompatible_packets="true"
       max_bundle_size="64K"
       max_bundle_timeout="30"
       enable_bundling="true"
       use_send_queues="true"
       sock_conn_timeout="300"
       timer_type="new"
       timer.min_threads="4"
       timer.max_threads="10"
       timer.keep_alive_time="3000"
       timer.queue_max_size="500"
       thread_pool.enabled="true"
       thread_pool.min_threads="2"
       thread_pool.max_threads="30"
       thread_pool.keep_alive_time="60000"
       thread_pool.queue_enabled="false"
       thread_pool.queue_max_size="100"
       thread_pool.rejection_policy="discard"
       oob_thread_pool.enabled="true"
       oob_thread_pool.min_threads="2"
       oob_thread_pool.max_threads="30"
       oob_thread_pool.keep_alive_time="60000"
       oob_thread_pool.queue_enabled="false"
       oob_thread_pool.queue_max_size="100"
       oob_thread_pool.rejection_policy="discard"
	   />

    <!-- <TCP_NIO -->
    <!--         bind_port="7800" -->
    <!--         bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
    <!--         use_send_queues="true" -->
    <!--         sock_conn_timeout="300" -->
    <!--         reader_threads="3" -->
    <!--         writer_threads="3" -->
    <!--         processor_threads="0" -->
    <!--         processor_minThreads="0" -->
    <!--         processor_maxThreads="0" -->
    <!--         processor_queueSize="100" -->
    <!--         processor_keepAliveTime="9223372036854775807"/> -->
    <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
    <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}" 
            port_range="2"  timeout="3000" /> -->
    <MERGE2 max_interval="30000" min_interval="10000"/>
    <FD_SOCK/>
    <FD timeout="3000" max_tries="3"/>
    <VERIFY_SUSPECT timeout="1500"/>
    <pbcast.NAKACK
        use_mcast_xmit="false"
        retransmit_timeout="300,600,1200,2400,4800"
        discard_delivered_msgs="true"/>
    <UNICAST2 timeout="300,600,1200"
              stable_interval="5000"
			  log_not_found_msgs="true"
              max_bytes="400000"/>
    <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="400000"/>
    <pbcast.GMS print_local_addr="true" join_timeout="5000" merge_timeout="5000" log_collect_msgs="true" log_view_warnings="true" max_join_attempts="0" view_bundling="true"/>
    <UFC max_credits="200k" min_threshold="0.20"/>
    <MFC max_credits="200k" min_threshold="0.20"/>
    <FRAG2 frag_size="35000"/>
    <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" throw_exception_on_timeout="true" />
    
</config>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Would be very helpful if you could kindly provide us some working combination of xml configs with "Infinsipan 8.2.2 + JGroup 3.6.7" or
let us know what configuration needs to be changed above.

Regs,
Manohar.

-----Original Message-----
From: Bela Ban [mailto:bban at redhat.com] 
Sent: Monday, August 22, 2016 1:58 PM
To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List <infinispan-dev at lists.jboss.org>
Cc: Sathish Kumar Tippeshappa <sathish.kumar.tippeshappa at ericsson.com>
Subject: Re: JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":



On 22/08/16 10:23, Manohar SL wrote:
> Hi Bela Ban,
>
> Thanks for the input points.
>
> Answers are highlighted below:
>
> 1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
> 	Actually, we need the Replication Mode of Infinispan, but after Upgrading to Infinispan 8.2.2, the replication mode is not working.
> 	The data updated in Infinispan is getting written to only the local node and not replicated to other nodes in the Cluster (we have deployed 3 Nodes).
> 	Since we saw the NPE in JGroups, we suspected this could be the reason for the Replication not functioning as expected, as it also complains of not
> 	being able to send data to the other nodes in the Cluster.

OK

> 2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
> 	Our understating based on the JGroups code analysis, in the context of this issue, is:
> 		=> In the BaseServer.java file, the below method tries to send data to the other Nodes:
> 			public void send(Address dest, byte[] data, int offset, int length) throws Exception
> 		      The below logic is not able to retrieve the Connection Object:
> ----------------------------------------------------------------------
> --------------------------------------------------
> // Get a connection (or create one if not yet existent) and send the data
>          Connection conn=null;
>          try {
>              conn=getConnection(dest);
>              conn.send(data, offset, length);
>          }
>          catch(Exception ex) {
>              removeConnectionIfPresent(dest, conn);
>              throw ex;
>          }
> ------------------------------------------------------------------------------------------------------------------------
> 		This eventually leads to the NPE.


Insert a ex.printStackTrace() statement in the catch clause.


> 3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
> 	Infinispan 8.2.2 by default uses JGroups 3.6.7, but we tried upgrading to JGroups 3.6.8.

Then 3.6.8 may not be supported. I'm sure REPL mode wa tested before releasing 8.2.2, so perhaps 3.6.8 doesn't work with 8.2.2...
  		
> 4: Is this reproduceable? If yes, goto step #2
> 	Yes, this is consistently reproducible.
>
> It would be really helpful if you could kindly mail us working JGroups configuration with Infinispan 8.2.2 version.

Well, I suggest take the TCP based config shipped with 8.2.2 and use it as base, ie. make changes to it.

> Also, if you see any issues with our current set of configurations, highlighted in the below mail,  please let us know the needed changes to this.
>
> Thanks again for all the help.
>
> Regs,
> Manohar
>
>
> -----Original Message-----
> From: Bela Ban [mailto:bban at redhat.com]
> Sent: Monday, August 22, 2016 1:31 PM
> To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List 
> <infinispan-dev at lists.jboss.org>
> Cc: Sathish Kumar Tippeshappa <sathish.kumar.tippeshappa at ericsson.com>
> Subject: Re: JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":
>
> 1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
>
> 2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
>
> 3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
>
> 4: Is this reproduceable? If yes, goto step #2
>
>
>
> On 20/08/16 09:38, Manohar SL wrote:
>> Hi Bela Ban,
>>
>> We have been trying to use JGroups 3.6.8 with Infinispan 8.2.2., in this context we are observing an issue with the Replication Mode of Infinispan usage.
>> We see the below exception from Jgroups:
>> JGRP000029: failed sending message to <>:7800 (100 bytes):
>> java.lang.NullPointerException, headers: TCPGOSSIP:
>> [type=GET_MBRS_REQ, cluster=x-cluster], TP: [cluster_name=x-cluster]
>>
>> The configurations used are highlighted below:
>>
>> Infinispan config
>>
>> <infinispan>
>>
>>       <jgroups>
>>
>>           <stack-file name="configurationFile"
>> path="config/jgroups.xml"/>
>>
>>       </jgroups>
>>
>>       <cache-container>
>>
>>           <transport cluster="x-cluster" stack="configurationFile" />
>>
>> <replicated-cache name="transactional-type" mode="SYNC">
>>
>>               <transaction mode="NON_XA"  locking="OPTIMISTIC"
>> transaction-manager-lookup="org.infinispan.transaction.lookup.JBossSt
>> a ndaloneJTAManagerLookup" auto-commit="true"  />
>>
>> <locking acquire-timeout="60000"/>
>>
>> <expiration lifespan="43200000"/>
>>
>>           </replicated-cache>
>>
>>       </cache-container>
>>
>> </infinispan>
>>
>>
>>
>> Jgroups configuration
>>
>> <!--
>>
>>       TCP based stack, with flow control and message bundling. This 
>> is usually used when IP
>>
>>       multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
>>
>>       Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
>>
>>       -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>>
>>       author: Bela Ban
>>
>> -->
>>
>> <config xmlns="urn:org:jgroups"
>>
>>           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>
>> xsi:schemaLocation="urn:org:jgroups
>> http://www.jgroups.org/schema/jgroups-3.6.xsd">
>>
>>
>>
>>     <TCP loopback="true"
>>
>> bind_addr="${jgroups.tcp.address:127.0.0.1}"
>>
>> bind_port="${jgroups.tcp.port:7800}"
>>
>> recv_buf_size="${tcp.recv_buf_size:20M}"
>>
>> send_buf_size="${tcp.send_buf_size:640K}"
>>
>> discard_incompatible_packets="true"
>>
>> max_bundle_size="64K"
>>
>> max_bundle_timeout="30"
>>
>> enable_bundling="true"
>>
>>          use_send_queues="true"
>>
>> sock_conn_timeout="300"
>>
>> timer_type="new"
>>
>> timer.min_threads="4"
>>
>> timer.max_threads="10"
>>
>> timer.keep_alive_time="3000"
>>
>> timer.queue_max_size="500"
>>
>> thread_pool.enabled="true"
>>
>> thread_pool.min_threads="2"
>>
>> thread_pool.max_threads="30"
>>
>> thread_pool.keep_alive_time="60000"
>>
>> thread_pool.queue_enabled="false"
>>
>> thread_pool.queue_max_size="100"
>>
>> thread_pool.rejection_policy="discard"
>>
>> oob_thread_pool.enabled="true"
>>
>>          oob_thread_pool.min_threads="2"
>>
>> oob_thread_pool.max_threads="30"
>>
>> oob_thread_pool.keep_alive_time="60000"
>>
>> oob_thread_pool.queue_enabled="false"
>>
>> oob_thread_pool.queue_max_size="100"
>>
>> oob_thread_pool.rejection_policy="discard"/>
>>
>>
>>
>>       <!-- <TCP_NIO -->
>>
>> <!-- bind_port="7800" -->
>>
>> <!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>>
>> <!-- use_send_queues="true" -->
>>
>> <!-- sock_conn_timeout="300" -->
>>
>> <!--         reader_threads="3" -->
>>
>> <!-- writer_threads="3" -->
>>
>> <!-- processor_threads="0" -->
>>
>> <!-- processor_minThreads="0" -->
>>
>> <!-- processor_maxThreads="0" -->
>>
>> <!-- processor_queueSize="100" -->
>>
>> <!--         processor_keepAliveTime="9223372036854775807"/> -->
>>
>>       <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>>
>>       <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"
>>
>>               port_range="2" timeout="3000" /> -->
>>
>>       <MERGE2 max_interval="30000" min_interval="10000"/>
>>
>>       <FD_SOCK/>
>>
>>       <FD timeout="3000" max_tries="3"/>
>>
>>       <VERIFY_SUSPECT timeout="1500"/>
>>
>>       <pbcast.NAKACK
>>
>> use_mcast_xmit="false"
>>
>> retransmit_timeout="300,600,1200,2400,4800"
>>
>> discard_delivered_msgs="false"/>
>>
>>       <UNICAST2 timeout="300,600,1200"
>>
>> stable_interval="5000"
>>
>> max_bytes="1m"/>
>>
>>       <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000"
>> max_bytes="1m"/>
>>
>>       <pbcast.GMS print_local_addr="false" join_timeout="3000"
>> view_bundling="true"/>
>>
>>       <UFC max_credits="200k" min_threshold="0.20"/>
>>
>>       <MFC max_credits="200k" min_threshold="0.20"/>
>>
>>       <FRAG2 frag_size="60000"/>
>>
>>       <RSVP timeout="60000" resend_interval="500"
>> ack_on_delivery="false" />
>>
>> </config>
>>
>> Any help on this would be really great.
>> Kindly let us know if you would need any further information on this.
>>
>> Regs,
>> Manohar.
>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>

--
Bela Ban, JGroups lead (http://www.jgroups.org)



From bban at redhat.com  Mon Aug 22 07:26:50 2016
From: bban at redhat.com (Bela Ban)
Date: Mon, 22 Aug 2016 13:26:50 +0200
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <DB5PR0701MB2072E8D44B7633509FC775E9EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
	<DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB7A0.4070204@redhat.com>
	<DB5PR0701MB2072E8D44B7633509FC775E9EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
Message-ID: <57BAE17A.2060202@redhat.com>



On 22/08/16 12:56, Manohar SL wrote:
> Hi Bela Ben,
>
> We just now retested the combination of "Infinispan 8.2.2 + JGroups 3.6.7", we face the same issue,
> data replication across Nodes does not go through.

If this is a supported configuration, then I suggest post this to the 
infinispan-dev list. Oops, this *is* the list :-) Any takers?

Did you print the stack trace?


> Highlighted below are configurations used (derived from the Base configurations):
> Infinispan-config.xml:
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> <infinispan>
>      <jgroups>
>          <stack-file name="configurationFile" path="config/jgroups.xml"/>
>      </jgroups>
>      <cache-container>
>          <transport  cluster="x-cluster" stack="configurationFile" />
>          <replicated-cache name="transactional-type" mode="SYNC">
>              <transaction mode="NON_XA"  locking="OPTIMISTIC" transaction-manager-lookup="org.infinispan.transaction.lookup.GenericTransactionManagerLookup"  auto-commit="true"  />
>                <locking acquire-timeout="60000"/>
>                <expiration lifespan="43200000"/>
> 	<state-transfer enabled="true" timeout="240000" chunk-size="10000" />
> 	<locking isolation="READ_COMMITTED"  acquire-timeout="20000"  write-skew="false"  concurrency-level="5000"  striping="false"  />
>          </replicated-cache>
>      </cache-container>
> </infinispan>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> JGroups.xml
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> <!--
>      TCP based stack, with flow control and message bundling. This is usually used when IP
>      multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
>      Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
>      -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>      author: Bela Ban
> -->
> <config xmlns="urn:org:jgroups"
>          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>          xsi:schemaLocation="urn:org:jgroups  http://www.jgroups.org/schema/jgroups-3.6.xsd">
>
>    <TCP loopback="true"
>         bind_addr="${jgroups.tcp.address:127.0.0.1}"
>         bind_port="${jgroups.tcp.port:7800}"
>         recv_buf_size="${tcp.recv_buf_size:130k}"
>         send_buf_size="${tcp.send_buf_size:130k}"
>         discard_incompatible_packets="true"
>         max_bundle_size="64K"
>         max_bundle_timeout="30"
>         enable_bundling="true"
>         use_send_queues="true"
>         sock_conn_timeout="300"
>         timer_type="new"
>         timer.min_threads="4"
>         timer.max_threads="10"
>         timer.keep_alive_time="3000"
>         timer.queue_max_size="500"
>         thread_pool.enabled="true"
>         thread_pool.min_threads="2"
>         thread_pool.max_threads="30"
>         thread_pool.keep_alive_time="60000"
>         thread_pool.queue_enabled="false"
>         thread_pool.queue_max_size="100"
>         thread_pool.rejection_policy="discard"
>         oob_thread_pool.enabled="true"
>         oob_thread_pool.min_threads="2"
>         oob_thread_pool.max_threads="30"
>         oob_thread_pool.keep_alive_time="60000"
>         oob_thread_pool.queue_enabled="false"
>         oob_thread_pool.queue_max_size="100"
>         oob_thread_pool.rejection_policy="discard"
> 	   />
>
>      <!-- <TCP_NIO -->
>      <!--         bind_port="7800" -->
>      <!--         bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>      <!--         use_send_queues="true" -->
>      <!--         sock_conn_timeout="300" -->
>      <!--         reader_threads="3" -->
>      <!--         writer_threads="3" -->
>      <!--         processor_threads="0" -->
>      <!--         processor_minThreads="0" -->
>      <!--         processor_maxThreads="0" -->
>      <!--         processor_queueSize="100" -->
>      <!--         processor_keepAliveTime="9223372036854775807"/> -->
>      <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>      <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"
>              port_range="2"  timeout="3000" /> -->
>      <MERGE2 max_interval="30000" min_interval="10000"/>
>      <FD_SOCK/>
>      <FD timeout="3000" max_tries="3"/>
>      <VERIFY_SUSPECT timeout="1500"/>
>      <pbcast.NAKACK
>          use_mcast_xmit="false"
>          retransmit_timeout="300,600,1200,2400,4800"
>          discard_delivered_msgs="true"/>
>      <UNICAST2 timeout="300,600,1200"
>                stable_interval="5000"
> 			  log_not_found_msgs="true"
>                max_bytes="400000"/>
>      <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="400000"/>
>      <pbcast.GMS print_local_addr="true" join_timeout="5000" merge_timeout="5000" log_collect_msgs="true" log_view_warnings="true" max_join_attempts="0" view_bundling="true"/>
>      <UFC max_credits="200k" min_threshold="0.20"/>
>      <MFC max_credits="200k" min_threshold="0.20"/>
>      <FRAG2 frag_size="35000"/>
>      <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" throw_exception_on_timeout="true" />
>
> </config>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Would be very helpful if you could kindly provide us some working combination of xml configs with "Infinsipan 8.2.2 + JGroup 3.6.7" or
> let us know what configuration needs to be changed above.
>
> Regs,
> Manohar.
>
> -----Original Message-----
> From: Bela Ban [mailto:bban at redhat.com]
> Sent: Monday, August 22, 2016 1:58 PM
> To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List <infinispan-dev at lists.jboss.org>
> Cc: Sathish Kumar Tippeshappa <sathish.kumar.tippeshappa at ericsson.com>
> Subject: Re: JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":
>
>
>
> On 22/08/16 10:23, Manohar SL wrote:
>> Hi Bela Ban,
>>
>> Thanks for the input points.
>>
>> Answers are highlighted below:
>>
>> 1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
>> 	Actually, we need the Replication Mode of Infinispan, but after Upgrading to Infinispan 8.2.2, the replication mode is not working.
>> 	The data updated in Infinispan is getting written to only the local node and not replicated to other nodes in the Cluster (we have deployed 3 Nodes).
>> 	Since we saw the NPE in JGroups, we suspected this could be the reason for the Replication not functioning as expected, as it also complains of not
>> 	being able to send data to the other nodes in the Cluster.
>
> OK
>
>> 2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
>> 	Our understating based on the JGroups code analysis, in the context of this issue, is:
>> 		=> In the BaseServer.java file, the below method tries to send data to the other Nodes:
>> 			public void send(Address dest, byte[] data, int offset, int length) throws Exception
>> 		      The below logic is not able to retrieve the Connection Object:
>> ----------------------------------------------------------------------
>> --------------------------------------------------
>> // Get a connection (or create one if not yet existent) and send the data
>>           Connection conn=null;
>>           try {
>>               conn=getConnection(dest);
>>               conn.send(data, offset, length);
>>           }
>>           catch(Exception ex) {
>>               removeConnectionIfPresent(dest, conn);
>>               throw ex;
>>           }
>> ------------------------------------------------------------------------------------------------------------------------
>> 		This eventually leads to the NPE.
>
>
> Insert a ex.printStackTrace() statement in the catch clause.
>
>
>> 3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
>> 	Infinispan 8.2.2 by default uses JGroups 3.6.7, but we tried upgrading to JGroups 3.6.8.
>
> Then 3.6.8 may not be supported. I'm sure REPL mode wa tested before releasing 8.2.2, so perhaps 3.6.8 doesn't work with 8.2.2...
>    		
>> 4: Is this reproduceable? If yes, goto step #2
>> 	Yes, this is consistently reproducible.
>>
>> It would be really helpful if you could kindly mail us working JGroups configuration with Infinispan 8.2.2 version.
>
> Well, I suggest take the TCP based config shipped with 8.2.2 and use it as base, ie. make changes to it.
>
>> Also, if you see any issues with our current set of configurations, highlighted in the below mail,  please let us know the needed changes to this.
>>
>> Thanks again for all the help.
>>
>> Regs,
>> Manohar
>>
>>
>> -----Original Message-----
>> From: Bela Ban [mailto:bban at redhat.com]
>> Sent: Monday, August 22, 2016 1:31 PM
>> To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List
>> <infinispan-dev at lists.jboss.org>
>> Cc: Sathish Kumar Tippeshappa <sathish.kumar.tippeshappa at ericsson.com>
>> Subject: Re: JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":
>>
>> 1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
>>
>> 2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
>>
>> 3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
>>
>> 4: Is this reproduceable? If yes, goto step #2
>>
>>
>>
>> On 20/08/16 09:38, Manohar SL wrote:
>>> Hi Bela Ban,
>>>
>>> We have been trying to use JGroups 3.6.8 with Infinispan 8.2.2., in this context we are observing an issue with the Replication Mode of Infinispan usage.
>>> We see the below exception from Jgroups:
>>> JGRP000029: failed sending message to <>:7800 (100 bytes):
>>> java.lang.NullPointerException, headers: TCPGOSSIP:
>>> [type=GET_MBRS_REQ, cluster=x-cluster], TP: [cluster_name=x-cluster]
>>>
>>> The configurations used are highlighted below:
>>>
>>> Infinispan config
>>>
>>> <infinispan>
>>>
>>>        <jgroups>
>>>
>>>            <stack-file name="configurationFile"
>>> path="config/jgroups.xml"/>
>>>
>>>        </jgroups>
>>>
>>>        <cache-container>
>>>
>>>            <transport cluster="x-cluster" stack="configurationFile" />
>>>
>>> <replicated-cache name="transactional-type" mode="SYNC">
>>>
>>>                <transaction mode="NON_XA"  locking="OPTIMISTIC"
>>> transaction-manager-lookup="org.infinispan.transaction.lookup.JBossSt
>>> a ndaloneJTAManagerLookup" auto-commit="true"  />
>>>
>>> <locking acquire-timeout="60000"/>
>>>
>>> <expiration lifespan="43200000"/>
>>>
>>>            </replicated-cache>
>>>
>>>        </cache-container>
>>>
>>> </infinispan>
>>>
>>>
>>>
>>> Jgroups configuration
>>>
>>> <!--
>>>
>>>        TCP based stack, with flow control and message bundling. This
>>> is usually used when IP
>>>
>>>        multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
>>>
>>>        Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
>>>
>>>        -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>>>
>>>        author: Bela Ban
>>>
>>> -->
>>>
>>> <config xmlns="urn:org:jgroups"
>>>
>>>            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>
>>> xsi:schemaLocation="urn:org:jgroups
>>> http://www.jgroups.org/schema/jgroups-3.6.xsd">
>>>
>>>
>>>
>>>      <TCP loopback="true"
>>>
>>> bind_addr="${jgroups.tcp.address:127.0.0.1}"
>>>
>>> bind_port="${jgroups.tcp.port:7800}"
>>>
>>> recv_buf_size="${tcp.recv_buf_size:20M}"
>>>
>>> send_buf_size="${tcp.send_buf_size:640K}"
>>>
>>> discard_incompatible_packets="true"
>>>
>>> max_bundle_size="64K"
>>>
>>> max_bundle_timeout="30"
>>>
>>> enable_bundling="true"
>>>
>>>           use_send_queues="true"
>>>
>>> sock_conn_timeout="300"
>>>
>>> timer_type="new"
>>>
>>> timer.min_threads="4"
>>>
>>> timer.max_threads="10"
>>>
>>> timer.keep_alive_time="3000"
>>>
>>> timer.queue_max_size="500"
>>>
>>> thread_pool.enabled="true"
>>>
>>> thread_pool.min_threads="2"
>>>
>>> thread_pool.max_threads="30"
>>>
>>> thread_pool.keep_alive_time="60000"
>>>
>>> thread_pool.queue_enabled="false"
>>>
>>> thread_pool.queue_max_size="100"
>>>
>>> thread_pool.rejection_policy="discard"
>>>
>>> oob_thread_pool.enabled="true"
>>>
>>>           oob_thread_pool.min_threads="2"
>>>
>>> oob_thread_pool.max_threads="30"
>>>
>>> oob_thread_pool.keep_alive_time="60000"
>>>
>>> oob_thread_pool.queue_enabled="false"
>>>
>>> oob_thread_pool.queue_max_size="100"
>>>
>>> oob_thread_pool.rejection_policy="discard"/>
>>>
>>>
>>>
>>>        <!-- <TCP_NIO -->
>>>
>>> <!-- bind_port="7800" -->
>>>
>>> <!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>>>
>>> <!-- use_send_queues="true" -->
>>>
>>> <!-- sock_conn_timeout="300" -->
>>>
>>> <!--         reader_threads="3" -->
>>>
>>> <!-- writer_threads="3" -->
>>>
>>> <!-- processor_threads="0" -->
>>>
>>> <!-- processor_minThreads="0" -->
>>>
>>> <!-- processor_maxThreads="0" -->
>>>
>>> <!-- processor_queueSize="100" -->
>>>
>>> <!--         processor_keepAliveTime="9223372036854775807"/> -->
>>>
>>>        <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>>>
>>>        <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"
>>>
>>>                port_range="2" timeout="3000" /> -->
>>>
>>>        <MERGE2 max_interval="30000" min_interval="10000"/>
>>>
>>>        <FD_SOCK/>
>>>
>>>        <FD timeout="3000" max_tries="3"/>
>>>
>>>        <VERIFY_SUSPECT timeout="1500"/>
>>>
>>>        <pbcast.NAKACK
>>>
>>> use_mcast_xmit="false"
>>>
>>> retransmit_timeout="300,600,1200,2400,4800"
>>>
>>> discard_delivered_msgs="false"/>
>>>
>>>        <UNICAST2 timeout="300,600,1200"
>>>
>>> stable_interval="5000"
>>>
>>> max_bytes="1m"/>
>>>
>>>        <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000"
>>> max_bytes="1m"/>
>>>
>>>        <pbcast.GMS print_local_addr="false" join_timeout="3000"
>>> view_bundling="true"/>
>>>
>>>        <UFC max_credits="200k" min_threshold="0.20"/>
>>>
>>>        <MFC max_credits="200k" min_threshold="0.20"/>
>>>
>>>        <FRAG2 frag_size="60000"/>
>>>
>>>        <RSVP timeout="60000" resend_interval="500"
>>> ack_on_delivery="false" />
>>>
>>> </config>
>>>
>>> Any help on this would be really great.
>>> Kindly let us know if you would need any further information on this.
>>>
>>> Regs,
>>> Manohar.
>>>
>>
>> --
>> Bela Ban, JGroups lead (http://www.jgroups.org)
>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From afield at redhat.com  Mon Aug 22 14:14:56 2016
From: afield at redhat.com (Alan Field)
Date: Mon, 22 Aug 2016 14:14:56 -0400 (EDT)
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
Message-ID: <250027944.6121458.1471889696295.JavaMail.zimbra@redhat.com>

A little late to the discussion but if all of the pods share the same set of volumes, couldn't you also use the FILE_PING discovery protocol between the pods? 

Thanks, 
Alan 

----- Original Message -----

> From: "Sebastian Laskawiec" <slaskawi at redhat.com>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>, "Bela Ban"
> <bban at redhat.com>, "Kevin Conner" <kconner at redhat.com>, "Rob Cernich"
> <rcernich at redhat.com>
> Sent: Friday, August 19, 2016 5:00:12 AM
> Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets

> Hey!

> I've been playing with Kubernetes PetSets [1] for a while and I'd like to
> share some thoughts. Before I dig in, let me give you some PetSets
> highlights:

> * PetSets are alpha resources for managing stateful apps in Kubernetes 1.3
> (and OpenShift Origin 1.3).
> * Since this is an alpha resource, there are no guarantees about backwards
> compatibility. Alpha resources can also be disabled in some public cloud
> providers (you can control which API versions are accessible [2]).
> * PetSets allows starting pods in sequence (not relevant for us, but this is
> a killer feature for master-slave systems).
> * Each Pod has it's own unique entry in DNS, which makes discovery very
> simple (I'll dig into that a bit later)
> * Volumes are always mounted to the same Pods, which is very important in
> Cache Store scenarios when we restart pods (e.g. Rolling Upgrades [3]).

> Thoughts and ideas after spending some time playing with this feature:

> * PetSets make discovery a lot easier. It's a combination of two things -
> Headless Services [4] which create multiple A records in DNS and predictable
> host names. Each Pod has it's own unique DNS entry following pattern:
> {PetSetName}-{PodIndex}.{ServiceName} [5]. Here's an example of an
> Infinispan PetSet deployed on my local cluster [6]. As you can see we have
> all domain names and IPs from a single DNS query.
> * Maybe we could perform discovery using this mechanism? I'm aware of DNS
> discovery implemented in KUBE_PING [7][8] but the code looks trivial [9] so
> maybe it should be implement inside JGroups? @Bela - WDYT?
> * PetSets do not integrate well with OpenShift 'new-app' command. In other
> words, our users will need to use provided yaml (or json) files to create
> Infinispan cluster. It's not a show-stopper but it's a bit less convenient
> than 'oc new-app'.
> * Since PetSets are alpha resources they need to be considered as secondary
> way to deploy Infinispan on Kubernetes and OpenShift.
> * Finally, the persistent volumes - since a Pod always gets the same volume,
> it would be safe to use any file-based cache store.

> If you'd like to play with PetSets on your local environment, here are
> necessary yaml files [10].

> Thanks
> Sebastian

> [1] http://kubernetes.io/docs/user-guide/petset/
> [2] For checking which APIs are accessible, use 'kubectl api-versions'
> [3]
> http://infinispan.org/docs/stable/user_guide/user_guide.html#_Rolling_chapter
> [4] http://kubernetes.io/docs/user-guide/services/#headless-services
> [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
> [6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
> [7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
> [8] https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
> [9] http://stackoverflow.com/a/12405896/562699
> [10] You might need to adjust ImageStream.
> https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0

> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160822/8c6bb6f5/attachment.html 

From mudokonman at gmail.com  Mon Aug 22 14:15:29 2016
From: mudokonman at gmail.com (William Burns)
Date: Mon, 22 Aug 2016 18:15:29 +0000
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
References: <57B76D47.10907@redhat.com>
	<c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
Message-ID: <CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>

I like the idea of having a variable to set on the listener annotation.
This way we can know for sure if we need to force previous values for some
listeners and not for others.

It seems the default should be to force the previous value to be more
inline with the current behavior, but I fear no one will use the opposite
in this case though.  What do you guys think?

On Mon, Aug 22, 2016 at 4:31 AM Adrian Nistor <anistor at redhat.com> wrote:

> Hi Radim,
>
> Continuous query is built on top of these listeners. CQ _always_ needs
> the previous value and it is very convenient in this case that the
> command is forced to load the previous value. I imagine there may be
> other use cases where we cannot live without the prev value.
>
> I think the listener should be able to state if it needs the prev value
> at registration time. Maybe add a new attribute in the Listener
> annotation? Similar to how we handled Observation.
>
> Adrian
>
> On 08/19/2016 11:34 PM, Radim Vansa wrote:
> > Hi,
> >
> > as I am trying to simplify current entry wrapping and distribution code,
> > I often find that listeners can get wrong previous value in the event,
> > and it sometimes forces the command to load the value even if it is not
> > needed for the command.
> >
> > I am wondering if we should change the previous value in events to
> > Optional - we can usually at least detect that we cannot provide a
> > reliable value (e.g. after retry due to topology change, or because the
> > command did not bothered to load the previous value from cache loader)
> > and return empty Optional.
> >
> > WDYT?
> >
> > Radim
> >
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160822/f2ddb319/attachment.html 

From manohar.sl at ericsson.com  Mon Aug 22 14:22:23 2016
From: manohar.sl at ericsson.com (Manohar SL)
Date: Mon, 22 Aug 2016 18:22:23 +0000
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <57BAE17A.2060202@redhat.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
	<DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB7A0.4070204@redhat.com>
	<DB5PR0701MB2072E8D44B7633509FC775E9EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAE17A.2060202@redhat.com>
Message-ID: <DB5PR0701MB2072A6220EF1A6849BE42482EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>

Yes, we need some quick help from Infinispan Folks on the needed changes to infinispan-config.xml and jgroups.xml to 
get the combination of "Infinispan 8.2.2 with JGroups 3.6.7" working for Replication mode of Infinispan usage.

Both file content are pasted below:
================================================================================================
infinispan-config.xml 

<infinispan>
    <jgroups>
        <stack-file name="configurationFile" path="config/jgroups.xml"/>
    </jgroups>
    <cache-container> 
        <transport  cluster="x-cluster" stack="configurationFile" />
        <replicated-cache name="transactional-type" mode="SYNC"> 
            <transaction mode="NON_XA"  locking="OPTIMISTIC" transaction-manager-lookup="org.infinispan.transaction.lookup.GenericTransactionManagerLookup"  auto-commit="true"  />
            <locking acquire-timeout="60000"/>
            <expiration lifespan="43200000"/>  
			<state-transfer enabled="true" timeout="240000" chunk-size="10000" /> 
			<locking isolation="READ_COMMITTED"  acquire-timeout="20000"  write-skew="false"  concurrency-level="5000"  striping="false"  /> 
        </replicated-cache>
    </cache-container>
</infinispan>

================================================================================================
jgroups.xml

<!--
    TCP based stack, with flow control and message bundling. This is usually used when IP
    multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
    Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
    -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
    author: Bela Ban
-->
<config xmlns="urn:org:jgroups"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:org:jgroups  http://www.jgroups.org/schema/jgroups-3.6.xsd">

  <TCP loopback="true"
       bind_addr="${jgroups.tcp.address:127.0.0.1}"
       bind_port="${jgroups.tcp.port:7800}"
       recv_buf_size="${tcp.recv_buf_size:130k}"
       send_buf_size="${tcp.send_buf_size:130k}"
       discard_incompatible_packets="true"
       max_bundle_size="64K"
       max_bundle_timeout="30"
       enable_bundling="true"
       use_send_queues="true"
       sock_conn_timeout="300"
       timer_type="new"
       timer.min_threads="4"
       timer.max_threads="10"
       timer.keep_alive_time="3000"
       timer.queue_max_size="500"
       thread_pool.enabled="true"
       thread_pool.min_threads="2"
       thread_pool.max_threads="30"
       thread_pool.keep_alive_time="60000"
       thread_pool.queue_enabled="false"
       thread_pool.queue_max_size="100"
       thread_pool.rejection_policy="discard"
       oob_thread_pool.enabled="true"
       oob_thread_pool.min_threads="2"
       oob_thread_pool.max_threads="30"
       oob_thread_pool.keep_alive_time="60000"
       oob_thread_pool.queue_enabled="false"
       oob_thread_pool.queue_max_size="100"
       oob_thread_pool.rejection_policy="discard"
	   />

    <!-- <TCP_NIO -->
    <!--         bind_port="7800" -->
    <!--         bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
    <!--         use_send_queues="true" -->
    <!--         sock_conn_timeout="300" -->
    <!--         reader_threads="3" -->
    <!--         writer_threads="3" -->
    <!--         processor_threads="0" -->
    <!--         processor_minThreads="0" -->
    <!--         processor_maxThreads="0" -->
    <!--         processor_queueSize="100" -->
    <!--         processor_keepAliveTime="9223372036854775807"/> -->
    <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
    <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}" 
            port_range="2"  timeout="3000" /> -->
    <MERGE2 max_interval="30000" min_interval="10000"/>
    <FD_SOCK/>
    <FD timeout="3000" max_tries="3"/>
    <VERIFY_SUSPECT timeout="1500"/>
    <pbcast.NAKACK
        use_mcast_xmit="false"
        retransmit_timeout="300,600,1200,2400,4800"
        discard_delivered_msgs="true"/>
    <UNICAST2 timeout="300,600,1200"
              stable_interval="5000"
			  log_not_found_msgs="true"
              max_bytes="400000"/>
    <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="400000"/>
    <pbcast.GMS print_local_addr="true" join_timeout="5000" merge_timeout="5000" log_collect_msgs="true" log_view_warnings="true" max_join_attempts="0" view_bundling="true"/>
    <UFC max_credits="200k" min_threshold="0.20"/>
    <MFC max_credits="200k" min_threshold="0.20"/>
    <FRAG2 frag_size="35000"/>
    <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" throw_exception_on_timeout="true" />
    
</config>

================================================================================================

@ Bela Ban:
	We could not add the print the stack trace.
	Since we do not build the JGroups code and directly use it from Infinispan library, we could not attempt this.

Regs,
Manohar.

-----Original Message-----
From: infinispan-dev-bounces at lists.jboss.org [mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Bela Ban
Sent: Monday, August 22, 2016 4:57 PM
To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List <infinispan-dev at lists.jboss.org>
Subject: Re: [infinispan-dev] JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":



On 22/08/16 12:56, Manohar SL wrote:
> Hi Bela Ben,
>
> We just now retested the combination of "Infinispan 8.2.2 + JGroups 
> 3.6.7", we face the same issue, data replication across Nodes does not go through.

If this is a supported configuration, then I suggest post this to the infinispan-dev list. Oops, this *is* the list :-) Any takers?

Did you print the stack trace?


> Highlighted below are configurations used (derived from the Base configurations):
> Infinispan-config.xml:
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ------
> <infinispan>
>      <jgroups>
>          <stack-file name="configurationFile" path="config/jgroups.xml"/>
>      </jgroups>
>      <cache-container>
>          <transport  cluster="x-cluster" stack="configurationFile" />
>          <replicated-cache name="transactional-type" mode="SYNC">
>              <transaction mode="NON_XA"  locking="OPTIMISTIC" transaction-manager-lookup="org.infinispan.transaction.lookup.GenericTransactionManagerLookup"  auto-commit="true"  />
>                <locking acquire-timeout="60000"/>
>                <expiration lifespan="43200000"/>
> 	<state-transfer enabled="true" timeout="240000" chunk-size="10000" />
> 	<locking isolation="READ_COMMITTED"  acquire-timeout="20000"  write-skew="false"  concurrency-level="5000"  striping="false"  />
>          </replicated-cache>
>      </cache-container>
> </infinispan>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ------
>
> JGroups.xml
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ------
>
> <!--
>      TCP based stack, with flow control and message bundling. This is usually used when IP
>      multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
>      Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
>      -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>      author: Bela Ban
> -->
> <config xmlns="urn:org:jgroups"
>          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>          xsi:schemaLocation="urn:org:jgroups  
> http://www.jgroups.org/schema/jgroups-3.6.xsd">
>
>    <TCP loopback="true"
>         bind_addr="${jgroups.tcp.address:127.0.0.1}"
>         bind_port="${jgroups.tcp.port:7800}"
>         recv_buf_size="${tcp.recv_buf_size:130k}"
>         send_buf_size="${tcp.send_buf_size:130k}"
>         discard_incompatible_packets="true"
>         max_bundle_size="64K"
>         max_bundle_timeout="30"
>         enable_bundling="true"
>         use_send_queues="true"
>         sock_conn_timeout="300"
>         timer_type="new"
>         timer.min_threads="4"
>         timer.max_threads="10"
>         timer.keep_alive_time="3000"
>         timer.queue_max_size="500"
>         thread_pool.enabled="true"
>         thread_pool.min_threads="2"
>         thread_pool.max_threads="30"
>         thread_pool.keep_alive_time="60000"
>         thread_pool.queue_enabled="false"
>         thread_pool.queue_max_size="100"
>         thread_pool.rejection_policy="discard"
>         oob_thread_pool.enabled="true"
>         oob_thread_pool.min_threads="2"
>         oob_thread_pool.max_threads="30"
>         oob_thread_pool.keep_alive_time="60000"
>         oob_thread_pool.queue_enabled="false"
>         oob_thread_pool.queue_max_size="100"
>         oob_thread_pool.rejection_policy="discard"
> 	   />
>
>      <!-- <TCP_NIO -->
>      <!--         bind_port="7800" -->
>      <!--         bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>      <!--         use_send_queues="true" -->
>      <!--         sock_conn_timeout="300" -->
>      <!--         reader_threads="3" -->
>      <!--         writer_threads="3" -->
>      <!--         processor_threads="0" -->
>      <!--         processor_minThreads="0" -->
>      <!--         processor_maxThreads="0" -->
>      <!--         processor_queueSize="100" -->
>      <!--         processor_keepAliveTime="9223372036854775807"/> -->
>      <TCPGOSSIP initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>      <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"
>              port_range="2"  timeout="3000" /> -->
>      <MERGE2 max_interval="30000" min_interval="10000"/>
>      <FD_SOCK/>
>      <FD timeout="3000" max_tries="3"/>
>      <VERIFY_SUSPECT timeout="1500"/>
>      <pbcast.NAKACK
>          use_mcast_xmit="false"
>          retransmit_timeout="300,600,1200,2400,4800"
>          discard_delivered_msgs="true"/>
>      <UNICAST2 timeout="300,600,1200"
>                stable_interval="5000"
> 			  log_not_found_msgs="true"
>                max_bytes="400000"/>
>      <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="400000"/>
>      <pbcast.GMS print_local_addr="true" join_timeout="5000" merge_timeout="5000" log_collect_msgs="true" log_view_warnings="true" max_join_attempts="0" view_bundling="true"/>
>      <UFC max_credits="200k" min_threshold="0.20"/>
>      <MFC max_credits="200k" min_threshold="0.20"/>
>      <FRAG2 frag_size="35000"/>
>      <RSVP timeout="60000" resend_interval="500" 
> ack_on_delivery="false" throw_exception_on_timeout="true" />
>
> </config>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ------
>
> Would be very helpful if you could kindly provide us some working 
> combination of xml configs with "Infinsipan 8.2.2 + JGroup 3.6.7" or let us know what configuration needs to be changed above.
>
> Regs,
> Manohar.
>
> -----Original Message-----
> From: Bela Ban [mailto:bban at redhat.com]
> Sent: Monday, August 22, 2016 1:58 PM
> To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List 
> <infinispan-dev at lists.jboss.org>
> Cc: Sathish Kumar Tippeshappa <sathish.kumar.tippeshappa at ericsson.com>
> Subject: Re: JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":
>
>
>
> On 22/08/16 10:23, Manohar SL wrote:
>> Hi Bela Ban,
>>
>> Thanks for the input points.
>>
>> Answers are highlighted below:
>>
>> 1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
>> 	Actually, we need the Replication Mode of Infinispan, but after Upgrading to Infinispan 8.2.2, the replication mode is not working.
>> 	The data updated in Infinispan is getting written to only the local node and not replicated to other nodes in the Cluster (we have deployed 3 Nodes).
>> 	Since we saw the NPE in JGroups, we suspected this could be the reason for the Replication not functioning as expected, as it also complains of not
>> 	being able to send data to the other nodes in the Cluster.
>
> OK
>
>> 2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
>> 	Our understating based on the JGroups code analysis, in the context of this issue, is:
>> 		=> In the BaseServer.java file, the below method tries to send data to the other Nodes:
>> 			public void send(Address dest, byte[] data, int offset, int length) throws Exception
>> 		      The below logic is not able to retrieve the Connection Object:
>> ---------------------------------------------------------------------
>> -
>> --------------------------------------------------
>> // Get a connection (or create one if not yet existent) and send the data
>>           Connection conn=null;
>>           try {
>>               conn=getConnection(dest);
>>               conn.send(data, offset, length);
>>           }
>>           catch(Exception ex) {
>>               removeConnectionIfPresent(dest, conn);
>>               throw ex;
>>           }
>> ------------------------------------------------------------------------------------------------------------------------
>> 		This eventually leads to the NPE.
>
>
> Insert a ex.printStackTrace() statement in the catch clause.
>
>
>> 3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
>> 	Infinispan 8.2.2 by default uses JGroups 3.6.7, but we tried upgrading to JGroups 3.6.8.
>
> Then 3.6.8 may not be supported. I'm sure REPL mode wa tested before releasing 8.2.2, so perhaps 3.6.8 doesn't work with 8.2.2...
>    		
>> 4: Is this reproduceable? If yes, goto step #2
>> 	Yes, this is consistently reproducible.
>>
>> It would be really helpful if you could kindly mail us working JGroups configuration with Infinispan 8.2.2 version.
>
> Well, I suggest take the TCP based config shipped with 8.2.2 and use it as base, ie. make changes to it.
>
>> Also, if you see any issues with our current set of configurations, highlighted in the below mail,  please let us know the needed changes to this.
>>
>> Thanks again for all the help.
>>
>> Regs,
>> Manohar
>>
>>
>> -----Original Message-----
>> From: Bela Ban [mailto:bban at redhat.com]
>> Sent: Monday, August 22, 2016 1:31 PM
>> To: Manohar SL <manohar.sl at ericsson.com>; infinispan -Dev List 
>> <infinispan-dev at lists.jboss.org>
>> Cc: Sathish Kumar Tippeshappa 
>> <sathish.kumar.tippeshappa at ericsson.com>
>> Subject: Re: JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":
>>
>> 1: What does the NPE cause? Incorrect behavior? This is in the discovery protocol, so I don't think it is a severe error.
>>
>> 2: If you change the JGroups code to print the exception (or set a breakpoint and debug) we'd know what the cause is.
>>
>> 3: Does Infinispan 8.2.2 use JGroups 3.6.8, or did you upgrade JGroups?
>>
>> 4: Is this reproduceable? If yes, goto step #2
>>
>>
>>
>> On 20/08/16 09:38, Manohar SL wrote:
>>> Hi Bela Ban,
>>>
>>> We have been trying to use JGroups 3.6.8 with Infinispan 8.2.2., in this context we are observing an issue with the Replication Mode of Infinispan usage.
>>> We see the below exception from Jgroups:
>>> JGRP000029: failed sending message to <>:7800 (100 bytes):
>>> java.lang.NullPointerException, headers: TCPGOSSIP:
>>> [type=GET_MBRS_REQ, cluster=x-cluster], TP: [cluster_name=x-cluster]
>>>
>>> The configurations used are highlighted below:
>>>
>>> Infinispan config
>>>
>>> <infinispan>
>>>
>>>        <jgroups>
>>>
>>>            <stack-file name="configurationFile"
>>> path="config/jgroups.xml"/>
>>>
>>>        </jgroups>
>>>
>>>        <cache-container>
>>>
>>>            <transport cluster="x-cluster" stack="configurationFile" 
>>> />
>>>
>>> <replicated-cache name="transactional-type" mode="SYNC">
>>>
>>>                <transaction mode="NON_XA"  locking="OPTIMISTIC"
>>> transaction-manager-lookup="org.infinispan.transaction.lookup.JBossS
>>> t a ndaloneJTAManagerLookup" auto-commit="true"  />
>>>
>>> <locking acquire-timeout="60000"/>
>>>
>>> <expiration lifespan="43200000"/>
>>>
>>>            </replicated-cache>
>>>
>>>        </cache-container>
>>>
>>> </infinispan>
>>>
>>>
>>>
>>> Jgroups configuration
>>>
>>> <!--
>>>
>>>        TCP based stack, with flow control and message bundling. This 
>>> is usually used when IP
>>>
>>>        multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
>>>
>>>        Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
>>>
>>>        -Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]".
>>>
>>>        author: Bela Ban
>>>
>>> -->
>>>
>>> <config xmlns="urn:org:jgroups"
>>>
>>>            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>
>>> xsi:schemaLocation="urn:org:jgroups
>>> http://www.jgroups.org/schema/jgroups-3.6.xsd">
>>>
>>>
>>>
>>>      <TCP loopback="true"
>>>
>>> bind_addr="${jgroups.tcp.address:127.0.0.1}"
>>>
>>> bind_port="${jgroups.tcp.port:7800}"
>>>
>>> recv_buf_size="${tcp.recv_buf_size:20M}"
>>>
>>> send_buf_size="${tcp.send_buf_size:640K}"
>>>
>>> discard_incompatible_packets="true"
>>>
>>> max_bundle_size="64K"
>>>
>>> max_bundle_timeout="30"
>>>
>>> enable_bundling="true"
>>>
>>>           use_send_queues="true"
>>>
>>> sock_conn_timeout="300"
>>>
>>> timer_type="new"
>>>
>>> timer.min_threads="4"
>>>
>>> timer.max_threads="10"
>>>
>>> timer.keep_alive_time="3000"
>>>
>>> timer.queue_max_size="500"
>>>
>>> thread_pool.enabled="true"
>>>
>>> thread_pool.min_threads="2"
>>>
>>> thread_pool.max_threads="30"
>>>
>>> thread_pool.keep_alive_time="60000"
>>>
>>> thread_pool.queue_enabled="false"
>>>
>>> thread_pool.queue_max_size="100"
>>>
>>> thread_pool.rejection_policy="discard"
>>>
>>> oob_thread_pool.enabled="true"
>>>
>>>           oob_thread_pool.min_threads="2"
>>>
>>> oob_thread_pool.max_threads="30"
>>>
>>> oob_thread_pool.keep_alive_time="60000"
>>>
>>> oob_thread_pool.queue_enabled="false"
>>>
>>> oob_thread_pool.queue_max_size="100"
>>>
>>> oob_thread_pool.rejection_policy="discard"/>
>>>
>>>
>>>
>>>        <!-- <TCP_NIO -->
>>>
>>> <!-- bind_port="7800" -->
>>>
>>> <!-- bind_interface="${jgroups.tcp_nio.bind_interface:bond0}" -->
>>>
>>> <!-- use_send_queues="true" -->
>>>
>>> <!-- sock_conn_timeout="300" -->
>>>
>>> <!--         reader_threads="3" -->
>>>
>>> <!-- writer_threads="3" -->
>>>
>>> <!-- processor_threads="0" -->
>>>
>>> <!-- processor_minThreads="0" -->
>>>
>>> <!-- processor_maxThreads="0" -->
>>>
>>> <!-- processor_queueSize="100" -->
>>>
>>> <!--         processor_keepAliveTime="9223372036854775807"/> -->
>>>
>>>        <TCPGOSSIP 
>>> initial_hosts="${jgroups.tcpgossip.initial_hosts}"/>
>>>
>>>        <!--    <TCPPING async_discovery="true" initial_hosts="${jgroups.tcpping.initial_hosts}"
>>>
>>>                port_range="2" timeout="3000" /> -->
>>>
>>>        <MERGE2 max_interval="30000" min_interval="10000"/>
>>>
>>>        <FD_SOCK/>
>>>
>>>        <FD timeout="3000" max_tries="3"/>
>>>
>>>        <VERIFY_SUSPECT timeout="1500"/>
>>>
>>>        <pbcast.NAKACK
>>>
>>> use_mcast_xmit="false"
>>>
>>> retransmit_timeout="300,600,1200,2400,4800"
>>>
>>> discard_delivered_msgs="false"/>
>>>
>>>        <UNICAST2 timeout="300,600,1200"
>>>
>>> stable_interval="5000"
>>>
>>> max_bytes="1m"/>
>>>
>>>        <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000"
>>> max_bytes="1m"/>
>>>
>>>        <pbcast.GMS print_local_addr="false" join_timeout="3000"
>>> view_bundling="true"/>
>>>
>>>        <UFC max_credits="200k" min_threshold="0.20"/>
>>>
>>>        <MFC max_credits="200k" min_threshold="0.20"/>
>>>
>>>        <FRAG2 frag_size="60000"/>
>>>
>>>        <RSVP timeout="60000" resend_interval="500"
>>> ack_on_delivery="false" />
>>>
>>> </config>
>>>
>>> Any help on this would be really great.
>>> Kindly let us know if you would need any further information on this.
>>>
>>> Regs,
>>> Manohar.
>>>
>>
>> --
>> Bela Ban, JGroups lead (http://www.jgroups.org)
>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>

--
Bela Ban, JGroups lead (http://www.jgroups.org)

_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev


From bban at redhat.com  Tue Aug 23 04:12:38 2016
From: bban at redhat.com (Bela Ban)
Date: Tue, 23 Aug 2016 10:12:38 +0200
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <DB5PR0701MB2072A6220EF1A6849BE42482EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
	<DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB7A0.4070204@redhat.com>
	<DB5PR0701MB2072E8D44B7633509FC775E9EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAE17A.2060202@redhat.com>
	<DB5PR0701MB2072A6220EF1A6849BE42482EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
Message-ID: <57BC0576.6090204@redhat.com>



On 22/08/16 20:22, Manohar SL wrote:
> Yes, we need some quick help from Infinispan Folks on the needed changes to infinispan-config.xml and jgroups.xml to
> get the combination of "Infinispan 8.2.2 with JGroups 3.6.7" working for Replication mode of Infinispan usage.

> @ Bela Ban:
> 	We could not add the print the stack trace.
> 	Since we do not build the JGroups code and directly use it from Infinispan library, we could not attempt this.


Why can't you do this?
- Checkout JGroups src code
- git checkout JGroups-3.6.8.Final
- Make the modification: ex.printStackTrace()
- ant clean jar
- Copy the result JAR file in ./dist to the maven dir


> Regs,
> Manohar.

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From manohar.sl at ericsson.com  Tue Aug 23 05:54:03 2016
From: manohar.sl at ericsson.com (Manohar SL)
Date: Tue, 23 Aug 2016 09:54:03 +0000
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <57BC0576.6090204@redhat.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
	<DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB7A0.4070204@redhat.com>
	<DB5PR0701MB2072E8D44B7633509FC775E9EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAE17A.2060202@redhat.com>
	<DB5PR0701MB2072A6220EF1A6849BE42482EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BC0576.6090204@redhat.com>
Message-ID: <DB5PR0701MB2072BF149869BDE71FADDB4FEEEB0@DB5PR0701MB2072.eurprd07.prod.outlook.com>

Sure Bela Ban, we will try this out and share the results.

Also, request any of the Infinispan Folks to help us to get the replication (mode: SYNC) working with Infinispan 8.2.2.
Any help on this would be really great.

Regs,
Manohar.

-----Original Message-----
From: infinispan-dev-bounces at lists.jboss.org [mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Bela Ban
Sent: Tuesday, August 23, 2016 1:43 PM
To: infinispan-dev at lists.jboss.org
Subject: Re: [infinispan-dev] JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":



On 22/08/16 20:22, Manohar SL wrote:
> Yes, we need some quick help from Infinispan Folks on the needed 
> changes to infinispan-config.xml and jgroups.xml to get the combination of "Infinispan 8.2.2 with JGroups 3.6.7" working for Replication mode of Infinispan usage.

> @ Bela Ban:
> 	We could not add the print the stack trace.
> 	Since we do not build the JGroups code and directly use it from Infinispan library, we could not attempt this.


Why can't you do this?
- Checkout JGroups src code
- git checkout JGroups-3.6.8.Final
- Make the modification: ex.printStackTrace()
- ant clean jar
- Copy the result JAR file in ./dist to the maven dir


> Regs,
> Manohar.

--
Bela Ban, JGroups lead (http://www.jgroups.org)

_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev


From bban at redhat.com  Tue Aug 23 07:23:41 2016
From: bban at redhat.com (Bela Ban)
Date: Tue, 23 Aug 2016 13:23:41 +0200
Subject: [infinispan-dev] JGroups 4.0.0.Alpha1
Message-ID: <57BC323D.5000400@redhat.com>

Most API changes have been done, 12 issues left. I expect the final 
release in October.

Note that JGroups-4.0.0-SNAPSHOT is also generated (on every change, IIRC).

Feedback appreciated!

[1] http://belaban.blogspot.ch/2016/08/jgroups-400alpha1-released.html

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From paul.ferraro at redhat.com  Tue Aug 23 10:01:25 2016
From: paul.ferraro at redhat.com (Paul Ferraro)
Date: Tue, 23 Aug 2016 10:01:25 -0400
Subject: [infinispan-dev] JGroups 4.0.0.Alpha1
In-Reply-To: <57BC323D.5000400@redhat.com>
References: <57BC323D.5000400@redhat.com>
Message-ID: <CAF5qqoNmbOkYHqV5MruBznpLtLwDaYA4QNO6ThBhs_-S4226rg@mail.gmail.com>

Bela,

Are you sure you want to drop the Channel interface [1] without any
deprecation?  WildFly references the Channel interface in public user
API [1].  I can certainly narrow the type for WF11, but this will
break user applications unexpectedly.

Paul

[1] https://issues.jboss.org/browse/JGRP-2018
[2] https://github.com/wildfly/wildfly/blob/master/clustering/jgroups/api/src/main/java/org/wildfly/clustering/jgroups/ChannelFactory.java

On Tue, Aug 23, 2016 at 7:23 AM, Bela Ban <bban at redhat.com> wrote:
> Most API changes have been done, 12 issues left. I expect the final release
> in October.
>
> Note that JGroups-4.0.0-SNAPSHOT is also generated (on every change, IIRC).
>
> Feedback appreciated!
>
> [1] http://belaban.blogspot.ch/2016/08/jgroups-400alpha1-released.html
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>

From bban at redhat.com  Tue Aug 23 11:08:55 2016
From: bban at redhat.com (Bela Ban)
Date: Tue, 23 Aug 2016 17:08:55 +0200
Subject: [infinispan-dev] JGroups 4.0.0.Alpha1
In-Reply-To: <CAF5qqoNmbOkYHqV5MruBznpLtLwDaYA4QNO6ThBhs_-S4226rg@mail.gmail.com>
References: <57BC323D.5000400@redhat.com>
	<CAF5qqoNmbOkYHqV5MruBznpLtLwDaYA4QNO6ThBhs_-S4226rg@mail.gmail.com>
Message-ID: <57BC6707.3010300@redhat.com>

Oh shoot, didn't know Channel was referenced directly!

I assume you're upgrading JGroups only on a new major version of 
Wildfly, aren't you? Then, returning a JChannel instead of a Channel 
shouldn't be an issue, should it?

On 23/08/16 16:01, Paul Ferraro wrote:
> Bela,
>
> Are you sure you want to drop the Channel interface [1] without any
> deprecation?  WildFly references the Channel interface in public user
> API [1].  I can certainly narrow the type for WF11, but this will
> break user applications unexpectedly.
>
> Paul
>
> [1] https://issues.jboss.org/browse/JGRP-2018
> [2] https://github.com/wildfly/wildfly/blob/master/clustering/jgroups/api/src/main/java/org/wildfly/clustering/jgroups/ChannelFactory.java
>
> On Tue, Aug 23, 2016 at 7:23 AM, Bela Ban <bban at redhat.com> wrote:
>> Most API changes have been done, 12 issues left. I expect the final release
>> in October.
>>
>> Note that JGroups-4.0.0-SNAPSHOT is also generated (on every change, IIRC).
>>
>> Feedback appreciated!
>>
>> [1] http://belaban.blogspot.ch/2016/08/jgroups-400alpha1-released.html
>>
>> --
>> Bela Ban, JGroups lead (http://www.jgroups.org)
>>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


From paul.ferraro at redhat.com  Tue Aug 23 12:33:39 2016
From: paul.ferraro at redhat.com (Paul Ferraro)
Date: Tue, 23 Aug 2016 12:33:39 -0400
Subject: [infinispan-dev] JGroups 4.0.0.Alpha1
In-Reply-To: <57BC6707.3010300@redhat.com>
References: <57BC323D.5000400@redhat.com>
	<CAF5qqoNmbOkYHqV5MruBznpLtLwDaYA4QNO6ThBhs_-S4226rg@mail.gmail.com>
	<57BC6707.3010300@redhat.com>
Message-ID: <CAF5qqoMYx0c4314FSqsMV4hXehqU67OJ9DECxyDX7dB08dixbg@mail.gmail.com>

The problem isn't modifying the WF API to return a narrower type. We
can certainly do that. The problem is that existing applications
expect a org.wildfly.clustering.jgroups.ChannelFactory to return an
org.jgroups.Channel.  When they upgrade to WF11, they should get a
deprecation warning, not a NoClassDefFoundError. Eventually, we can
break their application, as they would have been sufficiently warned
already.  I'm sure there are JGroups users that will be in the same
boat.

On Tue, Aug 23, 2016 at 11:08 AM, Bela Ban <bban at redhat.com> wrote:
> Oh shoot, didn't know Channel was referenced directly!
>
> I assume you're upgrading JGroups only on a new major version of Wildfly,
> aren't you? Then, returning a JChannel instead of a Channel shouldn't be an
> issue, should it?
>
>
> On 23/08/16 16:01, Paul Ferraro wrote:
>>
>> Bela,
>>
>> Are you sure you want to drop the Channel interface [1] without any
>> deprecation?  WildFly references the Channel interface in public user
>> API [1].  I can certainly narrow the type for WF11, but this will
>> break user applications unexpectedly.
>>
>> Paul
>>
>> [1] https://issues.jboss.org/browse/JGRP-2018
>> [2]
>> https://github.com/wildfly/wildfly/blob/master/clustering/jgroups/api/src/main/java/org/wildfly/clustering/jgroups/ChannelFactory.java
>>
>> On Tue, Aug 23, 2016 at 7:23 AM, Bela Ban <bban at redhat.com> wrote:
>>>
>>> Most API changes have been done, 12 issues left. I expect the final
>>> release
>>> in October.
>>>
>>> Note that JGroups-4.0.0-SNAPSHOT is also generated (on every change,
>>> IIRC).
>>>
>>> Feedback appreciated!
>>>
>>> [1] http://belaban.blogspot.ch/2016/08/jgroups-400alpha1-released.html
>>>
>>> --
>>> Bela Ban, JGroups lead (http://www.jgroups.org)
>>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>

From slaskawi at redhat.com  Wed Aug 24 02:55:58 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 24 Aug 2016 08:55:58 +0200
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <DB5PR0701MB2072BF149869BDE71FADDB4FEEEB0@DB5PR0701MB2072.eurprd07.prod.outlook.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
	<DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB7A0.4070204@redhat.com>
	<DB5PR0701MB2072E8D44B7633509FC775E9EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAE17A.2060202@redhat.com>
	<DB5PR0701MB2072A6220EF1A6849BE42482EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BC0576.6090204@redhat.com>
	<DB5PR0701MB2072BF149869BDE71FADDB4FEEEB0@DB5PR0701MB2072.eurprd07.prod.outlook.com>
Message-ID: <CAA9R9O5OEYDFveNvOrNWoyqMc5f53Wk0H9SKVsQPMS1JAO0RcA@mail.gmail.com>

Hey Manohar!

I sent an answer to this thread: https://developer.jboss.org/thread/271933

Thanks
Sebastian

On Tue, Aug 23, 2016 at 11:54 AM, Manohar SL <manohar.sl at ericsson.com>
wrote:

> Sure Bela Ban, we will try this out and share the results.
>
> Also, request any of the Infinispan Folks to help us to get the
> replication (mode: SYNC) working with Infinispan 8.2.2.
> Any help on this would be really great.
>
> Regs,
> Manohar.
>
> -----Original Message-----
> From: infinispan-dev-bounces at lists.jboss.org [mailto:infinispan-dev-
> bounces at lists.jboss.org] On Behalf Of Bela Ban
> Sent: Tuesday, August 23, 2016 1:43 PM
> To: infinispan-dev at lists.jboss.org
> Subject: Re: [infinispan-dev] JGroups Error: JGRP000029 observed with
> "JGroups 3.6.8 + Infinispan 8.2.2":
>
>
>
> On 22/08/16 20:22, Manohar SL wrote:
> > Yes, we need some quick help from Infinispan Folks on the needed
> > changes to infinispan-config.xml and jgroups.xml to get the combination
> of "Infinispan 8.2.2 with JGroups 3.6.7" working for Replication mode of
> Infinispan usage.
>
> > @ Bela Ban:
> >       We could not add the print the stack trace.
> >       Since we do not build the JGroups code and directly use it from
> Infinispan library, we could not attempt this.
>
>
> Why can't you do this?
> - Checkout JGroups src code
> - git checkout JGroups-3.6.8.Final
> - Make the modification: ex.printStackTrace()
> - ant clean jar
> - Copy the result JAR file in ./dist to the maven dir
>
>
> > Regs,
> > Manohar.
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160824/958f03a6/attachment-0001.html 

From slaskawi at redhat.com  Wed Aug 24 03:06:39 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 24 Aug 2016 09:06:39 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <2039814238.4486980.1471874577579.JavaMail.zimbra@redhat.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<57B6CEDB.1090700@redhat.com>
	<CAA9R9O7cDnsx3PBnhxZ58ZQp0x6TOp+Orq2TUFsrPyyG5ZYWzQ@mail.gmail.com>
	<447733132.4160093.1471644286616.JavaMail.zimbra@redhat.com>
	<CAA9R9O4jdfc30aVb+fRLPRkGkX3Rd-U2W9wFnxQEPWnNcNjU5w@mail.gmail.com>
	<2039814238.4486980.1471874577579.JavaMail.zimbra@redhat.com>
Message-ID: <CAA9R9O6GG8QW0v_GJ44KGdNW7vBPcm_P2OU-V5eX5kuLCVniGg@mail.gmail.com>

Thanks Rob! That clarifies a lot!

On Mon, Aug 22, 2016 at 4:02 PM, Rob Cernich <rcernich at redhat.com> wrote:

>
>
> ------------------------------
>
> Hey Rob!
>
> Thanks a lot for clarification!
>
> More comments inlined.
>
> Thanks
> Sebastian
>
> On Sat, Aug 20, 2016 at 12:04 AM, Rob Cernich <rcernich at redhat.com> wrote:
>
>> A couple of things...
>>
>> re. volumes:
>> We also need to consider the mounting behavior for scale down scenarios
>> and for overage scenarios when doing upgrades.  For the latter, OpenShift
>> can spin up pods of the new version before the older version pods have
>> terminated.  This may mean that some volumes from the old pods are
>> orphaned.  We did see this when testing A-MQ during upgrades.  With a
>> single pod, the upgrade process caused the new version to have a new mount
>> and the original mount was left orphaned (another upgrade would cause the
>> newer pod to pick up the orphaned mount, leaving the new mount orphaned).
>> I believe we worked around this by specifying an overage of 0% during
>> upgrades.  This ensured the new pods would pick up the volumes left behind
>> by the old pods.  (Actually, we were using subdirectories in the mount,
>> since all pods shared the same volume.)
>>
>>
> I think PetSets try to address this kind of problems. According to the
> manual page [11], the storage is linked to the Pod ordinal and hostaname
> and should be stable.
>
> [11] http://kubernetes.io/docs/user-guide/petset/#when-to-use-pet-set
>
>>
>> re. dns:
>> DNS should work fine as-is, but there are a couple things that you need
>> to consider.
>> 1. Service endpoints are only available in DNS after the pod becomes
>> ready (SVC records on the service name).  Because infinispan attaches
>> itself to the cluster, this meant pods were all started as cluster of one,
>> then merged once they noticed the other pods.  This had a significant
>> impact on startup.  Since then, OpenShift has added the ability to query
>> the endpoints associated with a service as soon as the pod is created,
>> which would allow initialization to work correctly.  To make this work,
>> we'd have to change the form of the DNS query to pick up the service
>> endpoints (I forget the naming scheme).
>>
>
> Yes, I agree. Adding nodes one after another will have significant impact
> on cluster startup time. However it should be safe to query the cluster
> (and even put data) during rebalance. So I would say, if a node is up, and
> cluster is not damaged - we should treat it as ready.
>
> To be clear, the issue was around the lifecycle and interaction with the
> readiness probe.  OpenShift/Kubernetes only add the pod to the service once
> it's "ready."  Our readiness probe defines ready as startup is complete
> (i.e. x of y services started).  The issue with this is that as pods come
> up, they only see other pods that are ready when they initialize, so if you
> start with an initial cluster size of five, the new nodes don't see any of
> the other nodes until they finish startup and refresh the list of nodes,
> after which they all have to merge with each other.  This had a significant
> impact on performance.  Since then, a feature has been added which allows
> you to query DNS for all endpoints regardless of ready state.  This allows
> nodes to see the other nodes before they become ready, which allows for a
> more natural cluster formation.  To reiterate, the impact on startup was
> significant, especially when scaling up under load.
>
>
> NB - I proposed a HealthCheck API to Infinispan 9 (currently under
> development) [12][13]. The overall cluster health can be in one of 3
> statuses - GREEN (everything is fine), YELLOW (rebalance in progress), RED
> (cluster not healthy). Kubernetes/OpenShift readiness probe should check if
> the status is GREEN or YELLOW. The HealthCheck API is attached to the WF
> management API so you can query it with CURL or using ispn_cli.sh script.
>
> [12] https://github.com/infinispan/infinispan/wiki/Health-check-API
> [13] https://github.com/infinispan/infinispan/pull/4499
>
>>
>> Another thing to keep in mind is that looking up pods by labels allows
>> any pod with the specified label to be added to the cluster.  I'm not sure
>> of a use case for this, but it would allow other deployments to be included
>> in the cluster.  (You could also argue that the service is the authority
>> for this and any pod with said label would be added as a service endpoint,
>> thus achieving the same behavior...probably more simply too.)
>>
>
> I think this is a scenario when someone might try to attach Infinispan in
> library mode (a dependency in WAR file for example) to the Hot Rod cluster.
> Gustavo answered question like this a while ago [14].
>
> [14] https://developer.jboss.org/message/961568
>
>> Lastly, DNS was a little flaky when we first implemented this, which was
>> part of the reason we went straight to kubernetes.  Users were using
>> dnsmasq with wildcards that worked well for routes, but ended up routing
>> services to the router ip instead of pod ip.  Needless to say, there were a
>> lot of complications trying to use DNS and debug user problems with service
>> resolution.
>>
>
> I think a governing headless service [15] is required here (PetSets
> require a service but considering how Infinispan works, it should be a
> headless service in my opinion).
>
> [15] http://kubernetes.io/docs/user-guide/services/#headless-services
>
>>
>>
>> Hope that helps,
>> Rob
>>
>> ------------------------------
>>
>> Hey Bela!
>>
>> No no, the resolution can be done with pure JDK.
>>
>> Thanks
>> Sebastian
>>
>> On Fri, Aug 19, 2016 at 11:18 AM, Bela Ban <bban at redhat.com> wrote:
>>
>>> Hi Sebastian
>>>
>>> the usual restrictions apply: if DNS discovery depends on external libs,
>>> then it should be hosted in jgroups-extras, otherwise we can add it to
>>> JGroups itself.
>>>
>>> On 19/08/16 11:00, Sebastian Laskawiec wrote:
>>>
>>>> Hey!
>>>>
>>>> I've been playing with Kubernetes PetSets [1] for a while and I'd like
>>>> to share some thoughts. Before I dig in, let me give you some PetSets
>>>> highlights:
>>>>
>>>>   * PetSets are alpha resources for managing stateful apps in Kubernetes
>>>>     1.3 (and OpenShift Origin 1.3).
>>>>   * Since this is an alpha resource, there are no guarantees about
>>>>     backwards compatibility. Alpha resources can also be disabled in
>>>>     some public cloud providers (you can control which API versions are
>>>>     accessible [2]).
>>>>   * PetSets allows starting pods in sequence (not relevant for us, but
>>>>     this is a killer feature for master-slave systems).
>>>>   * Each Pod has it's own unique entry in DNS, which makes discovery
>>>>     very simple (I'll dig into that a bit later)
>>>>   * Volumes are always mounted to the same Pods, which is very important
>>>>     in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades
>>>>     [3]).
>>>>
>>>> Thoughts and ideas after spending some time playing with this feature:
>>>>
>>>>   * PetSets make discovery a lot easier. It's a combination of two
>>>>     things - Headless Services [4] which create multiple A records in
>>>>     DNS and predictable host names. Each Pod has it's own unique DNS
>>>>     entry following pattern: {PetSetName}-{PodIndex}.{ServiceName} [5].
>>>>     Here's an example of an Infinispan PetSet deployed on my local
>>>>     cluster [6]. As you can see we have all domain names and IPs from a
>>>>     single DNS query.
>>>>   * Maybe we could perform discovery using this mechanism? I'm aware of
>>>>     DNS discovery implemented in KUBE_PING [7][8] but the code looks
>>>>     trivial [9] so maybe it should be implement inside JGroups? @Bela -
>>>>     WDYT?
>>>>   * PetSets do not integrate well with OpenShift 'new-app' command. In
>>>>     other words, our users will need to use provided yaml (or json)
>>>>     files to create Infinispan cluster. It's not a show-stopper but it's
>>>>     a bit less convenient than 'oc new-app'.
>>>>   * Since PetSets are alpha resources they need to be considered as
>>>>     secondary way to deploy Infinispan on Kubernetes and OpenShift.
>>>>   * Finally, the persistent volumes - since a Pod always gets the same
>>>>     volume, it would be safe to use any file-based cache store.
>>>>
>>>> If you'd like to play with PetSets on your local environment, here are
>>>> necessary yaml files [10].
>>>>
>>>> Thanks
>>>> Sebastian
>>>>
>>>>
>>>> [1] http://kubernetes.io/docs/user-guide/petset/
>>>> [2] For checking which APIs are accessible, use 'kubectl api-versions'
>>>> [3]
>>>> http://infinispan.org/docs/stable/user_guide/user_guide.
>>>> html#_Rolling_chapter
>>>> [4] http://kubernetes.io/docs/user-guide/services/#headless-services
>>>> [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
>>>> [6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
>>>> [7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
>>>> [8] https://github.com/jgroups-extras/jgroups-kubernetes/
>>>> tree/master/dns
>>>> [9] http://stackoverflow.com/a/12405896/562699
>>>> [10] You might need to adjust ImageStream.
>>>> https://gist.github.com/slaskawi/7cffb5588dabb770f654557579c5f2d0
>>>>
>>>
>>> --
>>> Bela Ban, JGroups lead (http://www.jgroups.org)
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160824/9ae5248c/attachment.html 

From slaskawi at redhat.com  Wed Aug 24 03:07:35 2016
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 24 Aug 2016 09:07:35 +0200
Subject: [infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
In-Reply-To: <250027944.6121458.1471889696295.JavaMail.zimbra@redhat.com>
References: <CAA9R9O70Gn30AG8_a1i4avgR0sUvtpAxv86qTGk52mTZ92Se+Q@mail.gmail.com>
	<250027944.6121458.1471889696295.JavaMail.zimbra@redhat.com>
Message-ID: <CAA9R9O7VQDx_NGgpc6f0qwah=PtGRt5TDyC-gWY1OQBc2rnMqg@mail.gmail.com>

Hey Alan!

Yeah, that could work :) I'll be doing some experiments with volumes
shortly so I'll definitely check that!

Thanks
Sebastian

On Mon, Aug 22, 2016 at 8:14 PM, Alan Field <afield at redhat.com> wrote:

> A little late to the discussion but if all of the pods share the same set
> of volumes, couldn't you also use the FILE_PING discovery protocol between
> the pods?
>
> Thanks,
> Alan
>
> ------------------------------
>
> *From: *"Sebastian Laskawiec" <slaskawi at redhat.com>
> *To: *"infinispan -Dev List" <infinispan-dev at lists.jboss.org>, "Bela Ban"
> <bban at redhat.com>, "Kevin Conner" <kconner at redhat.com>, "Rob Cernich" <
> rcernich at redhat.com>
> *Sent: *Friday, August 19, 2016 5:00:12 AM
> *Subject: *[infinispan-dev] Infinispan and OpenShift/Kubernetes PetSets
>
>
> Hey!
>
> I've been playing with Kubernetes PetSets [1] for a while and I'd like to
> share some thoughts. Before I dig in, let me give you some PetSets
> highlights:
>
>    - PetSets are alpha resources for managing stateful apps in Kubernetes
>    1.3 (and OpenShift Origin 1.3).
>    - Since this is an alpha resource, there are no guarantees about
>    backwards compatibility. Alpha resources can also be disabled in some
>    public cloud providers (you can control which API versions are accessible
>    [2]).
>    - PetSets allows starting pods in sequence (not relevant for us, but
>    this is a killer feature for master-slave systems).
>    - Each Pod has it's own unique entry in DNS, which makes discovery
>    very simple (I'll dig into that a bit later)
>    - Volumes are always mounted to the same Pods, which is very important
>    in Cache Store scenarios when we restart pods (e.g. Rolling Upgrades [3]).
>
> Thoughts and ideas after spending some time playing with this feature:
>
>    - PetSets make discovery a lot easier. It's a combination of two
>    things - Headless Services [4] which create multiple A records in DNS and
>    predictable host names. Each Pod has it's own unique DNS entry following
>    pattern: {PetSetName}-{PodIndex}.{ServiceName} [5]. Here's an example
>    of an Infinispan PetSet deployed on my local cluster [6]. As you can see we
>    have all domain names and IPs from a single DNS query.
>    - Maybe we could perform discovery using this mechanism? I'm aware of
>    DNS discovery implemented in KUBE_PING [7][8] but the code looks trivial
>    [9] so maybe it should be implement inside JGroups? @Bela - WDYT?
>    - PetSets do not integrate well with OpenShift 'new-app' command. In
>    other words, our users will need to use provided yaml (or json) files to
>    create Infinispan cluster. It's not a show-stopper but it's a bit less
>    convenient than 'oc new-app'.
>    - Since PetSets are alpha resources they need to be considered as
>    secondary way to deploy Infinispan on Kubernetes and OpenShift.
>    - Finally, the persistent volumes - since a Pod always gets the same
>    volume, it would be safe to use any file-based cache store.
>
> If you'd like to play with PetSets on your local environment, here are
> necessary yaml files [10].
>
> Thanks
> Sebastian
>
>
> [1] http://kubernetes.io/docs/user-guide/petset/
> [2] For checking which APIs are accessible, use 'kubectl api-versions'
> [3] http://infinispan.org/docs/stable/user_guide/user_
> guide.html#_Rolling_chapter
> [4] http://kubernetes.io/docs/user-guide/services/#headless-services
> [5] http://kubernetes.io/docs/user-guide/petset/#peer-discovery
> [6] https://gist.github.com/slaskawi/0866e63a39276f8ab66376229716a676
> [7] https://github.com/jboss-openshift/openshift-ping/tree/master/dns
> [8] https://github.com/jgroups-extras/jgroups-kubernetes/tree/master/dns
> [9] http://stackoverflow.com/a/12405896/562699
> [10] You might need to adjust ImageStream. https://gist.github.com/
> slaskawi/7cffb5588dabb770f654557579c5f2d0
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160824/5585ede7/attachment-0001.html 

From pedro at infinispan.org  Wed Aug 24 05:54:49 2016
From: pedro at infinispan.org (Pedro Ruivo)
Date: Wed, 24 Aug 2016 10:54:49 +0100
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
References: <57B76D47.10907@redhat.com>
	<c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
	<CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
Message-ID: <4afe8180-7593-3c95-5ce7-0ae24b9e3ded@infinispan.org>

If we can drop the pre-events, it would be possible to skip the 
loading/wrapping of the previous value. The method DataContainer.put() 
could return the previous value/metadata that could be used to trigger 
the post-event.

However, it has a problem that I haven't solved it yet: if the value is 
not in memory it will not be loaded (cache-stores / rebalance in progress).

Would it work for CQ?

Cheers,
Pedro

On 22-08-2016 19:15, William Burns wrote:
> I like the idea of having a variable to set on the listener annotation.
> This way we can know for sure if we need to force previous values for
> some listeners and not for others.
>
> It seems the default should be to force the previous value to be more
> inline with the current behavior, but I fear no one will use the
> opposite in this case though.  What do you guys think?
>
> On Mon, Aug 22, 2016 at 4:31 AM Adrian Nistor <anistor at redhat.com
> <mailto:anistor at redhat.com>> wrote:
>
>     Hi Radim,
>
>     Continuous query is built on top of these listeners. CQ _always_ needs
>     the previous value and it is very convenient in this case that the
>     command is forced to load the previous value. I imagine there may be
>     other use cases where we cannot live without the prev value.
>
>     I think the listener should be able to state if it needs the prev value
>     at registration time. Maybe add a new attribute in the Listener
>     annotation? Similar to how we handled Observation.
>
>     Adrian
>
>     On 08/19/2016 11:34 PM, Radim Vansa wrote:
>     > Hi,
>     >
>     > as I am trying to simplify current entry wrapping and distribution
>     code,
>     > I often find that listeners can get wrong previous value in the event,
>     > and it sometimes forces the command to load the value even if it
>     is not
>     > needed for the command.
>     >
>     > I am wondering if we should change the previous value in events to
>     > Optional - we can usually at least detect that we cannot provide a
>     > reliable value (e.g. after retry due to topology change, or
>     because the
>     > command did not bothered to load the previous value from cache loader)
>     > and return empty Optional.
>     >
>     > WDYT?
>     >
>     > Radim
>     >
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

From manohar.sl at ericsson.com  Wed Aug 24 06:01:58 2016
From: manohar.sl at ericsson.com (Manohar SL)
Date: Wed, 24 Aug 2016 10:01:58 +0000
Subject: [infinispan-dev] JGroups Error: JGRP000029 observed with
 "JGroups 3.6.8 + Infinispan 8.2.2":
In-Reply-To: <CAA9R9O5OEYDFveNvOrNWoyqMc5f53Wk0H9SKVsQPMS1JAO0RcA@mail.gmail.com>
References: <DB5PR0701MB20720B8143874BBDCA6B17FCEE170@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB154.9000601@redhat.com>
	<DB5PR0701MB20723870DA2CA28D4A2F8D50EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAB7A0.4070204@redhat.com>
	<DB5PR0701MB2072E8D44B7633509FC775E9EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BAE17A.2060202@redhat.com>
	<DB5PR0701MB2072A6220EF1A6849BE42482EEE80@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<57BC0576.6090204@redhat.com>
	<DB5PR0701MB2072BF149869BDE71FADDB4FEEEB0@DB5PR0701MB2072.eurprd07.prod.outlook.com>
	<CAA9R9O5OEYDFveNvOrNWoyqMc5f53Wk0H9SKVsQPMS1JAO0RcA@mail.gmail.com>
Message-ID: <DB5PR0701MB20722631B3411F6DAC70A3D8EEEA0@DB5PR0701MB2072.eurprd07.prod.outlook.com>

Hi Sebastian,

Thanks a lot for the reply and the help.
We finally managed to get the Replication (mode: SYNC) working with Infinispan 8.2.2 with the attached Infinispan config file ?

Thanks a lot for all the help.
Also, thanks to Bela Ban for all the help related to JGroups queries ?

Regs,
Manohar.

From: infinispan-dev-bounces at lists.jboss.org [mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Sebastian Laskawiec
Sent: Wednesday, August 24, 2016 12:26 PM
To: infinispan -Dev List <infinispan-dev at lists.jboss.org>
Subject: Re: [infinispan-dev] JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":

Hey Manohar!

I sent an answer to this thread: https://developer.jboss.org/thread/271933

Thanks
Sebastian

On Tue, Aug 23, 2016 at 11:54 AM, Manohar SL <manohar.sl at ericsson.com<mailto:manohar.sl at ericsson.com>> wrote:
Sure Bela Ban, we will try this out and share the results.

Also, request any of the Infinispan Folks to help us to get the replication (mode: SYNC) working with Infinispan 8.2.2.
Any help on this would be really great.

Regs,
Manohar.

-----Original Message-----
From: infinispan-dev-bounces at lists.jboss.org<mailto:infinispan-dev-bounces at lists.jboss.org> [mailto:infinispan-dev-bounces at lists.jboss.org<mailto:infinispan-dev-bounces at lists.jboss.org>] On Behalf Of Bela Ban
Sent: Tuesday, August 23, 2016 1:43 PM
To: infinispan-dev at lists.jboss.org<mailto:infinispan-dev at lists.jboss.org>
Subject: Re: [infinispan-dev] JGroups Error: JGRP000029 observed with "JGroups 3.6.8 + Infinispan 8.2.2":


On 22/08/16 20:22, Manohar SL wrote:
> Yes, we need some quick help from Infinispan Folks on the needed
> changes to infinispan-config.xml and jgroups.xml to get the combination of "Infinispan 8.2.2 with JGroups 3.6.7" working for Replication mode of Infinispan usage.

> @ Bela Ban:
>       We could not add the print the stack trace.
>       Since we do not build the JGroups code and directly use it from Infinispan library, we could not attempt this.


Why can't you do this?
- Checkout JGroups src code
- git checkout JGroups-3.6.8.Final
- Make the modification: ex.printStackTrace()
- ant clean jar
- Copy the result JAR file in ./dist to the maven dir


> Regs,
> Manohar.

--
Bela Ban, JGroups lead (http://www.jgroups.org)

_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org<mailto:infinispan-dev at lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/infinispan-dev

_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org<mailto:infinispan-dev at lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/infinispan-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160824/9a1e8855/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: infinispan-config.xml
Type: application/xml
Size: 1152 bytes
Desc: infinispan-config.xml
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160824/9a1e8855/attachment-0001.rdf 

From anistor at redhat.com  Wed Aug 24 08:38:17 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Wed, 24 Aug 2016 15:38:17 +0300
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <4afe8180-7593-3c95-5ce7-0ae24b9e3ded@infinispan.org>
References: <57B76D47.10907@redhat.com>
	<c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
	<CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
	<4afe8180-7593-3c95-5ce7-0ae24b9e3ded@infinispan.org>
Message-ID: <3312aedd-a6f1-57fb-f182-128706054a2b@redhat.com>

No. CQ absolutely needs the previous value.


On 08/24/2016 12:54 PM, Pedro Ruivo wrote:
> If we can drop the pre-events, it would be possible to skip the
> loading/wrapping of the previous value. The method DataContainer.put()
> could return the previous value/metadata that could be used to trigger
> the post-event.
>
> However, it has a problem that I haven't solved it yet: if the value is
> not in memory it will not be loaded (cache-stores / rebalance in progress).
>
> Would it work for CQ?
>
> Cheers,
> Pedro
>
> On 22-08-2016 19:15, William Burns wrote:
>> I like the idea of having a variable to set on the listener annotation.
>> This way we can know for sure if we need to force previous values for
>> some listeners and not for others.
>>
>> It seems the default should be to force the previous value to be more
>> inline with the current behavior, but I fear no one will use the
>> opposite in this case though.  What do you guys think?
>>
>> On Mon, Aug 22, 2016 at 4:31 AM Adrian Nistor <anistor at redhat.com
>> <mailto:anistor at redhat.com>> wrote:
>>
>>      Hi Radim,
>>
>>      Continuous query is built on top of these listeners. CQ _always_ needs
>>      the previous value and it is very convenient in this case that the
>>      command is forced to load the previous value. I imagine there may be
>>      other use cases where we cannot live without the prev value.
>>
>>      I think the listener should be able to state if it needs the prev value
>>      at registration time. Maybe add a new attribute in the Listener
>>      annotation? Similar to how we handled Observation.
>>
>>      Adrian
>>
>>      On 08/19/2016 11:34 PM, Radim Vansa wrote:
>>      > Hi,
>>      >
>>      > as I am trying to simplify current entry wrapping and distribution
>>      code,
>>      > I often find that listeners can get wrong previous value in the event,
>>      > and it sometimes forces the command to load the value even if it
>>      is not
>>      > needed for the command.
>>      >
>>      > I am wondering if we should change the previous value in events to
>>      > Optional - we can usually at least detect that we cannot provide a
>>      > reliable value (e.g. after retry due to topology change, or
>>      because the
>>      > command did not bothered to load the previous value from cache loader)
>>      > and return empty Optional.
>>      >
>>      > WDYT?
>>      >
>>      > Radim
>>      >
>>
>>      _______________________________________________
>>      infinispan-dev mailing list
>>      infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From anistor at redhat.com  Wed Aug 24 09:14:08 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Wed, 24 Aug 2016 16:14:08 +0300
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
References: <57B76D47.10907@redhat.com>
	<c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
	<CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
Message-ID: <2bfd8c97-c4b7-9a8b-586d-75abd7d4c8d3@redhat.com>

I remember there was some discussion to refactor listeners to drop the 
concept of pre-events and have the previous value available in the 
post-event (where applicable). I do not remember what decision was made 
about this. But if we do it, that would be already a big backwards 
incompatible change, so modifying some defaults regarding the behavior 
of the Listener annotation is just mildly disturbing. We'll have to 
apologize/document this anyway and also provide migration advice.

On 08/22/2016 09:15 PM, William Burns wrote:
> I like the idea of having a variable to set on the listener 
> annotation.  This way we can know for sure if we need to force 
> previous values for some listeners and not for others.
>
> It seems the default should be to force the previous value to be more 
> inline with the current behavior, but I fear no one will use the 
> opposite in this case though.  What do you guys think?
>
> On Mon, Aug 22, 2016 at 4:31 AM Adrian Nistor <anistor at redhat.com 
> <mailto:anistor at redhat.com>> wrote:
>
>     Hi Radim,
>
>     Continuous query is built on top of these listeners. CQ _always_ needs
>     the previous value and it is very convenient in this case that the
>     command is forced to load the previous value. I imagine there may be
>     other use cases where we cannot live without the prev value.
>
>     I think the listener should be able to state if it needs the prev
>     value
>     at registration time. Maybe add a new attribute in the Listener
>     annotation? Similar to how we handled Observation.
>
>     Adrian
>
>     On 08/19/2016 11:34 PM, Radim Vansa wrote:
>     > Hi,
>     >
>     > as I am trying to simplify current entry wrapping and
>     distribution code,
>     > I often find that listeners can get wrong previous value in the
>     event,
>     > and it sometimes forces the command to load the value even if it
>     is not
>     > needed for the command.
>     >
>     > I am wondering if we should change the previous value in events to
>     > Optional - we can usually at least detect that we cannot provide a
>     > reliable value (e.g. after retry due to topology change, or
>     because the
>     > command did not bothered to load the previous value from cache
>     loader)
>     > and return empty Optional.
>     >
>     > WDYT?
>     >
>     > Radim
>     >
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160824/a9ba856c/attachment.html 

From dan.berindei at gmail.com  Wed Aug 24 10:32:42 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Wed, 24 Aug 2016 17:32:42 +0300
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <2bfd8c97-c4b7-9a8b-586d-75abd7d4c8d3@redhat.com>
References: <57B76D47.10907@redhat.com>
	<c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
	<CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
	<2bfd8c97-c4b7-9a8b-586d-75abd7d4c8d3@redhat.com>
Message-ID: <CA+nfvwQG5JHgU8McU0j8eJUUZ8s=MvsSic1vz4YMtfWFds3gag@mail.gmail.com>

On Wed, Aug 24, 2016 at 4:14 PM, Adrian Nistor <anistor at redhat.com> wrote:
> I remember there was some discussion to refactor listeners to drop the
> concept of pre-events and have the previous value available in the
> post-event (where applicable). I do not remember what decision was made
> about this. But if we do it, that would be already a big backwards
> incompatible change, so modifying some defaults regarding the behavior of
> the Listener annotation is just mildly disturbing. We'll have to
> apologize/document this anyway and also provide migration advice.
>

That discussion died out because we couldn't decide whether we really
need to maintain the current listeners or we could switch to
supporting only JCache and FunctionalMap listener APIs.

FWIW I'm all for modernizing the core listeners, removing the pre
events, and allowing listeners to receive the previous value in the
post event. I'd also change the API to be interface-based instead of
annotation-based.

>
> On 08/22/2016 09:15 PM, William Burns wrote:
>
> I like the idea of having a variable to set on the listener annotation.
> This way we can know for sure if we need to force previous values for some
> listeners and not for others.
>
> It seems the default should be to force the previous value to be more inline
> with the current behavior, but I fear no one will use the opposite in this
> case though.  What do you guys think?

Actually, the current behaviour is *not* to force the previous value.
If you have the entry in the data container, yes, you'll see it in the
listener, but if the entry is in a store, you won't. Clustered
listeners do get the previous value even if it's remote, but not if
the entry is passivated.

>
> On Mon, Aug 22, 2016 at 4:31 AM Adrian Nistor <anistor at redhat.com> wrote:
>>
>> Hi Radim,
>>
>> Continuous query is built on top of these listeners. CQ _always_ needs
>> the previous value and it is very convenient in this case that the
>> command is forced to load the previous value. I imagine there may be
>> other use cases where we cannot live without the prev value.

Unfortunately, if a command is retried in a non-tx cache
(event.isCommandRetried() == true), the listener may receive the new
value as the previous value. So CQ needs to support this case, or
we'll have to finally fix it in core. I'd mention
versioning/tombstones, but I fear Sanne is going to read this and
derail the thread ;)

>>
>> I think the listener should be able to state if it needs the prev value
>> at registration time. Maybe add a new attribute in the Listener
>> annotation? Similar to how we handled Observation.
>>

Actually, with the current API, the only way to get the previous value
is with the pre event, so we could interpret @Listener(observation =
POST) as a sign that the listener doesn't need the previous value.

>> Adrian
>>
>> On 08/19/2016 11:34 PM, Radim Vansa wrote:
>> > Hi,
>> >
>> > as I am trying to simplify current entry wrapping and distribution code,
>> > I often find that listeners can get wrong previous value in the event,
>> > and it sometimes forces the command to load the value even if it is not
>> > needed for the command.
>> >
>> > I am wondering if we should change the previous value in events to
>> > Optional - we can usually at least detect that we cannot provide a
>> > reliable value (e.g. after retry due to topology change, or because the
>> > command did not bothered to load the previous value from cache loader)
>> > and return empty Optional.
>> >

It would be a breaking change without a lot of benefit, so I would not
make this change.

If we added a getPreviousValue() method for post events, then yes, it
could return an Optional. I'm not yet sure it's a good fit for
isCommandRetried() == true, since in that case we usually have a
previous value, we're just not sure it's the correct one.


>> > WDYT?
>> >
>> > Radim
>> >
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From rvansa at redhat.com  Mon Aug 29 07:33:47 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Mon, 29 Aug 2016 13:33:47 +0200
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <CA+nfvwQG5JHgU8McU0j8eJUUZ8s=MvsSic1vz4YMtfWFds3gag@mail.gmail.com>
References: <57B76D47.10907@redhat.com>
	<c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
	<CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
	<2bfd8c97-c4b7-9a8b-586d-75abd7d4c8d3@redhat.com>
	<CA+nfvwQG5JHgU8McU0j8eJUUZ8s=MvsSic1vz4YMtfWFds3gag@mail.gmail.com>
Message-ID: <57C41D9B.10509@redhat.com>

On 08/24/2016 04:32 PM, Dan Berindei wrote:
> On Wed, Aug 24, 2016 at 4:14 PM, Adrian Nistor <anistor at redhat.com> wrote:
>> I remember there was some discussion to refactor listeners to drop the
>> concept of pre-events and have the previous value available in the
>> post-event (where applicable). I do not remember what decision was made
>> about this. But if we do it, that would be already a big backwards
>> incompatible change, so modifying some defaults regarding the behavior of
>> the Listener annotation is just mildly disturbing. We'll have to
>> apologize/document this anyway and also provide migration advice.
>>
> That discussion died out because we couldn't decide whether we really
> need to maintain the current listeners or we could switch to
> supporting only JCache and FunctionalMap listener APIs.
>
> FWIW I'm all for modernizing the core listeners, removing the pre
> events, and allowing listeners to receive the previous value in the
> post event. I'd also change the API to be interface-based instead of
> annotation-based.
>
>> On 08/22/2016 09:15 PM, William Burns wrote:
>>
>> I like the idea of having a variable to set on the listener annotation.
>> This way we can know for sure if we need to force previous values for some
>> listeners and not for others.
>>
>> It seems the default should be to force the previous value to be more inline
>> with the current behavior, but I fear no one will use the opposite in this
>> case though.  What do you guys think?
> Actually, the current behaviour is *not* to force the previous value.
> If you have the entry in the data container, yes, you'll see it in the
> listener, but if the entry is in a store, you won't. Clustered
> listeners do get the previous value even if it's remote, but not if
> the entry is passivated.

So prev values are not working as doc states (silently returning wrong 
values) in case that:
- topology changes (command retry)
- persistence is used

These are quite non-obvious "if"s for users :-/ I'd call listeners a 
non-reliable feature.

>
>> On Mon, Aug 22, 2016 at 4:31 AM Adrian Nistor <anistor at redhat.com> wrote:
>>> Hi Radim,
>>>
>>> Continuous query is built on top of these listeners. CQ _always_ needs
>>> the previous value and it is very convenient in this case that the
>>> command is forced to load the previous value. I imagine there may be
>>> other use cases where we cannot live without the prev value.
> Unfortunately, if a command is retried in a non-tx cache
> (event.isCommandRetried() == true), the listener may receive the new
> value as the previous value. So CQ needs to support this case, or
> we'll have to finally fix it in core. I'd mention
> versioning/tombstones, but I fear Sanne is going to read this and
> derail the thread ;)
>
>>> I think the listener should be able to state if it needs the prev value
>>> at registration time. Maybe add a new attribute in the Listener
>>> annotation? Similar to how we handled Observation.
>>>
> Actually, with the current API, the only way to get the previous value
> is with the pre event, so we could interpret @Listener(observation =
> POST) as a sign that the listener doesn't need the previous value.

But pre events are also unreliable as they just notify that something 
may or may not happen in the future.

>
>>> Adrian
>>>
>>> On 08/19/2016 11:34 PM, Radim Vansa wrote:
>>>> Hi,
>>>>
>>>> as I am trying to simplify current entry wrapping and distribution code,
>>>> I often find that listeners can get wrong previous value in the event,
>>>> and it sometimes forces the command to load the value even if it is not
>>>> needed for the command.
>>>>
>>>> I am wondering if we should change the previous value in events to
>>>> Optional - we can usually at least detect that we cannot provide a
>>>> reliable value (e.g. after retry due to topology change, or because the
>>>> command did not bothered to load the previous value from cache loader)
>>>> and return empty Optional.
>>>>
> It would be a breaking change without a lot of benefit, so I would not
> make this change.
>
> If we added a getPreviousValue() method for post events, then yes, it
> could return an Optional. I'm not yet sure it's a good fit for
> isCommandRetried() == true, since in that case we usually have a
> previous value, we're just not sure it's the correct one.

What value has a "previous value" when you can't know it's correct? I 
see that as the main problem.

I wouldn't introduce any complexities (listener flags telling if it 
needs or does not need previous value) at this point - the less code the 
better, until we fix what we already have.

>
>
>>>> WDYT?
>>>>
>>>> Radim
>>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From anistor at redhat.com  Mon Aug 29 09:55:18 2016
From: anistor at redhat.com (Adrian Nistor)
Date: Mon, 29 Aug 2016 16:55:18 +0300
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <57C41D9B.10509@redhat.com>
References: <57B76D47.10907@redhat.com>
	<c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
	<CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
	<2bfd8c97-c4b7-9a8b-586d-75abd7d4c8d3@redhat.com>
	<CA+nfvwQG5JHgU8McU0j8eJUUZ8s=MvsSic1vz4YMtfWFds3gag@mail.gmail.com>
	<57C41D9B.10509@redhat.com>
Message-ID: <b80f5e59-a852-f25d-9b35-802429ce05ae@redhat.com>

That is bad and unfortunate because it breaks continuous query and also 
makes CEP impossible to implement.
As I've said, the previous value is absolutely necessary for these use 
cases. If the notification cannot provide it then (in theory) we have 
the alternative to make the CQ or CEP engine store the previous value. 
But that is absurd because it would duplicate the entire contents of the 
cache inside the CQ/CEP engine. So no.

And besides the already highlighted cases where this is broken we've 
recently added new ones. Some of the functional map write operations 
also do not provide the previous value. Actually they do not notify the 
classic @org.infinispan.notifications. Listener at all, in favor of the 
new org.infinispan.commons.api.functional.Listeners.

We badly need to unify these listener interfaces and make sure we always 
provide the previous value.


On 08/29/2016 02:33 PM, Radim Vansa wrote:
> On 08/24/2016 04:32 PM, Dan Berindei wrote:
>> On Wed, Aug 24, 2016 at 4:14 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>> I remember there was some discussion to refactor listeners to drop the
>>> concept of pre-events and have the previous value available in the
>>> post-event (where applicable). I do not remember what decision was made
>>> about this. But if we do it, that would be already a big backwards
>>> incompatible change, so modifying some defaults regarding the behavior of
>>> the Listener annotation is just mildly disturbing. We'll have to
>>> apologize/document this anyway and also provide migration advice.
>>>
>> That discussion died out because we couldn't decide whether we really
>> need to maintain the current listeners or we could switch to
>> supporting only JCache and FunctionalMap listener APIs.
>>
>> FWIW I'm all for modernizing the core listeners, removing the pre
>> events, and allowing listeners to receive the previous value in the
>> post event. I'd also change the API to be interface-based instead of
>> annotation-based.
>>
>>> On 08/22/2016 09:15 PM, William Burns wrote:
>>>
>>> I like the idea of having a variable to set on the listener annotation.
>>> This way we can know for sure if we need to force previous values for some
>>> listeners and not for others.
>>>
>>> It seems the default should be to force the previous value to be more inline
>>> with the current behavior, but I fear no one will use the opposite in this
>>> case though.  What do you guys think?
>> Actually, the current behaviour is *not* to force the previous value.
>> If you have the entry in the data container, yes, you'll see it in the
>> listener, but if the entry is in a store, you won't. Clustered
>> listeners do get the previous value even if it's remote, but not if
>> the entry is passivated.
> So prev values are not working as doc states (silently returning wrong
> values) in case that:
> - topology changes (command retry)
> - persistence is used
>
> These are quite non-obvious "if"s for users :-/ I'd call listeners a
> non-reliable feature.
>
>>> On Mon, Aug 22, 2016 at 4:31 AM Adrian Nistor <anistor at redhat.com> wrote:
>>>> Hi Radim,
>>>>
>>>> Continuous query is built on top of these listeners. CQ _always_ needs
>>>> the previous value and it is very convenient in this case that the
>>>> command is forced to load the previous value. I imagine there may be
>>>> other use cases where we cannot live without the prev value.
>> Unfortunately, if a command is retried in a non-tx cache
>> (event.isCommandRetried() == true), the listener may receive the new
>> value as the previous value. So CQ needs to support this case, or
>> we'll have to finally fix it in core. I'd mention
>> versioning/tombstones, but I fear Sanne is going to read this and
>> derail the thread ;)
>>
>>>> I think the listener should be able to state if it needs the prev value
>>>> at registration time. Maybe add a new attribute in the Listener
>>>> annotation? Similar to how we handled Observation.
>>>>
>> Actually, with the current API, the only way to get the previous value
>> is with the pre event, so we could interpret @Listener(observation =
>> POST) as a sign that the listener doesn't need the previous value.
> But pre events are also unreliable as they just notify that something
> may or may not happen in the future.
>
>>>> Adrian
>>>>
>>>> On 08/19/2016 11:34 PM, Radim Vansa wrote:
>>>>> Hi,
>>>>>
>>>>> as I am trying to simplify current entry wrapping and distribution code,
>>>>> I often find that listeners can get wrong previous value in the event,
>>>>> and it sometimes forces the command to load the value even if it is not
>>>>> needed for the command.
>>>>>
>>>>> I am wondering if we should change the previous value in events to
>>>>> Optional - we can usually at least detect that we cannot provide a
>>>>> reliable value (e.g. after retry due to topology change, or because the
>>>>> command did not bothered to load the previous value from cache loader)
>>>>> and return empty Optional.
>>>>>
>> It would be a breaking change without a lot of benefit, so I would not
>> make this change.
>>
>> If we added a getPreviousValue() method for post events, then yes, it
>> could return an Optional. I'm not yet sure it's a good fit for
>> isCommandRetried() == true, since in that case we usually have a
>> previous value, we're just not sure it's the correct one.
> What value has a "previous value" when you can't know it's correct? I
> see that as the main problem.
>
> I wouldn't introduce any complexities (listener flags telling if it
> needs or does not need previous value) at this point - the less code the
> better, until we fix what we already have.
>
>>
>>>>> WDYT?
>>>>>
>>>>> Radim
>>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>


From rvansa at redhat.com  Mon Aug 29 11:53:40 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Mon, 29 Aug 2016 17:53:40 +0200
Subject: [infinispan-dev] Unwrapping exceptions
Message-ID: <57C45A84.9060801@redhat.com>

When enhancing functional API implementation, I've stumbled upon the way 
the exceptions from lambdas should be thrown to user code.

Intuitively I've expected that when my lambda throws an exception, I'll 
catch it from cache.eval(...). That may be the case when the lambda is 
still executed locally, but when it goes remote, the exception is 
wrapped into RemoteException etc.

In the past, exceptions being thrown were a matter of failure inside 
Infinispan, but what about here? User does not need to know where the 
functional expression has been executed, so it seems to me that he 
should get the exception directly.

WDYT?

Radim

-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team	


From dan.berindei at gmail.com  Mon Aug 29 12:07:01 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 29 Aug 2016 19:07:01 +0300
Subject: [infinispan-dev] Unwrapping exceptions
In-Reply-To: <57C45A84.9060801@redhat.com>
References: <57C45A84.9060801@redhat.com>
Message-ID: <CA+nfvwQBP9j0iMfbHd2OFHZiEjGeceFQ9y8mTA2+BcFVp5B8tw@mail.gmail.com>

-1, I don't think we need to protect the user from knowing that his
lambda was executed on a remote node. On the contrary, it might help
him/her with debugging.

FWIW, I don't think the cache should ever throw the exact exception
that the lambda raised - I'd always wrap it in some kind of
CacheException.

Cheers
Dan

From rvansa at redhat.com  Mon Aug 29 12:54:12 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Mon, 29 Aug 2016 18:54:12 +0200
Subject: [infinispan-dev] Unwrapping exceptions
In-Reply-To: <CA+nfvwQBP9j0iMfbHd2OFHZiEjGeceFQ9y8mTA2+BcFVp5B8tw@mail.gmail.com>
References: <57C45A84.9060801@redhat.com>
	<CA+nfvwQBP9j0iMfbHd2OFHZiEjGeceFQ9y8mTA2+BcFVp5B8tw@mail.gmail.com>
Message-ID: <57C468B4.1070000@redhat.com>

The intention was not to protect the user from knowing where the code 
was executed, but rather simplify exception handling when he wants to 
handle different exceptions from his code (though, throwing exception on 
remote node is not too efficient). And the argument was that he does not 
*need* to know it.

As for the debugging aid, it could make sense to add the remote stack 
trace to suppressed exceptions, though I don't think that it will be of 
any use to him.

R.

On 08/29/2016 06:07 PM, Dan Berindei wrote:
> -1, I don't think we need to protect the user from knowing that his
> lambda was executed on a remote node. On the contrary, it might help
> him/her with debugging.
>
> FWIW, I don't think the cache should ever throw the exact exception
> that the lambda raised - I'd always wrap it in some kind of
> CacheException.
>
> Cheers
> Dan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From rvansa at redhat.com  Tue Aug 30 03:30:17 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 30 Aug 2016 09:30:17 +0200
Subject: [infinispan-dev] SKIP_CACHE_LOAD and write skew check
Message-ID: <57C53609.4020409@redhat.com>

I've noticed that SKIP_CACHE_LOAD flag is ignored when performing the 
write skew check (the old entry is loaded from cache store), though, it 
is not ignored (and the entry is not loaded) when there's 
CACHE_MODE_LOCAL [2]. Is that intended behaviour, or just "conserved" by 
tests?

The flag itself allows to break things [3], so I guess that user sets 
the flag only when he knows that there is no such entry in cache store 
and it's safe to not load it.

Moreover, I don't understand why the WSC should be performed on 
originator (or backup owners) [4].

Thanks for explanations (I'll try to add them as comments to the code - 
or make the reasoning more obvious).

Radim

[1] 
https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L132
[2] 
https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L96
[3] 
https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/context/Flag.java#L89
[4] 
https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L85


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From emmanuel at hibernate.org  Tue Aug 30 04:38:42 2016
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Tue, 30 Aug 2016 10:38:42 +0200
Subject: [infinispan-dev] Optional in listener events
In-Reply-To: <b80f5e59-a852-f25d-9b35-802429ce05ae@redhat.com>
References: <57B76D47.10907@redhat.com>
	<c6b6b86f-dee3-ab26-e60d-d69687a39b26@redhat.com>
	<CA+YCuUOW4kRhtvXhefC1Hdmt2i0vzFYh9BKny+JHzPRe6vw_+w@mail.gmail.com>
	<2bfd8c97-c4b7-9a8b-586d-75abd7d4c8d3@redhat.com>
	<CA+nfvwQG5JHgU8McU0j8eJUUZ8s=MvsSic1vz4YMtfWFds3gag@mail.gmail.com>
	<57C41D9B.10509@redhat.com>
	<b80f5e59-a852-f25d-9b35-802429ce05ae@redhat.com>
Message-ID: <20160830083842.GL90254@hibernate.org>

Just so you see Adrian is a lone wolf, there is no way to implement
continuous query nor listeners whose function is essentially
accumulators if we don't have the previous value. This is an important
feature.

Topology change is a situation that can be detected and notified to
rebuild the accumulator / CQ explicitly. But not sending the previous
value arbitrarily is something the CQ engine cannot account for and is a
no go.

Of course, reliability during things like topology changes would be even
better.

Emmanuel

On Mon 2016-08-29 16:55, Adrian Nistor wrote:
> That is bad and unfortunate because it breaks continuous query and also 
> makes CEP impossible to implement.
> As I've said, the previous value is absolutely necessary for these use 
> cases. If the notification cannot provide it then (in theory) we have 
> the alternative to make the CQ or CEP engine store the previous value. 
> But that is absurd because it would duplicate the entire contents of the 
> cache inside the CQ/CEP engine. So no.
> 
> And besides the already highlighted cases where this is broken we've 
> recently added new ones. Some of the functional map write operations 
> also do not provide the previous value. Actually they do not notify the 
> classic @org.infinispan.notifications. Listener at all, in favor of the 
> new org.infinispan.commons.api.functional.Listeners.
> 
> We badly need to unify these listener interfaces and make sure we always 
> provide the previous value.
> 
> 
> On 08/29/2016 02:33 PM, Radim Vansa wrote:
> > On 08/24/2016 04:32 PM, Dan Berindei wrote:
> >> On Wed, Aug 24, 2016 at 4:14 PM, Adrian Nistor <anistor at redhat.com> wrote:
> >>> I remember there was some discussion to refactor listeners to drop the
> >>> concept of pre-events and have the previous value available in the
> >>> post-event (where applicable). I do not remember what decision was made
> >>> about this. But if we do it, that would be already a big backwards
> >>> incompatible change, so modifying some defaults regarding the behavior of
> >>> the Listener annotation is just mildly disturbing. We'll have to
> >>> apologize/document this anyway and also provide migration advice.
> >>>
> >> That discussion died out because we couldn't decide whether we really
> >> need to maintain the current listeners or we could switch to
> >> supporting only JCache and FunctionalMap listener APIs.
> >>
> >> FWIW I'm all for modernizing the core listeners, removing the pre
> >> events, and allowing listeners to receive the previous value in the
> >> post event. I'd also change the API to be interface-based instead of
> >> annotation-based.
> >>
> >>> On 08/22/2016 09:15 PM, William Burns wrote:
> >>>
> >>> I like the idea of having a variable to set on the listener annotation.
> >>> This way we can know for sure if we need to force previous values for some
> >>> listeners and not for others.
> >>>
> >>> It seems the default should be to force the previous value to be more inline
> >>> with the current behavior, but I fear no one will use the opposite in this
> >>> case though.  What do you guys think?
> >> Actually, the current behaviour is *not* to force the previous value.
> >> If you have the entry in the data container, yes, you'll see it in the
> >> listener, but if the entry is in a store, you won't. Clustered
> >> listeners do get the previous value even if it's remote, but not if
> >> the entry is passivated.
> > So prev values are not working as doc states (silently returning wrong
> > values) in case that:
> > - topology changes (command retry)
> > - persistence is used
> >
> > These are quite non-obvious "if"s for users :-/ I'd call listeners a
> > non-reliable feature.
> >
> >>> On Mon, Aug 22, 2016 at 4:31 AM Adrian Nistor <anistor at redhat.com> wrote:
> >>>> Hi Radim,
> >>>>
> >>>> Continuous query is built on top of these listeners. CQ _always_ needs
> >>>> the previous value and it is very convenient in this case that the
> >>>> command is forced to load the previous value. I imagine there may be
> >>>> other use cases where we cannot live without the prev value.
> >> Unfortunately, if a command is retried in a non-tx cache
> >> (event.isCommandRetried() == true), the listener may receive the new
> >> value as the previous value. So CQ needs to support this case, or
> >> we'll have to finally fix it in core. I'd mention
> >> versioning/tombstones, but I fear Sanne is going to read this and
> >> derail the thread ;)
> >>
> >>>> I think the listener should be able to state if it needs the prev value
> >>>> at registration time. Maybe add a new attribute in the Listener
> >>>> annotation? Similar to how we handled Observation.
> >>>>
> >> Actually, with the current API, the only way to get the previous value
> >> is with the pre event, so we could interpret @Listener(observation =
> >> POST) as a sign that the listener doesn't need the previous value.
> > But pre events are also unreliable as they just notify that something
> > may or may not happen in the future.
> >
> >>>> Adrian
> >>>>
> >>>> On 08/19/2016 11:34 PM, Radim Vansa wrote:
> >>>>> Hi,
> >>>>>
> >>>>> as I am trying to simplify current entry wrapping and distribution code,
> >>>>> I often find that listeners can get wrong previous value in the event,
> >>>>> and it sometimes forces the command to load the value even if it is not
> >>>>> needed for the command.
> >>>>>
> >>>>> I am wondering if we should change the previous value in events to
> >>>>> Optional - we can usually at least detect that we cannot provide a
> >>>>> reliable value (e.g. after retry due to topology change, or because the
> >>>>> command did not bothered to load the previous value from cache loader)
> >>>>> and return empty Optional.
> >>>>>
> >> It would be a breaking change without a lot of benefit, so I would not
> >> make this change.
> >>
> >> If we added a getPreviousValue() method for post events, then yes, it
> >> could return an Optional. I'm not yet sure it's a good fit for
> >> isCommandRetried() == true, since in that case we usually have a
> >> previous value, we're just not sure it's the correct one.
> > What value has a "previous value" when you can't know it's correct? I
> > see that as the main problem.
> >
> > I wouldn't introduce any complexities (listener flags telling if it
> > needs or does not need previous value) at this point - the less code the
> > better, until we fix what we already have.
> >
> >>
> >>>>> WDYT?
> >>>>>
> >>>>> Radim
> >>>>>
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From sanne at infinispan.org  Tue Aug 30 06:06:25 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 30 Aug 2016 11:06:25 +0100
Subject: [infinispan-dev] SKIP_CACHE_LOAD and write skew check
In-Reply-To: <57C53609.4020409@redhat.com>
References: <57C53609.4020409@redhat.com>
Message-ID: <CAFm4XO3Kq4AnppDmy068wycciWj7TVwddkZSRZ8JpKOvB2bQTQ@mail.gmail.com>

I'm not sure why this specific Flag behaves like this, but it's clear
that Flags can break many things; I suspect originally they were meant
to be internal-only, and I got them exposed when as a user I was
having big performance trouble when doing Put operations of 1GB each
and the damn thing would load the previous value from the MySQL backed
Cachestore, while my puts didn't use the return value :)

Maybe it's time to reconsider this and split the flags up in "internal
ones" and public API flags? It would be nice to limit the damage that
end users can do, many flags could be taken away, or reshape our APIs
to not really need so many (for example not having the put method
return a value, or automating such optimisations with internal magic).

+1 to perform the WSC only on the primary owner, that seems like a severe bug?


On 30 August 2016 at 08:30, Radim Vansa <rvansa at redhat.com> wrote:
> I've noticed that SKIP_CACHE_LOAD flag is ignored when performing the
> write skew check (the old entry is loaded from cache store), though, it
> is not ignored (and the entry is not loaded) when there's
> CACHE_MODE_LOCAL [2]. Is that intended behaviour, or just "conserved" by
> tests?
>
> The flag itself allows to break things [3], so I guess that user sets
> the flag only when he knows that there is no such entry in cache store
> and it's safe to not load it.
>
> Moreover, I don't understand why the WSC should be performed on
> originator (or backup owners) [4].
>
> Thanks for explanations (I'll try to add them as comments to the code -
> or make the reasoning more obvious).
>
> Radim
>
> [1]
> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L132
> [2]
> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L96
> [3]
> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/context/Flag.java#L89
> [4]
> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L85
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From sanne at infinispan.org  Tue Aug 30 06:11:13 2016
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 30 Aug 2016 11:11:13 +0100
Subject: [infinispan-dev] Unwrapping exceptions
In-Reply-To: <57C468B4.1070000@redhat.com>
References: <57C45A84.9060801@redhat.com>
	<CA+nfvwQBP9j0iMfbHd2OFHZiEjGeceFQ9y8mTA2+BcFVp5B8tw@mail.gmail.com>
	<57C468B4.1070000@redhat.com>
Message-ID: <CAFm4XO2Jge+TK3PA6vgW6XDDxZ5Ezn-4+6b2jy_XnCZ6i=o6Ag@mail.gmail.com>

Yes please make sure that the kind of end users exceptions are the
same for the client, regardless if the originator happens to be an
owner as well.

It's valuable to know that the exception happened on another node, but
the exception type (and primary message) should be the same.

Bonus points to not have exceptions at all :)
In Elasticsearch they developed a new scripting language for a use
case similar to our "lambda execution" which basically restricts in
the language itself what is safe to do vs what you can't do.
I'm not sure about developing a new language but from this point of
view it's brilliant..



On 29 August 2016 at 17:54, Radim Vansa <rvansa at redhat.com> wrote:
> The intention was not to protect the user from knowing where the code
> was executed, but rather simplify exception handling when he wants to
> handle different exceptions from his code (though, throwing exception on
> remote node is not too efficient). And the argument was that he does not
> *need* to know it.
>
> As for the debugging aid, it could make sense to add the remote stack
> trace to suppressed exceptions, though I don't think that it will be of
> any use to him.
>
> R.
>
> On 08/29/2016 06:07 PM, Dan Berindei wrote:
>> -1, I don't think we need to protect the user from knowing that his
>> lambda was executed on a remote node. On the contrary, it might help
>> him/her with debugging.
>>
>> FWIW, I don't think the cache should ever throw the exact exception
>> that the lambda raised - I'd always wrap it in some kind of
>> CacheException.
>>
>> Cheers
>> Dan
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From pedro at infinispan.org  Tue Aug 30 06:52:14 2016
From: pedro at infinispan.org (Pedro Ruivo)
Date: Tue, 30 Aug 2016 11:52:14 +0100
Subject: [infinispan-dev] SKIP_CACHE_LOAD and write skew check
In-Reply-To: <CAFm4XO3Kq4AnppDmy068wycciWj7TVwddkZSRZ8JpKOvB2bQTQ@mail.gmail.com>
References: <57C53609.4020409@redhat.com>
	<CAFm4XO3Kq4AnppDmy068wycciWj7TVwddkZSRZ8JpKOvB2bQTQ@mail.gmail.com>
Message-ID: <4c270c88-ea79-3c94-3c12-81b02e6a4fe9@infinispan.org>

The WSC is only performed on the primary owner. But the originator needs 
to keep track of the version read in order to the WSC be performed.

I've changed this behavior for IGNORE_RETURN_VALUE flags, where the 
entry is mark as "not-read" (if it wasn't read before). Then, the WSC is 
skipped for this entry.

I probably should have done the same thing for SKIP_CACHE_LOAD.

CACHE_MODE_LOCAL is other flag that, IMO, shouldn't be supported by 
Transactional Caches.

I agree with Sanne and we should re-check all flags since most of them 
can hurt the data consistency.

On 30-08-2016 11:06, Sanne Grinovero wrote:
> I'm not sure why this specific Flag behaves like this, but it's clear
> that Flags can break many things; I suspect originally they were meant
> to be internal-only, and I got them exposed when as a user I was
> having big performance trouble when doing Put operations of 1GB each
> and the damn thing would load the previous value from the MySQL backed
> Cachestore, while my puts didn't use the return value :)
>
> Maybe it's time to reconsider this and split the flags up in "internal
> ones" and public API flags? It would be nice to limit the damage that
> end users can do, many flags could be taken away, or reshape our APIs
> to not really need so many (for example not having the put method
> return a value, or automating such optimisations with internal magic).
>
> +1 to perform the WSC only on the primary owner, that seems like a severe bug?
>
>
> On 30 August 2016 at 08:30, Radim Vansa <rvansa at redhat.com> wrote:
>> I've noticed that SKIP_CACHE_LOAD flag is ignored when performing the
>> write skew check (the old entry is loaded from cache store), though, it
>> is not ignored (and the entry is not loaded) when there's
>> CACHE_MODE_LOCAL [2]. Is that intended behaviour, or just "conserved" by
>> tests?
>>
>> The flag itself allows to break things [3], so I guess that user sets
>> the flag only when he knows that there is no such entry in cache store
>> and it's safe to not load it.
>>
>> Moreover, I don't understand why the WSC should be performed on
>> originator (or backup owners) [4].
>>
>> Thanks for explanations (I'll try to add them as comments to the code -
>> or make the reasoning more obvious).
>>
>> Radim
>>
>> [1]
>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L132
>> [2]
>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L96
>> [3]
>> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/context/Flag.java#L89
>> [4]
>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L85
>>
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

From rvansa at redhat.com  Tue Aug 30 06:59:45 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 30 Aug 2016 12:59:45 +0200
Subject: [infinispan-dev] SKIP_CACHE_LOAD and write skew check
In-Reply-To: <4c270c88-ea79-3c94-3c12-81b02e6a4fe9@infinispan.org>
References: <57C53609.4020409@redhat.com>
	<CAFm4XO3Kq4AnppDmy068wycciWj7TVwddkZSRZ8JpKOvB2bQTQ@mail.gmail.com>
	<4c270c88-ea79-3c94-3c12-81b02e6a4fe9@infinispan.org>
Message-ID: <57C56721.9020809@redhat.com>

On 08/30/2016 12:52 PM, Pedro Ruivo wrote:
> The WSC is only performed on the primary owner. But the originator needs
> to keep track of the version read in order to the WSC be performed.
>
> I've changed this behavior for IGNORE_RETURN_VALUE flags, where the
> entry is mark as "not-read" (if it wasn't read before). Then, the WSC is
> skipped for this entry.
>
> I probably should have done the same thing for SKIP_CACHE_LOAD.
>
> CACHE_MODE_LOCAL is other flag that, IMO, shouldn't be supported by
> Transactional Caches.
>
> I agree with Sanne and we should re-check all flags since most of them
> can hurt the data consistency.

I think there are valid use cases for skipping the cache load, and it's 
hidden well enough that users won't use that just accidentally. But I 
don't see any opinions if the WSC should honor or ignore this flag (I 
would honor it).

Radim

>
> On 30-08-2016 11:06, Sanne Grinovero wrote:
>> I'm not sure why this specific Flag behaves like this, but it's clear
>> that Flags can break many things; I suspect originally they were meant
>> to be internal-only, and I got them exposed when as a user I was
>> having big performance trouble when doing Put operations of 1GB each
>> and the damn thing would load the previous value from the MySQL backed
>> Cachestore, while my puts didn't use the return value :)
>>
>> Maybe it's time to reconsider this and split the flags up in "internal
>> ones" and public API flags? It would be nice to limit the damage that
>> end users can do, many flags could be taken away, or reshape our APIs
>> to not really need so many (for example not having the put method
>> return a value, or automating such optimisations with internal magic).
>>
>> +1 to perform the WSC only on the primary owner, that seems like a severe bug?
>>
>>
>> On 30 August 2016 at 08:30, Radim Vansa <rvansa at redhat.com> wrote:
>>> I've noticed that SKIP_CACHE_LOAD flag is ignored when performing the
>>> write skew check (the old entry is loaded from cache store), though, it
>>> is not ignored (and the entry is not loaded) when there's
>>> CACHE_MODE_LOCAL [2]. Is that intended behaviour, or just "conserved" by
>>> tests?
>>>
>>> The flag itself allows to break things [3], so I guess that user sets
>>> the flag only when he knows that there is no such entry in cache store
>>> and it's safe to not load it.
>>>
>>> Moreover, I don't understand why the WSC should be performed on
>>> originator (or backup owners) [4].
>>>
>>> Thanks for explanations (I'll try to add them as comments to the code -
>>> or make the reasoning more obvious).
>>>
>>> Radim
>>>
>>> [1]
>>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L132
>>> [2]
>>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L96
>>> [3]
>>> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/context/Flag.java#L89
>>> [4]
>>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L85
>>>
>>>
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Tue Aug 30 07:29:11 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Tue, 30 Aug 2016 14:29:11 +0300
Subject: [infinispan-dev] SKIP_CACHE_LOAD and write skew check
In-Reply-To: <57C56721.9020809@redhat.com>
References: <57C53609.4020409@redhat.com>
	<CAFm4XO3Kq4AnppDmy068wycciWj7TVwddkZSRZ8JpKOvB2bQTQ@mail.gmail.com>
	<4c270c88-ea79-3c94-3c12-81b02e6a4fe9@infinispan.org>
	<57C56721.9020809@redhat.com>
Message-ID: <CA+nfvwRiYL26F+YzmQE3Di0FccbnvggKKCyTvSorBdy-gpEF2Q@mail.gmail.com>

+1 to honor it. I should have done it back when I did ISPN-5643 [1],
but I guess I was afraid about breaking more tests.

Note that before ISPN-5643 conditional writes also used to ignore
SKIP_CACHE_LOAD, and the javadoc encouraged users to enable it
together with SKIP_REMOTE_LOOKUP.

[1]: https://issues.jboss.org/browse/ISPN-5643

Cheers
Dan


On Tue, Aug 30, 2016 at 1:59 PM, Radim Vansa <rvansa at redhat.com> wrote:
> On 08/30/2016 12:52 PM, Pedro Ruivo wrote:
>> The WSC is only performed on the primary owner. But the originator needs
>> to keep track of the version read in order to the WSC be performed.
>>
>> I've changed this behavior for IGNORE_RETURN_VALUE flags, where the
>> entry is mark as "not-read" (if it wasn't read before). Then, the WSC is
>> skipped for this entry.
>>
>> I probably should have done the same thing for SKIP_CACHE_LOAD.
>>
>> CACHE_MODE_LOCAL is other flag that, IMO, shouldn't be supported by
>> Transactional Caches.
>>
>> I agree with Sanne and we should re-check all flags since most of them
>> can hurt the data consistency.
>
> I think there are valid use cases for skipping the cache load, and it's
> hidden well enough that users won't use that just accidentally. But I
> don't see any opinions if the WSC should honor or ignore this flag (I
> would honor it).
>
> Radim
>
>>
>> On 30-08-2016 11:06, Sanne Grinovero wrote:
>>> I'm not sure why this specific Flag behaves like this, but it's clear
>>> that Flags can break many things; I suspect originally they were meant
>>> to be internal-only, and I got them exposed when as a user I was
>>> having big performance trouble when doing Put operations of 1GB each
>>> and the damn thing would load the previous value from the MySQL backed
>>> Cachestore, while my puts didn't use the return value :)
>>>
>>> Maybe it's time to reconsider this and split the flags up in "internal
>>> ones" and public API flags? It would be nice to limit the damage that
>>> end users can do, many flags could be taken away, or reshape our APIs
>>> to not really need so many (for example not having the put method
>>> return a value, or automating such optimisations with internal magic).
>>>
>>> +1 to perform the WSC only on the primary owner, that seems like a severe bug?
>>>
>>>
>>> On 30 August 2016 at 08:30, Radim Vansa <rvansa at redhat.com> wrote:
>>>> I've noticed that SKIP_CACHE_LOAD flag is ignored when performing the
>>>> write skew check (the old entry is loaded from cache store), though, it
>>>> is not ignored (and the entry is not loaded) when there's
>>>> CACHE_MODE_LOCAL [2]. Is that intended behaviour, or just "conserved" by
>>>> tests?
>>>>
>>>> The flag itself allows to break things [3], so I guess that user sets
>>>> the flag only when he knows that there is no such entry in cache store
>>>> and it's safe to not load it.
>>>>
>>>> Moreover, I don't understand why the WSC should be performed on
>>>> originator (or backup owners) [4].
>>>>
>>>> Thanks for explanations (I'll try to add them as comments to the code -
>>>> or make the reasoning more obvious).
>>>>
>>>> Radim
>>>>
>>>> [1]
>>>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L132
>>>> [2]
>>>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L96
>>>> [3]
>>>> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/context/Flag.java#L89
>>>> [4]
>>>> https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/api/flags/FlagsEnabledTest.java#L85
>>>>
>>>>
>>>> --
>>>> Radim Vansa <rvansa at redhat.com>
>>>> JBoss Performance Team
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From dan.berindei at gmail.com  Tue Aug 30 08:16:00 2016
From: dan.berindei at gmail.com (Dan Berindei)
Date: Tue, 30 Aug 2016 15:16:00 +0300
Subject: [infinispan-dev] Unwrapping exceptions
In-Reply-To: <CAFm4XO2Jge+TK3PA6vgW6XDDxZ5Ezn-4+6b2jy_XnCZ6i=o6Ag@mail.gmail.com>
References: <57C45A84.9060801@redhat.com>
	<CA+nfvwQBP9j0iMfbHd2OFHZiEjGeceFQ9y8mTA2+BcFVp5B8tw@mail.gmail.com>
	<57C468B4.1070000@redhat.com>
	<CAFm4XO2Jge+TK3PA6vgW6XDDxZ5Ezn-4+6b2jy_XnCZ6i=o6Ag@mail.gmail.com>
Message-ID: <CA+nfvwRo89e_Xb7+_Bg6k0ATqbM9zJ-v6fW3dMtjQgLhTObVdw@mail.gmail.com>

On Tue, Aug 30, 2016 at 1:11 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
> Yes please make sure that the kind of end users exceptions are the
> same for the client, regardless if the originator happens to be an
> owner as well.
>
> It's valuable to know that the exception happened on another node, but
> the exception type (and primary message) should be the same.
>

Is it really a problem if the local exceptions are wrapped in
CacheException, and the remote ones are wrapped in RemoteException?
RemoteException is a subtype of CacheException, so `catch
(CacheException e)` works with all of them.

Future.get() wraps all exceptions in ExecutionException,
CompletableFuture.join() wraps all exceptions in CompletionException.
So we'd be an outlier if we *didn't* wrap user exceptions.

> Bonus points to not have exceptions at all :)

Error codes FTW? ;)

> In Elasticsearch they developed a new scripting language for a use
> case similar to our "lambda execution" which basically restricts in
> the language itself what is safe to do vs what you can't do.
> I'm not sure about developing a new language but from this point of
> view it's brilliant..
>

I think you impose the same kind of restrictions at the bytecode
level, with something like JaQue [1]. Still, considering that you also
need a way to ship the lambda to the server, a DSL doesn't sound too
bad.

[1]: https://github.com/TrigerSoft/jaque

>
>
> On 29 August 2016 at 17:54, Radim Vansa <rvansa at redhat.com> wrote:
>> The intention was not to protect the user from knowing where the code
>> was executed, but rather simplify exception handling when he wants to
>> handle different exceptions from his code (though, throwing exception on
>> remote node is not too efficient). And the argument was that he does not
>> *need* to know it.
>>

But does the user really need to know that it was an exception in
their lambda vs an exception in Infinispan itself? Most of the time,
there's nothing you can do about it anyway...

>> As for the debugging aid, it could make sense to add the remote stack
>> trace to suppressed exceptions, though I don't think that it will be of
>> any use to him.
>>

If we throw the exact exception that the lambda raised on the remote
node, the user is going to see *only* the remote stack trace.

TBH we have the same problem with RemoteExceptions now: instead of
having a stack trace pointing to the user code calling into
Infinispan, our stack trace points to the Infinispan code handling the
response. But at least we have a chance to "fix" the stack trace of
the wrapper exception in AsyncInterceptorChainImpl.invoke to that it
has the caller's stack trace instead. If we don't have a wrapper,
that's no longer possible.

Cheers
Dan

From rvansa at redhat.com  Tue Aug 30 08:47:49 2016
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 30 Aug 2016 14:47:49 +0200
Subject: [infinispan-dev] Unwrapping exceptions
In-Reply-To: <CA+nfvwRo89e_Xb7+_Bg6k0ATqbM9zJ-v6fW3dMtjQgLhTObVdw@mail.gmail.com>
References: <57C45A84.9060801@redhat.com>
	<CA+nfvwQBP9j0iMfbHd2OFHZiEjGeceFQ9y8mTA2+BcFVp5B8tw@mail.gmail.com>
	<57C468B4.1070000@redhat.com>
	<CAFm4XO2Jge+TK3PA6vgW6XDDxZ5Ezn-4+6b2jy_XnCZ6i=o6Ag@mail.gmail.com>
	<CA+nfvwRo89e_Xb7+_Bg6k0ATqbM9zJ-v6fW3dMtjQgLhTObVdw@mail.gmail.com>
Message-ID: <57C58075.3060303@redhat.com>

On 08/30/2016 02:16 PM, Dan Berindei wrote:
> On Tue, Aug 30, 2016 at 1:11 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
>> Yes please make sure that the kind of end users exceptions are the
>> same for the client, regardless if the originator happens to be an
>> owner as well.
>>
>> It's valuable to know that the exception happened on another node, but
>> the exception type (and primary message) should be the same.
>>
> Is it really a problem if the local exceptions are wrapped in
> CacheException, and the remote ones are wrapped in RemoteException?
> RemoteException is a subtype of CacheException, so `catch
> (CacheException e)` works with all of them.
>
> Future.get() wraps all exceptions in ExecutionException,
> CompletableFuture.join() wraps all exceptions in CompletionException.
> So we'd be an outlier if we *didn't* wrap user exceptions.

Okay, the exceptions can be wrapped in one (and always exactly one) 
level of CacheException. Not as convenient for filtering (try-catch 
block vs. instanceofs on e.getCause()), but makes (enough) sense. I'll 
adapt the PR.

>
>> Bonus points to not have exceptions at all :)
> Error codes FTW? ;)
>
>> In Elasticsearch they developed a new scripting language for a use
>> case similar to our "lambda execution" which basically restricts in
>> the language itself what is safe to do vs what you can't do.
>> I'm not sure about developing a new language but from this point of
>> view it's brilliant..
>>
> I think you impose the same kind of restrictions at the bytecode
> level, with something like JaQue [1]. Still, considering that you also
> need a way to ship the lambda to the server, a DSL doesn't sound too
> bad.
>
> [1]: https://github.com/TrigerSoft/jaque
>
>>
>> On 29 August 2016 at 17:54, Radim Vansa <rvansa at redhat.com> wrote:
>>> The intention was not to protect the user from knowing where the code
>>> was executed, but rather simplify exception handling when he wants to
>>> handle different exceptions from his code (though, throwing exception on
>>> remote node is not too efficient). And the argument was that he does not
>>> *need* to know it.
>>>
> But does the user really need to know that it was an exception in
> their lambda vs an exception in Infinispan itself? Most of the time,
> there's nothing you can do about it anyway...
>
>>> As for the debugging aid, it could make sense to add the remote stack
>>> trace to suppressed exceptions, though I don't think that it will be of
>>> any use to him.
>>>
> If we throw the exact exception that the lambda raised on the remote
> node, the user is going to see *only* the remote stack trace.
>
> TBH we have the same problem with RemoteExceptions now: instead of
> having a stack trace pointing to the user code calling into
> Infinispan, our stack trace points to the Infinispan code handling the
> response. But at least we have a chance to "fix" the stack trace of
> the wrapper exception in AsyncInterceptorChainImpl.invoke to that it
> has the caller's stack trace instead. If we don't have a wrapper,
> that's no longer possible.
>
> Cheers
> Dan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team