From gustavo at infinispan.org  Thu Jun  1 02:58:18 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Thu, 1 Jun 2017 07:58:18 +0100
Subject: [infinispan-dev] Allocation costs of
	TypeConverterDelegatingAdvancedCache
In-Reply-To: <CA+YCuUNbt2_2-NT2f9ehc=4Uo-fqJ1TAHbne6gkcVCa8PD4=0g@mail.gmail.com>
References: <CAFm4XO1gAHeacGXQ-=8kfWK+zCCZ=k_33DMQ+LBYXt2cMS3d8A@mail.gmail.com>
	<CA+YCuUNbt2_2-NT2f9ehc=4Uo-fqJ1TAHbne6gkcVCa8PD4=0g@mail.gmail.com>
Message-ID: <CAH8Ud1E8sPZD3VxMQJ482Gyn1-eVbf+XPtaTGUeqjwNqoF3NQA@mail.gmail.com>

On Wed, May 31, 2017 at 8:05 PM, William Burns <mudokonman at gmail.com> wrote:

> Let me explain why it is there first :) This class was added for two main
> reasons: as a replacement for compatibility and for supporting equality of
> byte[] object. What this class does is at the user side is box the given
> arguments (eg. byte[] -> WrappedByteArray) then the cache only ever deals
> with the boxed types and then does the unboxing for any values that are
> returned.
>
> There are some exceptions with how the boxing/unboxing works for both
> cases such as Streams and Indexing which have to rebox the data to work
> properly. But the cost is pretty minimal.
>
> While compatibility is always on or off unfortunately anyone can pass in a
> byte[] at any point for a key or value. So we need to have these wrappers
> there to make sure they work properly.
>
> We could add a option to the cache, which people didn't show interest
> before, to have a cache that doesn't support byte[] or compatibility. In
> this case there would be no need for the wrapper.
>
>
> Compatibility
>
> By using the wrapper around the cache, compatibility becomes quite trivial
> since we just need a converter in the wrapper and it does everything else
> for us. My guess is the new encoding changes will utilize these wrapper
> classes as well as they are quite easy to plug in and have things just work.
>


New encoding will use a wrapper as well, so that the same cache can use
different data conversions on demand.

Gustavo


>
> Equality
>
> With the rewrite for eviction we lost the ability to use custom Equality
> in the data container. The only option for that is to wrap a byte[] to
> provide our own equality. Therefore the wrapper does this conversion for us
> by converting between automatically.
>
> On Wed, May 31, 2017 at 2:34 PM Sanne Grinovero <sanne at infinispan.org>
> wrote:
>
>> Hi all,
>>
>> I've been running some benchmarks and for the fist time playing with
>> Infinispan 9+, so please bear with me as I might shoot some dumb
>> questions to the list in the following days.
>>
>> The need for TypeConverterDelegatingAdvancedCache to wrap most
>> operations - especially "convertKeys" - is highlighet as one of the
>>
>
> It should be wrapping every operation pretty much.
>
> Unfortunately the methods this hurts the most are putAll, getAll etc as
> they have to not only box every entry but copy them into a new collection
> as you saw in "convertKeys". And for getAll it also has to unbox the return
> as well.
>
> We could reduce allocations in the collection methods by not creating the
> new collection until we run into one key or value that required
> boxing/unboxing. This would still require fully iterating over the
> collection at best case. This should perform well in majority of cases as I
> would expect all or almost all entries either require or don't require the
> boxing. The cases that would be harmed most would be ones that have a
> sparse number of entries that require boxing.
>
>
>> high allocators in my Search-centric use case.
>>
>> I'm wondering:
>>  A - Could this implementation be improved?
>>
>
> Most anything can be improved :) The best way would be to add another knob.
>
>  B - Could I bypass / disable it? Not sure why it's there.
>>
>
> There is no way to bypass currently. Explained above.
>
>
>
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170601/cbe51235/attachment-0001.html 

From slaskawi at redhat.com  Thu Jun  1 02:59:38 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 01 Jun 2017 06:59:38 +0000
Subject: [infinispan-dev] Using load balancers for Infinispan in
	Kubernetes
In-Reply-To: <38965B3D-F854-47BC-9F0B-F56BAEB78FD5@hibernate.org>
References: <CAA9R9O5Z7OW994bewSHRBHLd4gmztkOKnTOoxV22kk15PwwAVA@mail.gmail.com>
	<38965B3D-F854-47BC-9F0B-F56BAEB78FD5@hibernate.org>
Message-ID: <CAA9R9O7stmxcDB1R4fVyc4tzfVpPaYyCrG8WrNBv2pGOVUwFiw@mail.gmail.com>

Hey Emmanuel,

Comments inlined.

Thanks,
Sebastian

On Wed, May 31, 2017 at 2:56 PM Emmanuel Bernard <emmanuel at hibernate.org>
wrote:

> To Sanne?s point, I think HTTP(/2) would be a better longer term path if
> we think we can make it as efficient as current HR. But let?s evaluate the
> numbers of cycles to reach that point. Doing Seb?s approach might be a good
> first step.
>

I will be looking into HTTP/2 implementation starting from today/tomorrow.
So it should be there soon. And of course I will do some benchmarks (or
even help Jiri to upgrade perfCheck to do benchmarks using HTTP/1.1 and
HTTP/2).

Also please bear in mind that there will probably be two ways of switching
protocols - using HTTP/1.1 Upgrade header and TLS/ALPN negotiation. As you
might expect, the latter enforces TLS (and therefore the throughput will be
lower).


> Speaking of Sebastian, I have been discussing with Burr, Edson on the idea
> of a *node* sidecar (as opposed to a *pod* sidecar). To your problem, could
> you use Daemonset to enforce one Load Balancer per node or at least per
> project instead of one per pod deployed with Infinispan in it?
>

Unless I missed anything, it won't buy us anything. The idea behind the POC
is to make all Infinispan nodes directly accessible from the outside world.
The client must be able to access whichever node it wishes. This is
achieved by creating a load balancer per Infinispan pod. So the load
balancer works more like an external IP rather than a "real" load balancer.

Just FYI, another round of comments on L3/L4 TCP Ingress has just started:
https://github.com/kubernetes/kubernetes/issues/23291
The rough estimate is to get it in Kube 1.8. Once this is implemented, we
could use a TCP Ingress per pod (instead of load balancer per pod). The
main difference will probably be in $$$. Load balancers are pretty
expensive.


>
> WDYT, is it possible?
>
> On 30 May 2017, at 20:43, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>
> Hey guys!
>
> Over past few weeks I've been working on accessing Infinispan cluster
> deployed inside Kubernetes from the outside world. The POC diagram looks
> like the following:
>
> <pasted1.png>
>
>
> As a reminder, the easiest (though not the most effective) way to do it is
> to expose a load balancer Service (or a Node Port Service) and access it
> using a client with basic intelligence (so that it doesn't try to update
> server list based on topology information). As you might expect, this won't
> give you much performance but at least you could access the cluster.
> Another approach is to use TLS/SNI but again, the performance would be even
> worse.
>
> During the research I tried to answer this problem and created "External
> IP Controller" [1] (and corresponding Pull Request for mapping
> internal/external addresses [2]). The main idea is to create a controller
> deployed inside Kubernetes which will create (and destroy if not needed) a
> load balancer per Infinispan Pod. Additionally the controller exposes
> mapping between internal and external addresses which allows the client to
> properly update server list as well as consistent hash information. A full
> working example is located here [3].
>
> The biggest question is whether it's worth it? The short answer is yes.
> Here are some benchmark results of performing 10k puts and 10k puts&gets
> (please take them with a big grain of salt, I didn't optimize any server
> settings):
>
>    - Benchmark app deployed inside Kuberenetes and using internal
>    addresses (baseline):
>       - 10k puts: 674.244 ?  16.654
>       - 10k puts&gets: 1288.437 ? 136.207
>    - Benchamrking app deployed in a VM outside of Kubernetes with basic
>    intelligence:
>       - *10k puts: 1465.567 ? 176.349*
>       - *10k puts&gets: 2684.984 ? 114.993*
>    - Benchamrking app deployed in a VM outside of Kubernetes with address
>    mapping and topology aware hashing:
>       - *10k puts: 1052.891 ?  31.218*
>       - *10k puts&gets: 2465.586 ?  85.034*
>
> Note that benchmarking Infinispan from a VM might be very misleading since
> it depends on data center configuration. Benchmarks above definitely
> contain some delay between Google Compute Engine VM and a Kubernetes
> cluster deployed in Google Container Engine. How big is the delay? Hard to
> tell. What counts is the difference between client using basic intelligence
> and topology aware intelligence. And as you can see it's not that small.
>
> So the bottom line - if you can, deploy your application along with
> Infinispan cluster inside Kubernetes. That's the fastest configuration
> since only iptables are involved. Otherwise use a load balancer per pod
> with External IP Controller. If you don't care about performance, just use
> basic client intelligence and expose everything using single load balancer.
>
> Thanks,
> Sebastian
>
> [1] https://github.com/slaskawi/external-ip-proxy
> [2] https://github.com/infinispan/infinispan/pull/5164
> [3] https://github.com/slaskawi/external-ip-proxy/tree/master/benchmark
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170601/d0a193cb/attachment.html 

From slaskawi at redhat.com  Thu Jun  1 03:09:40 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 01 Jun 2017 07:09:40 +0000
Subject: [infinispan-dev] Using load balancers for Infinispan in
	Kubernetes
In-Reply-To: <CAFm4XO3V-f0937Vf8cSfP0KuDEYCPUWRRpvH9b=+xYAuxDaCfw@mail.gmail.com>
References: <CAA9R9O5Z7OW994bewSHRBHLd4gmztkOKnTOoxV22kk15PwwAVA@mail.gmail.com>
	<CAFm4XO22YOCZY8VfEarv8us-=aPo1Tw5hG7AaMq0vCQfAxtiQg@mail.gmail.com>
	<CAA9R9O4a2TkR3PFcrUep=_UKM0bfnZ4dc=Fhmn1sZDLCKWy_9g@mail.gmail.com>
	<1E9DF83C-0BEC-4F7B-B91C-F1B4313F1E9C@redhat.com>
	<CAFm4XO3V-f0937Vf8cSfP0KuDEYCPUWRRpvH9b=+xYAuxDaCfw@mail.gmail.com>
Message-ID: <CAA9R9O4CrfxHsr0pOGJmq6LBSeF0s2CWiXTTZT1WRu4JXx_uYA@mail.gmail.com>

I'm calm!! I'm calm!!
[image: pasted1]
Being serious now, no offence taken and thanks for clarification Sanne.
Next time I'll just publish some percentage difference which will probably
express better what I want to tell. And if you see anything wrong with the
test methodology that could influence the relation between those tests,
just let me know.

Don't you think guys we should have a reference OpenShift/Kubernetes
environment for testing (minikube/minishift are great but it's always "it
works on my machine" type of thing)? I expect there will be more and more
discussions around cloud topics. If you are interested in this, please let
me know off-list. I will try to arrange something.

Thanks,
Sebastian

On Thu, Jun 1, 2017 at 4:49 AM Sanne Grinovero <sanne at infinispan.org> wrote:

> On 31 May 2017 at 17:48, Galder Zamarre?o <galder at redhat.com> wrote:
> > Cool down peoples!
> >
> > http://www.quickmeme.com/meme/35ovcy
> >
> > Sebastian, don't think Sanne was being rude, he's just blunt and we need
> his bluntness :)
> >
> > Sanne, be nice to Sebastian and get him a beer next time around ;)
>
> Hey, he started it! His email was formatted with HTML !? ;)
>
> But seriously, I didn't mean to be rude or disrespectful; if it comes
> across like that I'm sorry. FWIW the answers seemed cool to me too.
>
> Let me actually clarify that I love the attitude of Sebastian to try
> the various approaches and get with some measurements to help with
> important design decisions. It's good that we spend some time
> evaluating the alternatives, and it's equally good that we debate the
> trade-offs here.
>
> As warned in my email I'm "nit-picking" on the benchmark methodology,
> and probably more than the usual, because I care!
>
> I am highlighting what I believe to be useful advice though: the
> absolute metrics of such tests need not be taken as primary
> (exclusive?) decision factor. Which doesn't mean that performing such
> tests is not useful, they certainly provide a lot to think about.
> Yet the interpretation of such results need to not be generalised, and
> the interpretation process is more important than the absolute
> ballpark figures they provide; for example it's paramount to figure
> out which factors of the test could theoretically invert the results.
> Using them to identify a binary faster/slow yes/no to proof/disproof a
> design decision is a dangerous fallacy .. and I'm not picking on
> Sebastian specifically, just reminding about it as we've all been
> guilty of it: confirmation bias, etc..
>
> The best advice I've ever had myself in performance analysis is to not
> try to figure out which implementation is faster "on my machine", but
> to understand why it's producing a specific result, and what is
> preventing it to produce an higher figure.
> Once you know that, it's very valuable information as it will tell you
> either what needs fixing with a benchmark, or what needs to be done to
> improve the performance of your implementation ;)
> So that's why I personally don't publish figures often, but hey I
> still run such tests too and spend a lot of time analysing them, to
> eventually share what I figure out in the process...
>
> Thanks,
> Sanne
>
>
>
> >
> > Peace out! :)
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> >> On 31 May 2017, at 09:38, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
> >>
> >> Hey Sanne,
> >>
> >> Comments inlined.
> >>
> >> Thanks,
> >> Sebastian
> >>
> >> On Tue, May 30, 2017 at 5:58 PM Sanne Grinovero <sanne at infinispan.org>
> wrote:
> >> Hi Sebastian,
> >>
> >> the "intelligent routing" of Hot Rod being one of - if not the main -
> reason to use Hot Rod, I wonder if we shouldn't rather suggest people to
> stick with HTTP (REST) in such architectures.
> >>
> >> Several people have suggested in the past the need to have an HTTP
> smart load balancer which would be able to route the external REST requests
> to the right node. Essentially have people use REST over the wider network,
> up to reaching the Infinispan cluster where the service endpoint (the load
> balancer) can convert them to optimised Hot Rod calls, or just leave them
> in the same format but routing them with the same intelligence to the right
> nodes.
> >>
> >> I realise my proposal requires some work on several fronts, at very
> least we would need:
> >>  - feature parity Hot Rod / REST so that people can actually use it
> >>  - a REST load balancer
> >>
> >> But I think the output of such a direction would be far more reusable,
> as both these points are high on the wish list anyway.
> >>
> >> Unfortunately I'm not convinced into this idea. Let me elaborate...
> >>
> >> It goes without saying that HTTP payload is simply larger and require
> much more processing. That alone makes it slower than Hot Rod (I believe
> Martin could provide you some numbers on that). The second arguments is
> that switching/routing inside Kubernetes is bloody fast (since it's based
> on iptables) and some cloud vendors optimize it even further (e.g. Google
> Andromeda [1][2], I would be surprised if AWS didn't have anything
> similar). During the work on this prototype I wrote a simple async binary
> proxy [3] and measured GCP load balancer vs my proxy performance. They were
> twice as fast [4][5]. You may argue whether I could write a better proxy.
> Probably I could, but the bottom line is that another performance hit is
> inevitable. They are really fast and they operate on their own
> infrastructure (load balancers is something that is provided by the cloud
> vendor to Kubernetes, not the other way around).
> >>
> >> So with all that in mind, are we going to get better results comparing
> to my proposal for Hot Rod? I dare to doubt, even with HTTP/2 support
> (which comes really soon I hope). The second question is whether this new
> "REST load balancer" will work better than a standard load balancer using
> round robin strategy? Again I dare to doubt, even if you you're faster at
> routing request to proper node, you introduce another layer of latency.
> >>
> >> Of course the priority of this is up to Tristan but I definitely
> wouldn't place it high on todo list. And before even looking at it I would
> recommend taking Netty HTTP Proxy, putting it in the middle between real
> load balancer and Infinispan app and measure performance with and without
> it. Another test could be with 1 and 10 replicas to check the performance
> penalty of hitting 100% and 10% requests into proper node.
> >>
> >> [1]
> https://cloudplatform.googleblog.com/2014/08/containers-vms-kubernetes-and-vmware.html
> >> [2]
> https://cloudplatform.googleblog.com/2014/04/enter-andromeda-zone-google-cloud-platforms-latest-networking-stack.html
> >> [3]
> https://github.com/slaskawi/external-ip-proxy/blob/Benchmark_with_proxy/Proxy/Proxy.go
> >> [4]
> https://github.com/slaskawi/external-ip-proxy/blob/master/benchmark/results%20with%20proxy.txt
> >> [5]
> https://github.com/slaskawi/external-ip-proxy/blob/master/benchmark/results%20with%20loadbalancer.txt
> >>
> >> Not least having a "REST load balancer" would allow to deploy
> Infinispan as an HTTP cache; just honouring the HTTP caching protocols and
> existing standards would allow people to use any client to their liking,
> >>
> >> Could you please give me an example how this could work? The only way
> that I know is to plug a cache into reverse proxy. NGNIX supports pluggable
> Redis for example [6].
> >>
> >> [6] https://www.nginx.com/resources/wiki/modules/redis/
> >>
> >> without us having to maintain Hot Rod clients and support it on many
> exotic platforms - we would still have Hot Rod clients but we'd be able to
> pick a smaller set of strategical platforms (e.g. Windows doesn't have to
> be in that list).
> >>
> >> As I mentioned before, I really doubts HTTP will be faster then Hot Rod
> in any scenario.
> >>
> >> Such a load balancer could be written in Java (recent WildFly versions
> are able to do this efficiently) or it could be written in another
> language, all it takes is to integrate an Hot Rod client - or just the
> intelligence of it- as an extension into an existing load balancer of our
> choice.
> >>
> >> As I mentioned before, with custom load balancer you're introducing
> another layer of latency. It's not a free ride.
> >>
> >> Allow me a bit more nit-picking on your benchmarks ;)
> >> As you pointed out yourself there are several flaws in your setup:
> "didn't tune", "running in a VM", "benchmarked on a mac mini", ...if you
> know it's a flawed setup I'd rather not publish figures, especially not
> suggest to make decisions based on such results.
> >>
> >> Why not? Infinispan is a public project and anyone can benchmark it
> using JMH and taking decisions based on figures is always better than on
> intuition. Even though there were multiple unknown factors involved in this
> benchmark (this is why I pointed them out and asked to take the results
> with a grain of salt), the test conditions for all scenarios were the same.
> For me this is sufficient to give a general recommendation as I did. BTW,
> this recommendation fits exactly my expectations (communication inside Kube
> the fastest, LB per Pod a bit slower and no advanced routing the slowest).
> Finally, the recommendation is based on a POC which by definition means it
> doesn't fit all scenarios. You should always measure your system!
> >>
> >> So unless you can prove the benchmark results are fundamentally wrong
> and I have drawn wrong conclusions (e.g. a simple client is the fastest
> solution whereas inside Kubernetes communication is the slowest), please
> don't use "naaah, that's wrong" argument. It's rude.
> >>
> >> At this level of design need to focus on getting the architecture
> right; it should be self-speaking that your proposal of actually using
> intelligent routing in some way should be better than not using it.
> >>
> >> My benchmark confirmed this. But as always I would be happy to discuss
> some alternatives. But before trying to convince me to "REST Router",
> please prove that introducing a load balancer (or just a simple async proxy
> for start) gives similar or better performance than a simple load balancer
> with round robin strategy.
> >>
> >> Once we'll have an agreement on a sound architecture, then we'll be
> able to make the implementation efficient.
> >>
> >> Thanks,
> >> Sanne
> >>
> >>
> >>
> >>
> >> On 30 May 2017 at 13:43, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
> >> Hey guys!
> >>
> >> Over past few weeks I've been working on accessing Infinispan cluster
> deployed inside Kubernetes from the outside world. The POC diagram looks
> like the following:
> >>
> >> <pasted1.png>
> >>
> >> As a reminder, the easiest (though not the most effective) way to do it
> is to expose a load balancer Service (or a Node Port Service) and access it
> using a client with basic intelligence (so that it doesn't try to update
> server list based on topology information). As you might expect, this won't
> give you much performance but at least you could access the cluster.
> Another approach is to use TLS/SNI but again, the performance would be even
> worse.
> >>
> >> During the research I tried to answer this problem and created
> "External IP Controller" [1] (and corresponding Pull Request for mapping
> internal/external addresses [2]). The main idea is to create a controller
> deployed inside Kubernetes which will create (and destroy if not needed) a
> load balancer per Infinispan Pod. Additionally the controller exposes
> mapping between internal and external addresses which allows the client to
> properly update server list as well as consistent hash information. A full
> working example is located here [3].
> >>
> >> The biggest question is whether it's worth it? The short answer is yes.
> Here are some benchmark results of performing 10k puts and 10k puts&gets
> (please take them with a big grain of salt, I didn't optimize any server
> settings):
> >>       ? Benchmark app deployed inside Kuberenetes and using internal
> addresses (baseline):
> >>               ? 10k puts: 674.244 ?  16.654
> >>               ? 10k puts&gets: 1288.437 ? 136.207
> >>       ? Benchamrking app deployed in a VM outside of Kubernetes with
> basic intelligence:
> >>               ? 10k puts: 1465.567 ? 176.349
> >>               ? 10k puts&gets: 2684.984 ? 114.993
> >>       ? Benchamrking app deployed in a VM outside of Kubernetes with
> address mapping and topology aware hashing:
> >>               ? 10k puts: 1052.891 ?  31.218
> >>               ? 10k puts&gets: 2465.586 ?  85.034
> >> Note that benchmarking Infinispan from a VM might be very misleading
> since it depends on data center configuration. Benchmarks above definitely
> contain some delay between Google Compute Engine VM and a Kubernetes
> cluster deployed in Google Container Engine. How big is the delay? Hard to
> tell. What counts is the difference between client using basic intelligence
> and topology aware intelligence. And as you can see it's not that small.
> >>
> >> So the bottom line - if you can, deploy your application along with
> Infinispan cluster inside Kubernetes. That's the fastest configuration
> since only iptables are involved. Otherwise use a load balancer per pod
> with External IP Controller. If you don't care about performance, just use
> basic client intelligence and expose everything using single load balancer.
> >>
> >> Thanks,
> >> Sebastian
> >>
> >> [1] https://github.com/slaskawi/external-ip-proxy
> >> [2] https://github.com/infinispan/infinispan/pull/5164
> >> [3] https://github.com/slaskawi/external-ip-proxy/tree/master/benchmark
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >> --
> >> SEBASTIAN ?ASKAWIEC
> >> INFINISPAN DEVELOPER
> >> Red Hat EMEA
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170601/57fa9c19/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pasted1
Type: image/png
Size: 25404 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170601/57fa9c19/attachment-0001.png 

From slaskawi at redhat.com  Thu Jun  1 05:51:46 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 01 Jun 2017 09:51:46 +0000
Subject: [infinispan-dev] Netty SSL Context, was [Hot Rod secured by default]
In-Reply-To: <dec3db85-5b52-33bd-e9c7-a5a65cee3edc@redhat.com>
References: <f3cee023-96a1-3c7d-a3ce-b575cd5b0f7e@redhat.com>
	<CAA9R9O7S7frPE-sS60Z2E6jV_+OdL18+dvMDFJwQsd-FQfQt7w@mail.gmail.com>
	<df6c29dc-4d48-1dc7-6045-2a14e0dd01d8@redhat.com>
	<06b88c80-433f-3745-2112-1424bcd9c342@redhat.com>
	<CAKwmrdPrJgy_KPyZNHnK40mGrCqG05DyntvC_3R=TKK2P6-AoQ@mail.gmail.com>
	<CA+nfvwQjSy+oC0gwoEccEMs_E0k0ksD5JsF7rYb2Jdxm3g4mxA@mail.gmail.com>
	<CAA9R9O5eYa2GXuk+bmQcDYt7RVsNnKeJMadXM99+hTMhjGwueA@mail.gmail.com>
	<dec3db85-5b52-33bd-e9c7-a5a65cee3edc@redhat.com>
Message-ID: <CAA9R9O6Fw=hJrVz0VVa2eX3-jqAwvEG51ySs10VygDqMpdHJfw@mail.gmail.com>

I think I've just found the reason why we can not migrate in OpenSSL by
default :(

In server scenario we obtain S*SL*Context (the one from JDK; Netty has
similar S*sl*Context) from WildFly. It is already configured along with
sercurity realms, domains etc. We then get into this branch of code [1].

In order to do fancy things like SNI we need to remap JDK's SSLContext into
Netty's SslContext and the only implementation that can consume SSLContext
we have at hand is JdkSslContext.

I honestly have no idea how we could refactor this... And that's a shame
because OpenSSL is way faster...

[1]
https://github.com/infinispan/infinispan/blob/830b5314e5d6763aff239584026a7a9d15f01666/server/core/src/main/java/org/infinispan/server/core/utils/SslUtils.java#L29-L29

On Fri, Mar 31, 2017 at 6:02 PM Tristan Tarrant <ttarrant at redhat.com> wrote:

> You want to use OpenSSL with Netty:
>
> http://netty.io/wiki/requirements-for-4.x.html#wiki-h4-4
>
> Tristan
>
> On 31/03/2017 15:55, Sebastian Laskawiec wrote:
> > Unfortunately TLS still slows down stuff (a lot). When I was doing tests
> > for the multi-tenancy router (which is based on TLS/SNI), my average
> > results were like this:
> >
> > Use-caseTypeAvgError
> > initConnectionAndPerform10KPutsSingleServerNoSsl1034.81714.424
> > initConnectionAndPerform10KPutsSingleServerWithSsl1567.55324.872
> > initConnectionAndPerform10KPutsTwoServersWithSslSni1563.22934.05
> > initConnectionOnlySingleServerNoSsl*3.389*0.198
> > initConnectionOnlySingleServerWithSsl*14.086*0.794
> > initConnectionOnlyTwoServersWithSslSni*14.722*0.684
> > perform10KPutsSingleServerNoSsl*4.602*0.585
> > perform10KPutsSingleServerWithSsl*16.583*0.198
> > perform10KPutsTwoServersWithSslSni*17.02*0.794
> >
> > This is nothing new, but initializing Hot Rod connection took was ~4
> > times slower and putting 10K random strings (UUIDs) was also ~4 times
> > slower. But what's worth to mention, there is no significant difference
> > between TLS and TLS+SNI.
> >
> > As far as I know, it is possible to install specialized hardware to deal
> > with encryption in data centers. It is called SSL Acceleration [1].
> > However I'm not aware of any special processor instructions that can
> > help you with that. But the implementations are getting better and
> > better, so who knows...
> >
> > But getting back to the original question, I think the problem we are
> > trying to solve (correct me if I'm wrong) is to prevent unauthorized
> > folks to put their hands on a victims data (either pushing something
> > malicious/corrupted to the cache or obtaining something from the cache).
> > Another problem is transmission security - encryption. If we want our
> > new devs to be secured out of the box, I think we should do both - use
> > TLS (without trusting all certificated) and authentication. This makes
> > Infinispan harder to use of course. So the other extremum is to turn
> > both things off.
> >
> > I voted for the latter, making Infinispan super easy to use. But you
> > guys convinced me that we should care about the security in this case
> > too, so I would use PLAIN authentication + TLS. I would also love to see
> > one magic switch, for example `./bin/standalone.sh --dev-mode`, which
> > would turn all security off.
> >
> > Thanks,
> > Sebastian
> >
> > [1] https://en.wikipedia.org/wiki/SSL_acceleration
> >
> >
> > On Thu, Mar 30, 2017 at 9:22 PM Dan Berindei <dan.berindei at gmail.com
> > <mailto:dan.berindei at gmail.com>> wrote:
> >
> >     I agree with Radim, PLAIN authentication without encryption makes it
> >     too easy to sniff the password from another machine.
> >
> >     I have no idea how expensive SSL encryption is in WildFly, but I
> think
> >     all recent processors have specialized instructions for helping with
> >     encryption, so it may not be that bad.
> >
> >     Even with encryption, if the client trusts all certs, it may be
> >     possible for an attacker to insert itself in the middle and decode
> >     everything -- depending on network topology and what kind of access
> >     the attacker already has. I think it only makes sense to trust all
> >     certs if we also implement something like HPKP [1], to make it more
> >     like ssh.
> >
> >     [1]: https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning
> >
> >     Cheers
> >     Dan
> >
> >
> >
> >     On Thu, Mar 30, 2017 at 7:07 PM, Wolf Fink <wfink at redhat.com
> >     <mailto:wfink at redhat.com>> wrote:
> >      > +1 to make the default secure.
> >      >
> >      > -1 SSL by default as it makes it slower and I think not most will
> >     use it
> >      >
> >      > -1 easy trust all certs, That sounds to me we close one door and
> >     make it
> >      > possible to open another one
> >      >
> >      >
> >      > What if we add an example configuration unsecured which can be
> >     simple copied
> >      > for examples and to start.
> >      >
> >      >
> >      > On Thu, Mar 30, 2017 at 5:31 PM, Dennis Reed <dereed at redhat.com
> >     <mailto:dereed at redhat.com>> wrote:
> >      >>
> >      >> +1 to authentication and encryption by default.
> >      >>   This is 2017, that's how *everything* should be configured.
> >      >>
> >      >> -1 to making it easy to trust all certs.  That negates the point
> of
> >      >> using encryption in the first place and should really never be
> done.
> >      >>
> >      >> If it's too hard to configure the correct way that we think it
> would
> >      >> turn users away, that's a usability problem that needs to be
> fixed.
> >      >>
> >      >> -Dennis
> >      >>
> >      >>
> >      >> On 03/30/2017 09:29 AM, Tristan Tarrant wrote:
> >      >> > While the "unsecure" over loopback is quite tempting, I would
> >     prefer to
> >      >> > have homogeneous behaviour with the possibility to disable
> >     security
> >      >> > altogether for quick demos.
> >      >> > Otherwise a developer would need to code differently for the
> >     local use
> >      >> > case than for the remote one, causing more confusion.
> >      >> >
> >      >> > Tristan
> >      >> >
> >      >> > On 30/03/2017 14:54, Sebastian Laskawiec wrote:
> >      >> >> I agree the security out of the box is good. But at the same
> >     time we
> >      >> >> don't want to make Infinispan harder to use for new
> >     developers. Out of
> >      >> >> the box configuration should be "good enough" to start
> hacking.
> >      >> >>
> >      >> >> I would propose to make all the endpoints unprotected (with
> >      >> >> authentication disabled) on localhost/loopback and protected
> when
> >      >> >> calling from the outside world.
> >      >> >>
> >      >> >> On Thu, Mar 30, 2017 at 2:39 PM Tristan Tarrant
> >     <ttarrant at redhat.com <mailto:ttarrant at redhat.com>
> >      >> >> <mailto:ttarrant at redhat.com <mailto:ttarrant at redhat.com>>>
> wrote:
> >      >> >>
> >      >> >>     Dear all,
> >      >> >>
> >      >> >>     after a mini chat on IRC, I wanted to bring this to
> >     everybody's
> >      >> >>     attention.
> >      >> >>
> >      >> >>     We should make the Hot Rod endpoint require
> >     authentication in the
> >      >> >>     out-of-the-box configuration.
> >      >> >>     The proposal is to enable the PLAIN (or, preferably,
> >     DIGEST) SASL
> >      >> >>     mechanism against the ApplicationRealm and require users
> >     to run the
> >      >> >>     add-user script.
> >      >> >>     This would achieve two goals:
> >      >> >>     - secure out-of-the-box configuration, which is always a
> >     good idea
> >      >> >>     - access to the "protected" schema and script caches
> which is
> >      >> >> prevented
> >      >> >>     when not on loopback on non-authenticated endpoints.
> >      >> >>
> >      >> >>     Tristan
> >      >> >>     --
> >      >> >>     Tristan Tarrant
> >      >> >>     Infinispan Lead
> >      >> >>     JBoss, a division of Red Hat
> >      >> >>     _______________________________________________
> >      >> >>     infinispan-dev mailing list
> >      >> >> infinispan-dev at lists.jboss.org
> >     <mailto:infinispan-dev at lists.jboss.org>
> >      >> >> <mailto:infinispan-dev at lists.jboss.org
> >     <mailto:infinispan-dev at lists.jboss.org>>
> >      >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >      >> >>
> >      >> >>
> >      >> >>
> >      >> >> _______________________________________________
> >      >> >> infinispan-dev mailing list
> >      >> >> infinispan-dev at lists.jboss.org
> >     <mailto:infinispan-dev at lists.jboss.org>
> >      >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >      >> >>
> >      >> >
> >      >> _______________________________________________
> >      >> infinispan-dev mailing list
> >      >> infinispan-dev at lists.jboss.org
> >     <mailto:infinispan-dev at lists.jboss.org>
> >      >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >      >
> >      >
> >      >
> >      > _______________________________________________
> >      > infinispan-dev mailing list
> >      > infinispan-dev at lists.jboss.org
> >     <mailto:infinispan-dev at lists.jboss.org>
> >      > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >     _______________________________________________
> >     infinispan-dev mailing list
> >     infinispan-dev at lists.jboss.org <mailto:
> infinispan-dev at lists.jboss.org>
> >     https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170601/95f0c0ed/attachment-0001.html 

From dan.berindei at gmail.com  Thu Jun  1 06:12:48 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 1 Jun 2017 13:12:48 +0300
Subject: [infinispan-dev] Allocation costs of
	TypeConverterDelegatingAdvancedCache
In-Reply-To: <CA+YCuUNbt2_2-NT2f9ehc=4Uo-fqJ1TAHbne6gkcVCa8PD4=0g@mail.gmail.com>
References: <CAFm4XO1gAHeacGXQ-=8kfWK+zCCZ=k_33DMQ+LBYXt2cMS3d8A@mail.gmail.com>
	<CA+YCuUNbt2_2-NT2f9ehc=4Uo-fqJ1TAHbne6gkcVCa8PD4=0g@mail.gmail.com>
Message-ID: <CA+nfvwS-6HDtN=0eOgFzsh7nBnMMZVvu7W+VFh6UAfaAnQEg4Q@mail.gmail.com>

On Wed, May 31, 2017 at 10:05 PM, William Burns <mudokonman at gmail.com> wrote:
> Let me explain why it is there first :) This class was added for two main
> reasons: as a replacement for compatibility and for supporting equality of
> byte[] object. What this class does is at the user side is box the given
> arguments (eg. byte[] -> WrappedByteArray) then the cache only ever deals
> with the boxed types and then does the unboxing for any values that are
> returned.
>
> There are some exceptions with how the boxing/unboxing works for both cases
> such as Streams and Indexing which have to rebox the data to work properly.
> But the cost is pretty minimal.
>
> While compatibility is always on or off unfortunately anyone can pass in a
> byte[] at any point for a key or value. So we need to have these wrappers
> there to make sure they work properly.
>
> We could add a option to the cache, which people didn't show interest
> before, to have a cache that doesn't support byte[] or compatibility. In
> this case there would be no need for the wrapper.
>
>
> Compatibility
>
> By using the wrapper around the cache, compatibility becomes quite trivial
> since we just need a converter in the wrapper and it does everything else
> for us. My guess is the new encoding changes will utilize these wrapper
> classes as well as they are quite easy to plug in and have things just work.
>
> Equality
>
> With the rewrite for eviction we lost the ability to use custom Equality in
> the data container. The only option for that is to wrap a byte[] to provide
> our own equality. Therefore the wrapper does this conversion for us by
> converting between automatically.
>
> On Wed, May 31, 2017 at 2:34 PM Sanne Grinovero <sanne at infinispan.org>
> wrote:
>>
>> Hi all,
>>
>> I've been running some benchmarks and for the fist time playing with
>> Infinispan 9+, so please bear with me as I might shoot some dumb
>> questions to the list in the following days.
>>
>> The need for TypeConverterDelegatingAdvancedCache to wrap most
>> operations - especially "convertKeys" - is highlighet as one of the
>
>
> It should be wrapping every operation pretty much.
>
> Unfortunately the methods this hurts the most are putAll, getAll etc as they
> have to not only box every entry but copy them into a new collection as you
> saw in "convertKeys". And for getAll it also has to unbox the return as
> well.
>
> We could reduce allocations in the collection methods by not creating the
> new collection until we run into one key or value that required
> boxing/unboxing. This would still require fully iterating over the
> collection at best case. This should perform well in majority of cases as I
> would expect all or almost all entries either require or don't require the
> boxing. The cases that would be harmed most would be ones that have a sparse
> number of entries that require boxing.
>

+1 for checking the keys/values first and only creating a new
collection if necessary, although it doesn't seem possible with the
current TypeConverter interface.

I'm also fine with adding a configuration option to prohibit byte[]
keys/values, but I'd still want to check the collection and throw an
exception (at least for writes).

Thinking further, do we really need to wrap byte[] values?
DataContainer only has a compute() method, so we may be able to
replace the value wrappers with a TypeConverter.valuesEqual(a, b)
method.

Also, Sanne, could you change InternalCacheFactory.createAndWire() to
return the un-wrapped cache, and then see exactly much the wrapping
affects performance? With so many moving parts, I wouldn't be
surprised if the difference in numbers was < 1%.

>>
>> high allocators in my Search-centric use case.
>>
>> I'm wondering:
>>  A - Could this implementation be improved?
>
>
> Most anything can be improved :) The best way would be to add another knob.
>
>>  B - Could I bypass / disable it? Not sure why it's there.
>
>
> There is no way to bypass currently. Explained above.
>
>
>>
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From gustavo at infinispan.org  Thu Jun  1 07:17:47 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Thu, 1 Jun 2017 12:17:47 +0100
Subject: [infinispan-dev] Netty SSL Context,
	was [Hot Rod secured by default]
In-Reply-To: <CAA9R9O6Fw=hJrVz0VVa2eX3-jqAwvEG51ySs10VygDqMpdHJfw@mail.gmail.com>
References: <f3cee023-96a1-3c7d-a3ce-b575cd5b0f7e@redhat.com>
	<CAA9R9O7S7frPE-sS60Z2E6jV_+OdL18+dvMDFJwQsd-FQfQt7w@mail.gmail.com>
	<df6c29dc-4d48-1dc7-6045-2a14e0dd01d8@redhat.com>
	<06b88c80-433f-3745-2112-1424bcd9c342@redhat.com>
	<CAKwmrdPrJgy_KPyZNHnK40mGrCqG05DyntvC_3R=TKK2P6-AoQ@mail.gmail.com>
	<CA+nfvwQjSy+oC0gwoEccEMs_E0k0ksD5JsF7rYb2Jdxm3g4mxA@mail.gmail.com>
	<CAA9R9O5eYa2GXuk+bmQcDYt7RVsNnKeJMadXM99+hTMhjGwueA@mail.gmail.com>
	<dec3db85-5b52-33bd-e9c7-a5a65cee3edc@redhat.com>
	<CAA9R9O6Fw=hJrVz0VVa2eX3-jqAwvEG51ySs10VygDqMpdHJfw@mail.gmail.com>
Message-ID: <CAH8Ud1HLmLht1eZ-M7U-UWHEgpeiiM_OdraKrKwbcpqg8K+g6Q@mail.gmail.com>

On Thu, Jun 1, 2017 at 10:51 AM, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> I think I've just found the reason why we can not migrate in OpenSSL by
> default :(
>
> In server scenario we obtain S*SL*Context (the one from JDK; Netty has
> similar S*sl*Context) from WildFly. It is already configured along with
> sercurity realms, domains etc. We then get into this branch of code [1].
>
> In order to do fancy things like SNI we need to remap JDK's SSLContext
> into Netty's SslContext and the only implementation that can consume
> SSLContext we have at hand is JdkSslContext.
>
> I honestly have no idea how we could refactor this... And that's a shame
> because OpenSSL is way faster...
>


I tried migrating the SSL engine to Netty's in [1] and hit the same wall.
What I was told is that the SSLContext in Wildfly is now (version 11?) a
capability under 'org.wildfly.security.ssl-context'  and
can be replaced, but I did not try doing that.


[1] https://issues.jboss.org/browse/ISPN-6990

Gustavo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170601/3c911072/attachment.html 

From sanne at infinispan.org  Thu Jun  1 07:43:56 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 1 Jun 2017 12:43:56 +0100
Subject: [infinispan-dev] Allocation costs of
	TypeConverterDelegatingAdvancedCache
In-Reply-To: <CA+nfvwS-6HDtN=0eOgFzsh7nBnMMZVvu7W+VFh6UAfaAnQEg4Q@mail.gmail.com>
References: <CAFm4XO1gAHeacGXQ-=8kfWK+zCCZ=k_33DMQ+LBYXt2cMS3d8A@mail.gmail.com>
	<CA+YCuUNbt2_2-NT2f9ehc=4Uo-fqJ1TAHbne6gkcVCa8PD4=0g@mail.gmail.com>
	<CA+nfvwS-6HDtN=0eOgFzsh7nBnMMZVvu7W+VFh6UAfaAnQEg4Q@mail.gmail.com>
Message-ID: <CAFm4XO3K_nJZ9NCNJegpV_S6aww6J9=u7F-duReDCwE0VoRyww@mail.gmail.com>

Thanks all for the great explanations.

+1 to try to avoid this. Could the transformation be performed lazily?
We had a design meeting about transformers and that was a goal we had
agreed on.

For example in the Search case, I'm doing a getAll( keys ) just to
return the objects in a sorted list back to the user - in whatever
form they are.
In this case the engine doesn't need the objects to be in any specific
form, so the transformation / wrapping phase needs to be moved to the
client endpoint to become an optional, transparent step.

Some more comments inline:

On 1 June 2017 at 11:12, Dan Berindei <dan.berindei at gmail.com> wrote:
> On Wed, May 31, 2017 at 10:05 PM, William Burns <mudokonman at gmail.com> wrote:
>> Let me explain why it is there first :) This class was added for two main
>> reasons: as a replacement for compatibility and for supporting equality of
>> byte[] object. What this class does is at the user side is box the given
>> arguments (eg. byte[] -> WrappedByteArray) then the cache only ever deals
>> with the boxed types and then does the unboxing for any values that are
>> returned.
>>
>> There are some exceptions with how the boxing/unboxing works for both cases
>> such as Streams and Indexing which have to rebox the data to work properly.
>> But the cost is pretty minimal.
>>
>> While compatibility is always on or off unfortunately anyone can pass in a
>> byte[] at any point for a key or value. So we need to have these wrappers
>> there to make sure they work properly.
>>
>> We could add a option to the cache, which people didn't show interest
>> before, to have a cache that doesn't support byte[] or compatibility. In
>> this case there would be no need for the wrapper.

Let's try to avoid a configuration option; if the Transformers design
I mentioned above doesn't fly we could have Infinispan Query use a
dedicated internal API which skips such operations.

>>
>>
>> Compatibility
>>
>> By using the wrapper around the cache, compatibility becomes quite trivial
>> since we just need a converter in the wrapper and it does everything else
>> for us. My guess is the new encoding changes will utilise these wrapper
>> classes as well as they are quite easy to plug in and have things just work.

Remember there's more than byte[] vs Pojo, as we also have other
formats which need supporting, and plans to support JSON as well
(which is not a String nor a POJO).

The query engine will likely need to convert / interpret the "storage
format" into its own format, or like Adrian pointed out being a great
capability of Protobuf is to read only a selection of fields.

So I wouldn't automatically apply these operations, as they must be
client dependent. Tristan has the design documents from the Infinispan
Query meeting we held in London, I briefly pointed Will to them during
our Konstanz meeting. May I hope that's going to be the next step?
Sounds like it would also avoid this performance issue.


>> Equality
>>
>> With the rewrite for eviction we lost the ability to use custom Equality in
>> the data container. The only option for that is to wrap a byte[] to provide
>> our own equality. Therefore the wrapper does this conversion for us by
>> converting between automatically.
>>
>> On Wed, May 31, 2017 at 2:34 PM Sanne Grinovero <sanne at infinispan.org>
>> wrote:
>>>
>>> Hi all,
>>>
>>> I've been running some benchmarks and for the fist time playing with
>>> Infinispan 9+, so please bear with me as I might shoot some dumb
>>> questions to the list in the following days.
>>>
>>> The need for TypeConverterDelegatingAdvancedCache to wrap most
>>> operations - especially "convertKeys" - is highlighet as one of the
>>
>>
>> It should be wrapping every operation pretty much.
>>
>> Unfortunately the methods this hurts the most are putAll, getAll etc as they
>> have to not only box every entry but copy them into a new collection as you
>> saw in "convertKeys". And for getAll it also has to unbox the return as
>> well.
>>
>> We could reduce allocations in the collection methods by not creating the
>> new collection until we run into one key or value that required
>> boxing/unboxing. This would still require fully iterating over the
>> collection at best case. This should perform well in majority of cases as I
>> would expect all or almost all entries either require or don't require the
>> boxing. The cases that would be harmed most would be ones that have a sparse
>> number of entries that require boxing.
>>
>
> +1 for checking the keys/values first and only creating a new
> collection if necessary, although it doesn't seem possible with the
> current TypeConverter interface.

+1

> I'm also fine with adding a configuration option to prohibit byte[]
> keys/values, but I'd still want to check the collection and throw an
> exception (at least for writes).
>
> Thinking further, do we really need to wrap byte[] values?
> DataContainer only has a compute() method, so we may be able to
> replace the value wrappers with a TypeConverter.valuesEqual(a, b)
> method.
>
> Also, Sanne, could you change InternalCacheFactory.createAndWire() to
> return the un-wrapped cache, and then see exactly much the wrapping
> affects performance? With so many moving parts, I wouldn't be
> surprised if the difference in numbers was < 1%.

Indeed, I don't expect a significant change in throughput: it's an
allocation issue and as such it's very situational if it's going to
affect someone or not. Not worth measuring as my current bottleneck is
caused by other open issues.

But my main concern now is that apparently this approach does not
address some of our needs as previously agreed, which we need to
improve usability such as transparent transcoding. I guess that's
coming next? In that case we can drop it for now, it's no big deal and
I was just hoping to have identified a low hanging fruit.

Thanks,
Sanne

>
>>>
>>> high allocators in my Search-centric use case.
>>>
>>> I'm wondering:
>>>  A - Could this implementation be improved?
>>
>>
>> Most anything can be improved :) The best way would be to add another knob.
>>
>>>  B - Could I bypass / disable it? Not sure why it's there.
>>
>>
>> There is no way to bypass currently. Explained above.
>>
>>
>>>
>>>
>>> Thanks,
>>> Sanne
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From mudokonman at gmail.com  Thu Jun  1 11:14:20 2017
From: mudokonman at gmail.com (William Burns)
Date: Thu, 01 Jun 2017 15:14:20 +0000
Subject: [infinispan-dev] Allocation costs of
	TypeConverterDelegatingAdvancedCache
In-Reply-To: <CAFm4XO3K_nJZ9NCNJegpV_S6aww6J9=u7F-duReDCwE0VoRyww@mail.gmail.com>
References: <CAFm4XO1gAHeacGXQ-=8kfWK+zCCZ=k_33DMQ+LBYXt2cMS3d8A@mail.gmail.com>
	<CA+YCuUNbt2_2-NT2f9ehc=4Uo-fqJ1TAHbne6gkcVCa8PD4=0g@mail.gmail.com>
	<CA+nfvwS-6HDtN=0eOgFzsh7nBnMMZVvu7W+VFh6UAfaAnQEg4Q@mail.gmail.com>
	<CAFm4XO3K_nJZ9NCNJegpV_S6aww6J9=u7F-duReDCwE0VoRyww@mail.gmail.com>
Message-ID: <CA+YCuUPz7h2HW405r1XbOR20Sw8vv2oF-R7Xj8kPiChDWch_QQ@mail.gmail.com>

I think we may be confusing two different features. These changes are what
I was hoping could be used as a basis for the transformers work (doesn't
mean they will be). These changes were done before there was any
discussions about transformers though, so they may not have the use cases
covered. I think this might be why there was some confusion.

On Thu, Jun 1, 2017 at 8:11 AM Sanne Grinovero <sanne at infinispan.org> wrote:

> Thanks all for the great explanations.
>
> +1 to try to avoid this. Could the transformation be performed lazily?
>

I can't think of a reason we can't try to avoid the collection allocations
with what I proposed.


> We had a design meeting about transformers and that was a goal we had
> agreed on.


> For example in the Search case, I'm doing a getAll( keys ) just to
> return the objects in a sorted list back to the user - in whatever
> form they are.
> In this case the engine doesn't need the objects to be in any specific
> form, so the transformation / wrapping phase needs to be moved to the
> client endpoint to become an optional, transparent step.
>

These changes have nothing to do with the client itself. In fact these were
mostly added for byte[] requirements. I just added in Compatibility support
which required the same type of code.


>
> Some more comments inline:
>
> On 1 June 2017 at 11:12, Dan Berindei <dan.berindei at gmail.com> wrote:
> > On Wed, May 31, 2017 at 10:05 PM, William Burns <mudokonman at gmail.com>
> wrote:
> >> Let me explain why it is there first :) This class was added for two
> main
> >> reasons: as a replacement for compatibility and for supporting equality
> of
> >> byte[] object. What this class does is at the user side is box the given
> >> arguments (eg. byte[] -> WrappedByteArray) then the cache only ever
> deals
> >> with the boxed types and then does the unboxing for any values that are
> >> returned.
> >>
> >> There are some exceptions with how the boxing/unboxing works for both
> cases
> >> such as Streams and Indexing which have to rebox the data to work
> properly.
> >> But the cost is pretty minimal.
> >>
> >> While compatibility is always on or off unfortunately anyone can pass
> in a
> >> byte[] at any point for a key or value. So we need to have these
> wrappers
> >> there to make sure they work properly.
> >>
> >> We could add a option to the cache, which people didn't show interest
> >> before, to have a cache that doesn't support byte[] or compatibility. In
> >> this case there would be no need for the wrapper.
>
> Let's try to avoid a configuration option; if the Transformers design
> I mentioned above doesn't fly we could have Infinispan Query use a
> dedicated internal API which skips such operations.
>
> >>
> >>
> >> Compatibility
> >>
> >> By using the wrapper around the cache, compatibility becomes quite
> trivial
> >> since we just need a converter in the wrapper and it does everything
> else
> >> for us. My guess is the new encoding changes will utilise these wrapper
> >> classes as well as they are quite easy to plug in and have things just
> work.
>
> Remember there's more than byte[] vs Pojo, as we also have other
> formats which need supporting, and plans to support JSON as well
> (which is not a String nor a POJO).
>
> The query engine will likely need to convert / interpret the "storage
> format" into its own format, or like Adrian pointed out being a great
> capability of Protobuf is to read only a selection of fields.
>
> So I wouldn't automatically apply these operations, as they must be
> client dependent. Tristan has the design documents from the Infinispan
> Query meeting we held in London, I briefly pointed Will to them during
> our Konstanz meeting. May I hope that's going to be the next step?


This is what Gustavo and others are working on.


>

Sounds like it would also avoid this performance issue.
>

I can't say, but the byte[] is another issue, since it is needed outside of
client/server.


>
>
> >> Equality
> >>
> >> With the rewrite for eviction we lost the ability to use custom
> Equality in
> >> the data container. The only option for that is to wrap a byte[] to
> provide
> >> our own equality. Therefore the wrapper does this conversion for us by
> >> converting between automatically.
> >>
> >> On Wed, May 31, 2017 at 2:34 PM Sanne Grinovero <sanne at infinispan.org>
> >> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> I've been running some benchmarks and for the fist time playing with
> >>> Infinispan 9+, so please bear with me as I might shoot some dumb
> >>> questions to the list in the following days.
> >>>
> >>> The need for TypeConverterDelegatingAdvancedCache to wrap most
> >>> operations - especially "convertKeys" - is highlighet as one of the
> >>
> >>
> >> It should be wrapping every operation pretty much.
> >>
> >> Unfortunately the methods this hurts the most are putAll, getAll etc as
> they
> >> have to not only box every entry but copy them into a new collection as
> you
> >> saw in "convertKeys". And for getAll it also has to unbox the return as
> >> well.
> >>
> >> We could reduce allocations in the collection methods by not creating
> the
> >> new collection until we run into one key or value that required
> >> boxing/unboxing. This would still require fully iterating over the
> >> collection at best case. This should perform well in majority of cases
> as I
> >> would expect all or almost all entries either require or don't require
> the
> >> boxing. The cases that would be harmed most would be ones that have a
> sparse
> >> number of entries that require boxing.
> >>
> >
> > +1 for checking the keys/values first and only creating a new
> > collection if necessary, although it doesn't seem possible with the
> > current TypeConverter interface.
>
> +1
>
> > I'm also fine with adding a configuration option to prohibit byte[]
> > keys/values, but I'd still want to check the collection and throw an
> > exception (at least for writes).
> >
> > Thinking further, do we really need to wrap byte[] values?
> > DataContainer only has a compute() method, so we may be able to
> > replace the value wrappers with a TypeConverter.valuesEqual(a, b)
> > method.
> >
> > Also, Sanne, could you change InternalCacheFactory.createAndWire() to
> > return the un-wrapped cache, and then see exactly much the wrapping
> > affects performance? With so many moving parts, I wouldn't be
> > surprised if the difference in numbers was < 1%.
>
> Indeed, I don't expect a significant change in throughput: it's an
> allocation issue and as such it's very situational if it's going to
> affect someone or not. Not worth measuring as my current bottleneck is
> caused by other open issues.
>
> But my main concern now is that apparently this approach does not
> address some of our needs as previously agreed, which we need to
> improve usability such as transparent transcoding. I guess that's
> coming next? In that case we can drop it for now, it's no big deal and
> I was just hoping to have identified a low hanging fruit.
>

Yeah I am looking forward to the new changes as well :)


>
> Thanks,
> Sanne
>
> >
> >>>
> >>> high allocators in my Search-centric use case.
> >>>
> >>> I'm wondering:
> >>>  A - Could this implementation be improved?
> >>
> >>
> >> Most anything can be improved :) The best way would be to add another
> knob.
> >>
> >>>  B - Could I bypass / disable it? Not sure why it's there.
> >>
> >>
> >> There is no way to bypass currently. Explained above.
> >>
> >>
> >>>
> >>>
> >>> Thanks,
> >>> Sanne
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170601/e0ea4c9c/attachment-0001.html 

From mudokonman at gmail.com  Thu Jun  1 11:16:23 2017
From: mudokonman at gmail.com (William Burns)
Date: Thu, 01 Jun 2017 15:16:23 +0000
Subject: [infinispan-dev] Allocation costs of
	TypeConverterDelegatingAdvancedCache
In-Reply-To: <CA+nfvwS-6HDtN=0eOgFzsh7nBnMMZVvu7W+VFh6UAfaAnQEg4Q@mail.gmail.com>
References: <CAFm4XO1gAHeacGXQ-=8kfWK+zCCZ=k_33DMQ+LBYXt2cMS3d8A@mail.gmail.com>
	<CA+YCuUNbt2_2-NT2f9ehc=4Uo-fqJ1TAHbne6gkcVCa8PD4=0g@mail.gmail.com>
	<CA+nfvwS-6HDtN=0eOgFzsh7nBnMMZVvu7W+VFh6UAfaAnQEg4Q@mail.gmail.com>
Message-ID: <CA+YCuUMMshbyP-K2F7S7cgedJtRZouSRoNs0snazMJOwSM6N4A@mail.gmail.com>

On Thu, Jun 1, 2017 at 7:11 AM Dan Berindei <dan.berindei at gmail.com> wrote:

> On Wed, May 31, 2017 at 10:05 PM, William Burns <mudokonman at gmail.com>
> wrote:
> > Let me explain why it is there first :) This class was added for two main
> > reasons: as a replacement for compatibility and for supporting equality
> of
> > byte[] object. What this class does is at the user side is box the given
> > arguments (eg. byte[] -> WrappedByteArray) then the cache only ever deals
> > with the boxed types and then does the unboxing for any values that are
> > returned.
> >
> > There are some exceptions with how the boxing/unboxing works for both
> cases
> > such as Streams and Indexing which have to rebox the data to work
> properly.
> > But the cost is pretty minimal.
> >
> > While compatibility is always on or off unfortunately anyone can pass in
> a
> > byte[] at any point for a key or value. So we need to have these wrappers
> > there to make sure they work properly.
> >
> > We could add a option to the cache, which people didn't show interest
> > before, to have a cache that doesn't support byte[] or compatibility. In
> > this case there would be no need for the wrapper.
> >
> >
> > Compatibility
> >
> > By using the wrapper around the cache, compatibility becomes quite
> trivial
> > since we just need a converter in the wrapper and it does everything else
> > for us. My guess is the new encoding changes will utilize these wrapper
> > classes as well as they are quite easy to plug in and have things just
> work.
> >
> > Equality
> >
> > With the rewrite for eviction we lost the ability to use custom Equality
> in
> > the data container. The only option for that is to wrap a byte[] to
> provide
> > our own equality. Therefore the wrapper does this conversion for us by
> > converting between automatically.
> >
> > On Wed, May 31, 2017 at 2:34 PM Sanne Grinovero <sanne at infinispan.org>
> > wrote:
> >>
> >> Hi all,
> >>
> >> I've been running some benchmarks and for the fist time playing with
> >> Infinispan 9+, so please bear with me as I might shoot some dumb
> >> questions to the list in the following days.
> >>
> >> The need for TypeConverterDelegatingAdvancedCache to wrap most
> >> operations - especially "convertKeys" - is highlighet as one of the
> >
> >
> > It should be wrapping every operation pretty much.
> >
> > Unfortunately the methods this hurts the most are putAll, getAll etc as
> they
> > have to not only box every entry but copy them into a new collection as
> you
> > saw in "convertKeys". And for getAll it also has to unbox the return as
> > well.
> >
> > We could reduce allocations in the collection methods by not creating the
> > new collection until we run into one key or value that required
> > boxing/unboxing. This would still require fully iterating over the
> > collection at best case. This should perform well in majority of cases
> as I
> > would expect all or almost all entries either require or don't require
> the
> > boxing. The cases that would be harmed most would be ones that have a
> sparse
> > number of entries that require boxing.
> >
>
> +1 for checking the keys/values first and only creating a new
> collection if necessary, although it doesn't seem possible with the
> current TypeConverter interface.
>

What I have done in the past is that the TypeConverter just returns the
object as is if it didn't change anything. This way you can do instance
equality to verify if there were changes.


>
> I'm also fine with adding a configuration option to prohibit byte[]
> keys/values, but I'd still want to check the collection and throw an
> exception (at least for writes).
>
> Thinking further, do we really need to wrap byte[] values?
> DataContainer only has a compute() method, so we may be able to
> replace the value wrappers with a TypeConverter.valuesEqual(a, b)
> method.
>

I don't think we can work around it. The key also has to have the same
hashCode and equals and we would then have to wrap it elsewhere to have it
work properly with the underlying ConcurrentHashMap.


>
> Also, Sanne, could you change InternalCacheFactory.createAndWire() to
> return the un-wrapped cache, and then see exactly much the wrapping
> affects performance? With so many moving parts, I wouldn't be
> surprised if the difference in numbers was < 1%.
>
> >>
> >> high allocators in my Search-centric use case.
> >>
> >> I'm wondering:
> >>  A - Could this implementation be improved?
> >
> >
> > Most anything can be improved :) The best way would be to add another
> knob.
> >
> >>  B - Could I bypass / disable it? Not sure why it's there.
> >
> >
> > There is no way to bypass currently. Explained above.
> >
> >
> >>
> >>
> >> Thanks,
> >> Sanne
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170601/14a97bc5/attachment.html 

From gustavo at infinispan.org  Fri Jun  2 05:55:15 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Fri, 2 Jun 2017 10:55:15 +0100
Subject: [infinispan-dev] Jenkins migration
In-Reply-To: <CAA9R9O6W50bfcEGC=A2qkrcqXgCe0=yhiZ7PAhWOZwZqRWgbfg@mail.gmail.com>
References: <CAA9R9O7pUiDW2n+oUBHuMuOJu-pf3-WYFkxo0t=-eioY9vFKxA@mail.gmail.com>
	<CA+nfvwSr=aSbmmVZ0-ii0q290i3uWdMbskaC5y2ess=2zAEAVg@mail.gmail.com>
	<1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com>
	<c532b428-d6f5-b030-352f-a51f7286baf2@redhat.com>
	<CAA9R9O7geBAaKSwMe_NaO+dG8YgToW4++eD_8r-KFh-x2RrNAg@mail.gmail.com>
	<CA+nfvwS0o4q2OujeXZRMfEQRzbfOTvOsJ+SrjAMMNWv5Axo2=w@mail.gmail.com>
	<CAA9R9O6W50bfcEGC=A2qkrcqXgCe0=yhiZ7PAhWOZwZqRWgbfg@mail.gmail.com>
Message-ID: <CAH8Ud1F_caO2WS0SbprLLmO3e5RsYGDD1v9EC57=x74vQ6z57Q@mail.gmail.com>

The test reporting seems broken: in the last master build [1] Jenkins
reported 3 failures, but the console output shows:

       Tests run: 477, Failures: 0, Errors: 340, Skipped: 0

For the JCache TCK build.

Not sure if it's the report that is not generated correctly or the plugin
who publishes the test is not doing its job.
I could not reproduce locally since for me the build hangs indefinitely.

[1] http://ci.infinispan.org/job/Infinispan/job/master/146/

Gustavo


On Wed, Apr 26, 2017 at 11:17 AM, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> Final update (I hope so)...
>
> Phew, it was a busy time.... http://ci.infinispan.org
> <http://ci.infinispan.orgf> now points to Jenkins and all agents are up
> and running.
>
> Since all links on Pull Requests were broken (they were pointing to old
> Jenkins address), I retriggered all builds. The next things I would like to
> do (but no necessarily in closest future) are:
>
>    1. Add a new slave for "small jobs". It will be a slower machine but
>    will perform all JGroups, C++ etc builds.
>    2. Add some monitoring.
>
> Thanks,
> Sebastian
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170602/28710599/attachment.html 

From rory.odonnell at oracle.com  Fri Jun  2 06:28:28 2017
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Fri, 2 Jun 2017 11:28:28 +0100
Subject: [infinispan-dev] Announcing: Early-Access builds of JDK 9 for
 Alpine Linux/musl at jdk.java.net/9/
Message-ID: <09b33955-7a07-1752-9b15-9bea3495b765@oracle.com>


Hi Galder **

*Announcing: Early-Access builds of JDK 9 for Alpine Linux/musl at  
jdk.java.net/9/ [1]
*

  * As of today there are pre-built Early-Access (EA) JDK binaries for
    Alpine Linux/musl at  jdk.java.net/9/**
      o look for ?Alpine Linux?. [1]
  * The Alpine Linux build is compatible with linux distributions that
    use the musl C library. *[2]*

Feedback is very welcome via the portola-dev mailing list, remembering 
to subscribe to the mailing list first.


*Proposed schedule change for JDK 9 [3]*

    JDK 9 Project continues to work toward the current goal of producing
    an initial Release Candidate build on 22 June.
    This proposal is to adjust the GA date in order to accommodate the
    additional time required to move through the JCP process.
    To be specific, we propose to move the GA date out by eight weeks,
    from 27 July to 21 September. 


Rgds,Rory


[1] http://mail.openjdk.java.net/pipermail/portola-dev/2017-June/000191.html
[2] http://www.musl-libc.org/
[3] http://mail.openjdk.java.net/pipermail/jdk9-dev/2017-May/005864.html

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170602/5dd4defa/attachment.html 

From galder at redhat.com  Fri Jun  2 08:45:21 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 2 Jun 2017 14:45:21 +0200
Subject: [infinispan-dev] Proposal for moving Hibernate 2l provider to
	Infinispan
In-Reply-To: <CAFdSV9tKNpCMyK2eL-vTMnAJOaoFYzARazcs5OUxAeLzhOZ_zw@mail.gmail.com>
References: <46A375F3-84FB-46FD-8523-7F3340B4DCA5@redhat.com>
	<CAFdSV9tKNpCMyK2eL-vTMnAJOaoFYzARazcs5OUxAeLzhOZ_zw@mail.gmail.com>
Message-ID: <D3193D97-234D-4FFB-9822-E45A9F13E0BC@redhat.com>

I think it's going through, we've approved you in the past.

Replies below:

> On 31 May 2017, at 17:02, Steve Ebersole <steve at hibernate.org> wrote:
> 
> Just a heads up - FWIW I doubt my reply goes through to the entire infinispan-dev list.
> 
> Replies inline...
> 
> 
> 3. What should be the artifact name? Should it be 'hibernate-infinispan' like it is today? The difference with the existing cache provider would be the groupId. Or some other artifact id?
> 
> Since you use Maven (IIUC) you could just publish a relocation...

Oh, didn't know about that. Yeah, I think we'd do that:
https://maven.apache.org/guides/mini/guide-relocation.html

>  
> 
> 4. Should the main artifact contain the hibernate major version it belongs to? E.g. assuming we take 'hibernate-infinispan', should it be like that, or should it instead be 'hibernate5-infinispan'? This is where it'd be interesting to hear about our past Lucene directory or Query integration experience.
> 
> Probably, but (no promises) one thing I wanted to look at since Hibernate baselines on Java 8, is to maintain the existing SPI using default methods as a bridge.  But failing that, I think your suggestion is the best option.
> 
>  
> 5. A thing to consider also is whether to maintain same package naming. We're currently using 'org.hibernate.cache.infinispan.*'. From a compatibility sense, it'd help to keep same package since users reference region factory fully qualified class names. We'd also continue to be sole owners of 'org.hibernate.cache.infinispan.*'. However, I dunno whether having 'org.hibernate...' package name within Infinispan repo would create other issues?
> 
> FWIW Hibernate offers "short naming" or "friendly naming" for many configurable settings, cache providers being one.  For hibernate-infinispan we register 2: "infinispan" and "infinispan-jndi".  You can see this in org.hibernate.cache.infinispan.StrategyRegistrationProviderImpl.  That approach will continue to work when you move it.  The point being that users do not specify the class name in config, they'd just specify "infinispan", "infinispan-jndi", etc.

Ah good to know, I wasn't aware of it. I'll look into that.

> 6. Testing wise, the cache provider is currently tested one test at the time, using JUnit. The testsuite already runs fast enough and I'd prefer not to change anything in this area right now. Is that Ok? Or is there any desire to move it to TestNG?
> 
> Hmmm, that is actually surprising... I thought the hibernate-infinispan  provider tests were still disabled as they had routinely caused intermittent failures of the build.  I guess this was rectified?

They seem pretty stable to me when I run them locally. 






From steve at hibernate.org  Fri Jun  2 10:40:23 2017
From: steve at hibernate.org (Steve Ebersole)
Date: Fri, 02 Jun 2017 14:40:23 +0000
Subject: [infinispan-dev] Proposal for moving Hibernate 2l provider to
	Infinispan
In-Reply-To: <D3193D97-234D-4FFB-9822-E45A9F13E0BC@redhat.com>
References: <46A375F3-84FB-46FD-8523-7F3340B4DCA5@redhat.com>
	<CAFdSV9tKNpCMyK2eL-vTMnAJOaoFYzARazcs5OUxAeLzhOZ_zw@mail.gmail.com>
	<D3193D97-234D-4FFB-9822-E45A9F13E0BC@redhat.com>
Message-ID: <CAFdSV9ujRNZHhajHXg2FvpMwfNykaDQry-9iVKZnFDbNWB-96Q@mail.gmail.com>

Awesome, good to know...

About the tests... yes, that is the problem with "intermittent" ;)  You can
run it 10 times and then it will break.  Then you run it 500 times and it
breaks.  All without any code changes.  But like I said, if that is really
fixed... awesome


On Fri, Jun 2, 2017 at 9:13 AM Galder Zamarre?o <galder at redhat.com> wrote:

> I think it's going through, we've approved you in the past.
>
> Replies below:
>
> > On 31 May 2017, at 17:02, Steve Ebersole <steve at hibernate.org> wrote:
> >
> > Just a heads up - FWIW I doubt my reply goes through to the entire
> infinispan-dev list.
> >
> > Replies inline...
> >
> >
> > 3. What should be the artifact name? Should it be 'hibernate-infinispan'
> like it is today? The difference with the existing cache provider would be
> the groupId. Or some other artifact id?
> >
> > Since you use Maven (IIUC) you could just publish a relocation...
>
> Oh, didn't know about that. Yeah, I think we'd do that:
> https://maven.apache.org/guides/mini/guide-relocation.html
>
> >
> >
> > 4. Should the main artifact contain the hibernate major version it
> belongs to? E.g. assuming we take 'hibernate-infinispan', should it be like
> that, or should it instead be 'hibernate5-infinispan'? This is where it'd
> be interesting to hear about our past Lucene directory or Query integration
> experience.
> >
> > Probably, but (no promises) one thing I wanted to look at since
> Hibernate baselines on Java 8, is to maintain the existing SPI using
> default methods as a bridge.  But failing that, I think your suggestion is
> the best option.
> >
> >
> > 5. A thing to consider also is whether to maintain same package naming.
> We're currently using 'org.hibernate.cache.infinispan.*'. From a
> compatibility sense, it'd help to keep same package since users reference
> region factory fully qualified class names. We'd also continue to be sole
> owners of 'org.hibernate.cache.infinispan.*'. However, I dunno whether
> having 'org.hibernate...' package name within Infinispan repo would create
> other issues?
> >
> > FWIW Hibernate offers "short naming" or "friendly naming" for many
> configurable settings, cache providers being one.  For hibernate-infinispan
> we register 2: "infinispan" and "infinispan-jndi".  You can see this in
> org.hibernate.cache.infinispan.StrategyRegistrationProviderImpl.  That
> approach will continue to work when you move it.  The point being that
> users do not specify the class name in config, they'd just specify
> "infinispan", "infinispan-jndi", etc.
>
> Ah good to know, I wasn't aware of it. I'll look into that.
>
> > 6. Testing wise, the cache provider is currently tested one test at the
> time, using JUnit. The testsuite already runs fast enough and I'd prefer
> not to change anything in this area right now. Is that Ok? Or is there any
> desire to move it to TestNG?
> >
> > Hmmm, that is actually surprising... I thought the hibernate-infinispan
> provider tests were still disabled as they had routinely caused
> intermittent failures of the build.  I guess this was rectified?
>
> They seem pretty stable to me when I run them locally.
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170602/7c19dce2/attachment.html 

From sanne at infinispan.org  Fri Jun  2 11:20:59 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 2 Jun 2017 16:20:59 +0100
Subject: [infinispan-dev] Proposal for moving Hibernate 2l provider to
	Infinispan
In-Reply-To: <D3193D97-234D-4FFB-9822-E45A9F13E0BC@redhat.com>
References: <46A375F3-84FB-46FD-8523-7F3340B4DCA5@redhat.com>
	<CAFdSV9tKNpCMyK2eL-vTMnAJOaoFYzARazcs5OUxAeLzhOZ_zw@mail.gmail.com>
	<D3193D97-234D-4FFB-9822-E45A9F13E0BC@redhat.com>
Message-ID: <CAFm4XO33-gG493TpP+_xLurDoXdeEd=HLc-rnXUb_G=Y2QMa1w@mail.gmail.com>

A couple of comments inline:

On 2 June 2017 at 13:45, Galder Zamarre?o <galder at redhat.com> wrote:
> I think it's going through, we've approved you in the past.
>
> Replies below:
>
>> On 31 May 2017, at 17:02, Steve Ebersole <steve at hibernate.org> wrote:
>>
>> Just a heads up - FWIW I doubt my reply goes through to the entire infinispan-dev list.

We're good, I received your answers too.

>> Replies inline...
>>
>>
>> 3. What should be the artifact name? Should it be 'hibernate-infinispan' like it is today? The difference with the existing cache provider would be the groupId. Or some other artifact id?
>>
>> Since you use Maven (IIUC) you could just publish a relocation...
>
> Oh, didn't know about that. Yeah, I think we'd do that:
> https://maven.apache.org/guides/mini/guide-relocation.html

Here an example, from when the Hibernate Search / Infinispan Directory
component was moved to Infinispan:

 - https://github.com/hibernate/hibernate-search/blob/master/infinispan/pom.xml#L23-L30

I guess you'll need an example in Gradle, but at least it shows you
can redirect all of organizationId, artifactId and versions as this
has worked nicely.

It's transparent to the users but they'll get a warning in the build.

>> 4. Should the main artifact contain the hibernate major version it belongs to? E.g. assuming we take 'hibernate-infinispan', should it be like that, or should it instead be 'hibernate5-infinispan'? This is where it'd be interesting to hear about our past Lucene directory or Query integration experience.
>>
>> Probably, but (no promises) one thing I wanted to look at since Hibernate baselines on Java 8, is to maintain the existing SPI using default methods as a bridge.  But failing that, I think your suggestion is the best option.
>>
>>
>> 5. A thing to consider also is whether to maintain same package naming. We're currently using 'org.hibernate.cache.infinispan.*'. From a compatibility sense, it'd help to keep same package since users reference region factory fully qualified class names. We'd also continue to be sole owners of 'org.hibernate.cache.infinispan.*'. However, I dunno whether having 'org.hibernate...' package name within Infinispan repo would create other issues?
>>
>> FWIW Hibernate offers "short naming" or "friendly naming" for many configurable settings, cache providers being one.  For hibernate-infinispan we register 2: "infinispan" and "infinispan-jndi".  You can see this in org.hibernate.cache.infinispan.StrategyRegistrationProviderImpl.  That approach will continue to work when you move it.  The point being that users do not specify the class name in config, they'd just specify "infinispan", "infinispan-jndi", etc.
>
> Ah good to know, I wasn't aware of it. I'll look into that.
>
>> 6. Testing wise, the cache provider is currently tested one test at the time, using JUnit. The testsuite already runs fast enough and I'd prefer not to change anything in this area right now. Is that Ok? Or is there any desire to move it to TestNG?

I wouldn't convert them, unless you have a concrete need to reuse some
helpers, but even so I'd be conservative.
Several other modules within Infinispan are also using JUnit and this
is unlikely to change.

Eventually I'd actually like to discuss moving it all to JUnit 5, but
let's keep that debate for another day.

Thanks,
Sanne

>>
>> Hmmm, that is actually surprising... I thought the hibernate-infinispan  provider tests were still disabled as they had routinely caused intermittent failures of the build.  I guess this was rectified?
>
> They seem pretty stable to me when I run them locally.
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From ttarrant at redhat.com  Mon Jun  5 04:32:21 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 5 Jun 2017 10:32:21 +0200
Subject: [infinispan-dev] Netty SSL Context,
 was [Hot Rod secured by default]
In-Reply-To: <CAH8Ud1HLmLht1eZ-M7U-UWHEgpeiiM_OdraKrKwbcpqg8K+g6Q@mail.gmail.com>
References: <f3cee023-96a1-3c7d-a3ce-b575cd5b0f7e@redhat.com>
	<CAA9R9O7S7frPE-sS60Z2E6jV_+OdL18+dvMDFJwQsd-FQfQt7w@mail.gmail.com>
	<df6c29dc-4d48-1dc7-6045-2a14e0dd01d8@redhat.com>
	<06b88c80-433f-3745-2112-1424bcd9c342@redhat.com>
	<CAKwmrdPrJgy_KPyZNHnK40mGrCqG05DyntvC_3R=TKK2P6-AoQ@mail.gmail.com>
	<CA+nfvwQjSy+oC0gwoEccEMs_E0k0ksD5JsF7rYb2Jdxm3g4mxA@mail.gmail.com>
	<CAA9R9O5eYa2GXuk+bmQcDYt7RVsNnKeJMadXM99+hTMhjGwueA@mail.gmail.com>
	<dec3db85-5b52-33bd-e9c7-a5a65cee3edc@redhat.com>
	<CAA9R9O6Fw=hJrVz0VVa2eX3-jqAwvEG51ySs10VygDqMpdHJfw@mail.gmail.com>
	<CAH8Ud1HLmLht1eZ-M7U-UWHEgpeiiM_OdraKrKwbcpqg8K+g6Q@mail.gmail.com>
Message-ID: <ed68a0b1-92fe-d2a5-8640-d2db3678b7cc@redhat.com>

We should use this:

https://github.com/wildfly/wildfly-openssl

Tristan

On 6/1/17 1:17 PM, Gustavo Fernandes wrote:
> On Thu, Jun 1, 2017 at 10:51 AM, Sebastian Laskawiec 
> <slaskawi at redhat.com <mailto:slaskawi at redhat.com>> wrote:
> 
>     I think I've just found the reason why we can not migrate in OpenSSL
>     by default :(
> 
>     In server scenario we obtain S*SL*Context (the one from JDK; Netty
>     has similar S*sl*Context) from WildFly. It is already configured
>     along with sercurity realms, domains etc. We then get into this
>     branch of code [1].
> 
>     In order to do fancy things like SNI we need to remap JDK's
>     SSLContext into Netty's SslContext and the only implementation that
>     can consume SSLContext we have at hand is JdkSslContext.
> 
>     I honestly have no idea how we could refactor this... And that's a
>     shame because OpenSSL is way faster...
> 
> 
> 
> I tried migrating the SSL engine to Netty's in [1] and hit the same 
> wall. What I was told is that the SSLContext in Wildfly is now (version 
> 11?) a capability under 'org.wildfly.security.ssl-context'  and
> can be replaced, but I did not try doing that.
> 
> 
> [1] https://issues.jboss.org/browse/ISPN-6990 
> <https://issues.jboss.org/browse/ISPN-6990>
> 
> Gustavo
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From slaskawi at redhat.com  Mon Jun  5 06:29:20 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Mon, 05 Jun 2017 10:29:20 +0000
Subject: [infinispan-dev] Netty SSL Context,
	was [Hot Rod secured by default]
In-Reply-To: <ed68a0b1-92fe-d2a5-8640-d2db3678b7cc@redhat.com>
References: <f3cee023-96a1-3c7d-a3ce-b575cd5b0f7e@redhat.com>
	<CAA9R9O7S7frPE-sS60Z2E6jV_+OdL18+dvMDFJwQsd-FQfQt7w@mail.gmail.com>
	<df6c29dc-4d48-1dc7-6045-2a14e0dd01d8@redhat.com>
	<06b88c80-433f-3745-2112-1424bcd9c342@redhat.com>
	<CAKwmrdPrJgy_KPyZNHnK40mGrCqG05DyntvC_3R=TKK2P6-AoQ@mail.gmail.com>
	<CA+nfvwQjSy+oC0gwoEccEMs_E0k0ksD5JsF7rYb2Jdxm3g4mxA@mail.gmail.com>
	<CAA9R9O5eYa2GXuk+bmQcDYt7RVsNnKeJMadXM99+hTMhjGwueA@mail.gmail.com>
	<dec3db85-5b52-33bd-e9c7-a5a65cee3edc@redhat.com>
	<CAA9R9O6Fw=hJrVz0VVa2eX3-jqAwvEG51ySs10VygDqMpdHJfw@mail.gmail.com>
	<CAH8Ud1HLmLht1eZ-M7U-UWHEgpeiiM_OdraKrKwbcpqg8K+g6Q@mail.gmail.com>
	<ed68a0b1-92fe-d2a5-8640-d2db3678b7cc@redhat.com>
Message-ID: <CAA9R9O7qcstPy91D4yG6ECxUYfbxhn-5jtccW6zWiof+cn548Q@mail.gmail.com>

We actually have more alternatives - e.g. we could use OpenSSL via Boring
SSL library [1]. The root problem remains the same - we can use only what
we obtain from the WF server. And currently we obtain only JSSE
SSLContext...

[1] http://netty.io/wiki/forked-tomcat-native.html

On Mon, Jun 5, 2017 at 10:34 AM Tristan Tarrant <ttarrant at redhat.com> wrote:

> We should use this:
>
> https://github.com/wildfly/wildfly-openssl
>
> Tristan
>
> On 6/1/17 1:17 PM, Gustavo Fernandes wrote:
> > On Thu, Jun 1, 2017 at 10:51 AM, Sebastian Laskawiec
> > <slaskawi at redhat.com <mailto:slaskawi at redhat.com>> wrote:
> >
> >     I think I've just found the reason why we can not migrate in OpenSSL
> >     by default :(
> >
> >     In server scenario we obtain S*SL*Context (the one from JDK; Netty
> >     has similar S*sl*Context) from WildFly. It is already configured
> >     along with sercurity realms, domains etc. We then get into this
> >     branch of code [1].
> >
> >     In order to do fancy things like SNI we need to remap JDK's
> >     SSLContext into Netty's SslContext and the only implementation that
> >     can consume SSLContext we have at hand is JdkSslContext.
> >
> >     I honestly have no idea how we could refactor this... And that's a
> >     shame because OpenSSL is way faster...
> >
> >
> >
> > I tried migrating the SSL engine to Netty's in [1] and hit the same
> > wall. What I was told is that the SSLContext in Wildfly is now (version
> > 11?) a capability under 'org.wildfly.security.ssl-context'  and
> > can be replaced, but I did not try doing that.
> >
> >
> > [1] https://issues.jboss.org/browse/ISPN-6990
> > <https://issues.jboss.org/browse/ISPN-6990>
> >
> > Gustavo
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170605/8dd92679/attachment-0001.html 

From ttarrant at redhat.com  Mon Jun  5 08:02:14 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 5 Jun 2017 14:02:14 +0200
Subject: [infinispan-dev] Netty SSL Context,
 was [Hot Rod secured by default]
In-Reply-To: <CAA9R9O7qcstPy91D4yG6ECxUYfbxhn-5jtccW6zWiof+cn548Q@mail.gmail.com>
References: <f3cee023-96a1-3c7d-a3ce-b575cd5b0f7e@redhat.com>
	<CAA9R9O7S7frPE-sS60Z2E6jV_+OdL18+dvMDFJwQsd-FQfQt7w@mail.gmail.com>
	<df6c29dc-4d48-1dc7-6045-2a14e0dd01d8@redhat.com>
	<06b88c80-433f-3745-2112-1424bcd9c342@redhat.com>
	<CAKwmrdPrJgy_KPyZNHnK40mGrCqG05DyntvC_3R=TKK2P6-AoQ@mail.gmail.com>
	<CA+nfvwQjSy+oC0gwoEccEMs_E0k0ksD5JsF7rYb2Jdxm3g4mxA@mail.gmail.com>
	<CAA9R9O5eYa2GXuk+bmQcDYt7RVsNnKeJMadXM99+hTMhjGwueA@mail.gmail.com>
	<dec3db85-5b52-33bd-e9c7-a5a65cee3edc@redhat.com>
	<CAA9R9O6Fw=hJrVz0VVa2eX3-jqAwvEG51ySs10VygDqMpdHJfw@mail.gmail.com>
	<CAH8Ud1HLmLht1eZ-M7U-UWHEgpeiiM_OdraKrKwbcpqg8K+g6Q@mail.gmail.com>
	<ed68a0b1-92fe-d2a5-8640-d2db3678b7cc@redhat.com>
	<CAA9R9O7qcstPy91D4yG6ECxUYfbxhn-5jtccW6zWiof+cn548Q@mail.gmail.com>
Message-ID: <d3295341-edee-99c4-a29d-b948bb087860@redhat.com>

Actually, WildFly 11 will allow this.
Additionally, in our restructured server, we can do whatever we want.

Tristan

On 6/5/17 12:29 PM, Sebastian Laskawiec wrote:
> We actually have more alternatives - e.g. we could use OpenSSL via 
> Boring SSL library [1]. The root problem remains the same - we can use 
> only what we obtain from the WF server. And currently we obtain 
> only JSSE SSLContext...
> 
> [1] http://netty.io/wiki/forked-tomcat-native.html
> 
> On Mon, Jun 5, 2017 at 10:34 AM Tristan Tarrant <ttarrant at redhat.com 
> <mailto:ttarrant at redhat.com>> wrote:
> 
>     We should use this:
> 
>     https://github.com/wildfly/wildfly-openssl
> 
>     Tristan
> 
>     On 6/1/17 1:17 PM, Gustavo Fernandes wrote:
>      > On Thu, Jun 1, 2017 at 10:51 AM, Sebastian Laskawiec
>      > <slaskawi at redhat.com <mailto:slaskawi at redhat.com>
>     <mailto:slaskawi at redhat.com <mailto:slaskawi at redhat.com>>> wrote:
>      >
>      >     I think I've just found the reason why we can not migrate in
>     OpenSSL
>      >     by default :(
>      >
>      >     In server scenario we obtain S*SL*Context (the one from JDK;
>     Netty
>      >     has similar S*sl*Context) from WildFly. It is already configured
>      >     along with sercurity realms, domains etc. We then get into this
>      >     branch of code [1].
>      >
>      >     In order to do fancy things like SNI we need to remap JDK's
>      >     SSLContext into Netty's SslContext and the only
>     implementation that
>      >     can consume SSLContext we have at hand is JdkSslContext.
>      >
>      >     I honestly have no idea how we could refactor this... And
>     that's a
>      >     shame because OpenSSL is way faster...
>      >
>      >
>      >
>      > I tried migrating the SSL engine to Netty's in [1] and hit the same
>      > wall. What I was told is that the SSLContext in Wildfly is now
>     (version
>      > 11?) a capability under 'org.wildfly.security.ssl-context'  and
>      > can be replaced, but I did not try doing that.
>      >
>      >
>      > [1] https://issues.jboss.org/browse/ISPN-6990
>      > <https://issues.jboss.org/browse/ISPN-6990>
>      >
>      > Gustavo
>      >
>      >
>      > _______________________________________________
>      > infinispan-dev mailing list
>      > infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>      > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>      >
> 
>     --
>     Tristan Tarrant
>     Infinispan Lead
>     JBoss, a division of Red Hat
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> -- 
> 
> SEBASTIAN?ASKAWIEC
> 
> INFINISPAN DEVELOPER
> 
> Red HatEMEA <https://www.redhat.com/>
> 
> <https://red.ht/sig>
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From ttarrant at redhat.com  Mon Jun  5 11:05:09 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 5 Jun 2017 17:05:09 +0200
Subject: [infinispan-dev] Weekly IRC Meeting logs 2017-06-05
Message-ID: <2cf645c6-b0de-a15d-600c-e5795dce7e77@redhat.com>

Hi all,

the logs for this week's meeting are here:

http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2017/infinispan.2017-06-05-14.01.log.html

Tristan
-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From karesti at redhat.com  Tue Jun  6 03:44:29 2017
From: karesti at redhat.com (Katia Aresti)
Date: Tue, 6 Jun 2017 09:44:29 +0200
Subject: [infinispan-dev] Weekly IRC Meeting logs 2017-06-05
In-Reply-To: <2cf645c6-b0de-a15d-600c-e5795dce7e77@redhat.com>
References: <2cf645c6-b0de-a15d-600c-e5795dce7e77@redhat.com>
Message-ID: <CAKypdvmqqJJcUVKdQ9AhcC2xSCN3u1qKmMFhRYs9+WJV0U3+DA@mail.gmail.com>

Hi all !

I've been two Mondays in PTO :)

I've been working on Clustered Locks design and Multimap Cache support
(design and code).
I will be working on this during the week too.

Katia

On Mon, Jun 5, 2017 at 5:05 PM, Tristan Tarrant <ttarrant at redhat.com> wrote:

> Hi all,
>
> the logs for this week's meeting are here:
>
> http://transcripts.jboss.org/meeting/irc.freenode.org/
> infinispan/2017/infinispan.2017-06-05-14.01.log.html
>
> Tristan
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170606/5844eb60/attachment.html 

From slaskawi at redhat.com  Wed Jun  7 02:26:40 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 07 Jun 2017 06:26:40 +0000
Subject: [infinispan-dev] Rerun builds for your PRs
Message-ID: <CAA9R9O5pJB56v_5KO-ByOYJ8DYd9QGCXucrePQqBihhxX2cVFg@mail.gmail.com>

Hey,

Unfortunately Jenkins had some hiccup after Central CI outage this weekend
and I had to clear the job pending queue.

Please rerun your PR builds by navigating to the PR builds page and using
build now button.

Thanks,
Sebastian
-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170607/588a9b29/attachment.html 

From galder at redhat.com  Wed Jun  7 05:19:10 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 7 Jun 2017 11:19:10 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided dependency
Message-ID: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>

Hi all,

Re: https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579

St?phane makes a good point there, why did we make core provided dependency? It does feel a bit of a pain that anyone that depends on jcache embedded also needs to depend on core.

Any more details behind this decision?

Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat



From slaskawi at redhat.com  Wed Jun  7 05:50:47 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 07 Jun 2017 09:50:47 +0000
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
Message-ID: <CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>

Hey,

The change was introduced by this commit [1] and relates to this JIRAs
[2][3]. The root cause is in [3].

Imagine a scenario where you add JCache module to your together
infinispan-embedded. If your classpath was constructed in such a way that
infinispan-embedded was before infinispan-core (classpath is scanned from
left to right in standalone apps), we could get a relocated (uber jars move
some classes into other packages) logger. That caused class mismatch
errors. It is worth to mention that it will happen to all relocated
classes, logger was just an example. And we need to relocate them, since a
user might want to use his own, newer version of DMR or any other library.
So there's no perfect solution here.

Now a lot of time passed since then and we changed quite a few things. So
this topic probably needs to be revisited.

So the first question that we should ask, shall we allow putting jcache and
infinispan-embedded together on the classpath. If the answer is yes, I
believe it should stay as it is (since the user always have a choice
whether he wants to use jcache with or without uber jar). The same question
needs to be asked for Spring modules as well as all cache stores. The
behavior needs to be consistent across all those modules.

If the answer is no (which is also valid because jcache is already present
in embedded uber jar), we should migrate all JBoss Logging references to
Infinispan Common Logging (as Tristan did here [4]) and we can make
infinispan-core as a compile time dependency to jcache. Even though
migrating to Infinispan logger is not necessary, this way we won't break
users app which used infinispan-embedded + jcache approach. Of course the
same applies to Spring and Cache stores modules.

I think the latter approach deserves some exploration. I would vote for
moving that way.

Thanks,
Sebastian

[1]
https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
[2] https://issues.jboss.org/browse/ISPN-6295
[3] https://issues.jboss.org/browse/ISPN-6132
[4] https://github.com/infinispan/infinispan/pull/4140/files


On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com> wrote:

> Hi all,
>
> Re:
> https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>
> St?phane makes a good point there, why did we make core provided
> dependency? It does feel a bit of a pain that anyone that depends on jcache
> embedded also needs to depend on core.
>
> Any more details behind this decision?
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
> --

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170607/d2f207da/attachment-0001.html 

From galder at redhat.com  Wed Jun  7 06:02:01 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 7 Jun 2017 12:02:01 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
Message-ID: <EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>

As far as I see it:

* infinispan-embedded should never be a dependency in a Maven project.

* No uber jars should really be used as Maven dependencies because all the exclusion that fine grained dependencies allow you to do goes out of the window when all classes are inside a jar. This is not just theory, I've personally had such issues.

* Uber jars are designed for Ant or other build tool users that don't have a dependency resolution engine in place.

Cheers,

p.s. I thought we had already discussed this before?
--
Galder Zamarre?o
Infinispan, Red Hat

> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> 
> Hey,
> 
> The change was introduced by this commit [1] and relates to this JIRAs [2][3]. The root cause is in [3].
> 
> Imagine a scenario where you add JCache module to your together infinispan-embedded. If your classpath was constructed in such a way that infinispan-embedded was before infinispan-core (classpath is scanned from left to right in standalone apps), we could get a relocated (uber jars move some classes into other packages) logger. That caused class mismatch errors. It is worth to mention that it will happen to all relocated classes, logger was just an example. And we need to relocate them, since a user might want to use his own, newer version of DMR or any other library. So there's no perfect solution here.
> 
> Now a lot of time passed since then and we changed quite a few things. So this topic probably needs to be revisited. 
> 
> So the first question that we should ask, shall we allow putting jcache and infinispan-embedded together on the classpath. If the answer is yes, I believe it should stay as it is (since the user always have a choice whether he wants to use jcache with or without uber jar). The same question needs to be asked for Spring modules as well as all cache stores. The behavior needs to be consistent across all those modules.
> 
> If the answer is no (which is also valid because jcache is already present in embedded uber jar), we should migrate all JBoss Logging references to Infinispan Common Logging (as Tristan did here [4]) and we can make infinispan-core as a compile time dependency to jcache. Even though migrating to Infinispan logger is not necessary, this way we won't break users app which used infinispan-embedded + jcache approach. Of course the same applies to Spring and Cache stores modules.
> 
> I think the latter approach deserves some exploration. I would vote for moving that way.
> 
> Thanks,
> Sebastian
> 
> [1] https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
> [2] https://issues.jboss.org/browse/ISPN-6295
> [3] https://issues.jboss.org/browse/ISPN-6132
> [4] https://github.com/infinispan/infinispan/pull/4140/files
> 
> 
> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com> wrote:
> Hi all,
> 
> Re: https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
> 
> St?phane makes a good point there, why did we make core provided dependency? It does feel a bit of a pain that anyone that depends on jcache embedded also needs to depend on core.
> 
> Any more details behind this decision?
> 
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
> 
> -- 
> SEBASTIAN ?ASKAWIEC
> INFINISPAN DEVELOPER
> Red Hat EMEA
> 



From galder at redhat.com  Wed Jun  7 06:05:00 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 7 Jun 2017 12:05:00 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
Message-ID: <60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>

Moreover:

* The experience of Maven users should never be penalised by uber jars. Uber jar users should be a minority compared with Maven/Gradle...etc users that have dependency engines in place to select which components they want to depend on.

Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat

> On 7 Jun 2017, at 12:02, Galder Zamarre?o <galder at redhat.com> wrote:
> 
> As far as I see it:
> 
> * infinispan-embedded should never be a dependency in a Maven project.
> 
> * No uber jars should really be used as Maven dependencies because all the exclusion that fine grained dependencies allow you to do goes out of the window when all classes are inside a jar. This is not just theory, I've personally had such issues.
> 
> * Uber jars are designed for Ant or other build tool users that don't have a dependency resolution engine in place.
> 
> Cheers,
> 
> p.s. I thought we had already discussed this before?
> --
> Galder Zamarre?o
> Infinispan, Red Hat
> 
>> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>> 
>> Hey,
>> 
>> The change was introduced by this commit [1] and relates to this JIRAs [2][3]. The root cause is in [3].
>> 
>> Imagine a scenario where you add JCache module to your together infinispan-embedded. If your classpath was constructed in such a way that infinispan-embedded was before infinispan-core (classpath is scanned from left to right in standalone apps), we could get a relocated (uber jars move some classes into other packages) logger. That caused class mismatch errors. It is worth to mention that it will happen to all relocated classes, logger was just an example. And we need to relocate them, since a user might want to use his own, newer version of DMR or any other library. So there's no perfect solution here.
>> 
>> Now a lot of time passed since then and we changed quite a few things. So this topic probably needs to be revisited. 
>> 
>> So the first question that we should ask, shall we allow putting jcache and infinispan-embedded together on the classpath. If the answer is yes, I believe it should stay as it is (since the user always have a choice whether he wants to use jcache with or without uber jar). The same question needs to be asked for Spring modules as well as all cache stores. The behavior needs to be consistent across all those modules.
>> 
>> If the answer is no (which is also valid because jcache is already present in embedded uber jar), we should migrate all JBoss Logging references to Infinispan Common Logging (as Tristan did here [4]) and we can make infinispan-core as a compile time dependency to jcache. Even though migrating to Infinispan logger is not necessary, this way we won't break users app which used infinispan-embedded + jcache approach. Of course the same applies to Spring and Cache stores modules.
>> 
>> I think the latter approach deserves some exploration. I would vote for moving that way.
>> 
>> Thanks,
>> Sebastian
>> 
>> [1] https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>> [2] https://issues.jboss.org/browse/ISPN-6295
>> [3] https://issues.jboss.org/browse/ISPN-6132
>> [4] https://github.com/infinispan/infinispan/pull/4140/files
>> 
>> 
>> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com> wrote:
>> Hi all,
>> 
>> Re: https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>> 
>> St?phane makes a good point there, why did we make core provided dependency? It does feel a bit of a pain that anyone that depends on jcache embedded also needs to depend on core.
>> 
>> Any more details behind this decision?
>> 
>> Cheers,
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>> 
>> -- 
>> SEBASTIAN ?ASKAWIEC
>> INFINISPAN DEVELOPER
>> Red Hat EMEA
>> 
> 



From slaskawi at redhat.com  Wed Jun  7 06:23:12 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 07 Jun 2017 10:23:12 +0000
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>
Message-ID: <CAA9R9O4seehkxr=JxpeJ+h5nswT-TdUzzHKN8R+rHj+-7oj=zA@mail.gmail.com>

We discussed it number of times, including (but probably not limited to):

   -
   http://lists.jboss.org/pipermail/infinispan-dev/2016-February/016414.html
   - http://lists.jboss.org/pipermail/infinispan-dev/2016-March/016490.html

You might also want to look into the internal lists...

The biggest question - has anything changed? Do we have any other idea?

On Wed, Jun 7, 2017 at 12:05 PM Galder Zamarre?o <galder at redhat.com> wrote:

> Moreover:
>
> * The experience of Maven users should never be penalised by uber jars.
> Uber jar users should be a minority compared with Maven/Gradle...etc users
> that have dependency engines in place to select which components they want
> to depend on.
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
> > On 7 Jun 2017, at 12:02, Galder Zamarre?o <galder at redhat.com> wrote:
> >
> > As far as I see it:
> >
> > * infinispan-embedded should never be a dependency in a Maven project.
> >
> > * No uber jars should really be used as Maven dependencies because all
> the exclusion that fine grained dependencies allow you to do goes out of
> the window when all classes are inside a jar. This is not just theory, I've
> personally had such issues.
> >
> > * Uber jars are designed for Ant or other build tool users that don't
> have a dependency resolution engine in place.
> >
> > Cheers,
> >
> > p.s. I thought we had already discussed this before?
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> >> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
> >>
> >> Hey,
> >>
> >> The change was introduced by this commit [1] and relates to this JIRAs
> [2][3]. The root cause is in [3].
> >>
> >> Imagine a scenario where you add JCache module to your together
> infinispan-embedded. If your classpath was constructed in such a way that
> infinispan-embedded was before infinispan-core (classpath is scanned from
> left to right in standalone apps), we could get a relocated (uber jars move
> some classes into other packages) logger. That caused class mismatch
> errors. It is worth to mention that it will happen to all relocated
> classes, logger was just an example. And we need to relocate them, since a
> user might want to use his own, newer version of DMR or any other library.
> So there's no perfect solution here.
> >>
> >> Now a lot of time passed since then and we changed quite a few things.
> So this topic probably needs to be revisited.
> >>
> >> So the first question that we should ask, shall we allow putting jcache
> and infinispan-embedded together on the classpath. If the answer is yes, I
> believe it should stay as it is (since the user always have a choice
> whether he wants to use jcache with or without uber jar). The same question
> needs to be asked for Spring modules as well as all cache stores. The
> behavior needs to be consistent across all those modules.
> >>
> >> If the answer is no (which is also valid because jcache is already
> present in embedded uber jar), we should migrate all JBoss Logging
> references to Infinispan Common Logging (as Tristan did here [4]) and we
> can make infinispan-core as a compile time dependency to jcache. Even
> though migrating to Infinispan logger is not necessary, this way we won't
> break users app which used infinispan-embedded + jcache approach. Of course
> the same applies to Spring and Cache stores modules.
> >>
> >> I think the latter approach deserves some exploration. I would vote for
> moving that way.
> >>
> >> Thanks,
> >> Sebastian
> >>
> >> [1]
> https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
> >> [2] https://issues.jboss.org/browse/ISPN-6295
> >> [3] https://issues.jboss.org/browse/ISPN-6132
> >> [4] https://github.com/infinispan/infinispan/pull/4140/files
> >>
> >>
> >> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com>
> wrote:
> >> Hi all,
> >>
> >> Re:
> https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
> >>
> >> St?phane makes a good point there, why did we make core provided
> dependency? It does feel a bit of a pain that anyone that depends on jcache
> embedded also needs to depend on core.
> >>
> >> Any more details behind this decision?
> >>
> >> Cheers,
> >> --
> >> Galder Zamarre?o
> >> Infinispan, Red Hat
> >>
> >> --
> >> SEBASTIAN ?ASKAWIEC
> >> INFINISPAN DEVELOPER
> >> Red Hat EMEA
> >>
> >
>
> --

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170607/7a867720/attachment.html 

From gustavo at infinispan.org  Wed Jun  7 11:10:10 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Wed, 7 Jun 2017 16:10:10 +0100
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
Message-ID: <CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>

On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com> wrote:

> As far as I see it:
>
> * infinispan-embedded should never be a dependency in a Maven project.
>
> * No uber jars should really be used as Maven dependencies because all the
> exclusion that fine grained dependencies allow you to do goes out of the
> window when all classes are inside a jar. This is not just theory, I've
> personally had such issues.
>
> * Uber jars are designed for Ant or other build tool users that don't have
> a dependency resolution engine in place.
>
> Cheers,
>
> p.s. I thought we had already discussed this before?
>


I totally agree. In addition, uberjars should not be an osgi bundle or a
jboss module, for similar reasons.

P.S: Even Ant has a dependency mgmt available, which is Ivy.

Cheers,
Gustavo



> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
> > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
> >
> > Hey,
> >
> > The change was introduced by this commit [1] and relates to this JIRAs
> [2][3]. The root cause is in [3].
> >
> > Imagine a scenario where you add JCache module to your together
> infinispan-embedded. If your classpath was constructed in such a way that
> infinispan-embedded was before infinispan-core (classpath is scanned from
> left to right in standalone apps), we could get a relocated (uber jars move
> some classes into other packages) logger. That caused class mismatch
> errors. It is worth to mention that it will happen to all relocated
> classes, logger was just an example. And we need to relocate them, since a
> user might want to use his own, newer version of DMR or any other library.
> So there's no perfect solution here.
> >
> > Now a lot of time passed since then and we changed quite a few things.
> So this topic probably needs to be revisited.
> >
> > So the first question that we should ask, shall we allow putting jcache
> and infinispan-embedded together on the classpath. If the answer is yes, I
> believe it should stay as it is (since the user always have a choice
> whether he wants to use jcache with or without uber jar). The same question
> needs to be asked for Spring modules as well as all cache stores. The
> behavior needs to be consistent across all those modules.
> >
> > If the answer is no (which is also valid because jcache is already
> present in embedded uber jar), we should migrate all JBoss Logging
> references to Infinispan Common Logging (as Tristan did here [4]) and we
> can make infinispan-core as a compile time dependency to jcache. Even
> though migrating to Infinispan logger is not necessary, this way we won't
> break users app which used infinispan-embedded + jcache approach. Of course
> the same applies to Spring and Cache stores modules.
> >
> > I think the latter approach deserves some exploration. I would vote for
> moving that way.
> >
> > Thanks,
> > Sebastian
> >
> > [1] https://github.com/infinispan/infinispan/commit/
> 720f158cce38d86b292e1ce77b75509342007739
> > [2] https://issues.jboss.org/browse/ISPN-6295
> > [3] https://issues.jboss.org/browse/ISPN-6132
> > [4] https://github.com/infinispan/infinispan/pull/4140/files
> >
> >
> > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com>
> wrote:
> > Hi all,
> >
> > Re: https://github.com/spring-projects/spring-boot/pull/
> 9417#discussion_r120375579
> >
> > St?phane makes a good point there, why did we make core provided
> dependency? It does feel a bit of a pain that anyone that depends on jcache
> embedded also needs to depend on core.
> >
> > Any more details behind this decision?
> >
> > Cheers,
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> > --
> > SEBASTIAN ?ASKAWIEC
> > INFINISPAN DEVELOPER
> > Red Hat EMEA
> >
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170607/c6b9ed85/attachment-0001.html 

From sanne at infinispan.org  Wed Jun  7 11:35:58 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 7 Jun 2017 16:35:58 +0100
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>
Message-ID: <CAFm4XO0+-XGKJt0qKtjnU8TT=jsr_NcVg4GWoMTejDJgtSxLYQ@mail.gmail.com>

On 7 June 2017 at 11:05, Galder Zamarre?o <galder at redhat.com> wrote:
> Moreover:
>
> * The experience of Maven users should never be penalised by uber jars. Uber jar users should be a minority compared with Maven/Gradle...etc users that have dependency engines in place to select which components they want to depend on.
>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 7 Jun 2017, at 12:02, Galder Zamarre?o <galder at redhat.com> wrote:
>>
>> As far as I see it:
>>
>> * infinispan-embedded should never be a dependency in a Maven project.
>>
>> * No uber jars should really be used as Maven dependencies because all the exclusion that fine grained dependencies allow you to do goes out of the window when all classes are inside a jar. This is not just theory, I've personally had such issues.
>>
>> * Uber jars are designed for Ant or other build tool users that don't have a dependency resolution engine in place.
>>
>> Cheers,
>>
>> p.s. I thought we had already discussed this before?

Memory eviction skills are strong with this team ;)

 - http://lists.jboss.org/pipermail/infinispan-dev/2016-November/017006.html
 - http://lists.jboss.org/pipermail/infinispan-dev/2016-March/016526.html
 - http://lists.jboss.org/pipermail/infinispan-dev/2017-April/017480.html
 - http://lists.jboss.org/pipermail/infinispan-dev/2016-March/016503.html
   -- http://lists.jboss.org/pipermail/infinispan-dev/2016-March/016504.html
 - http://lists.jboss.org/pipermail/infinispan-dev/2015-September/016187.html
 - ...



>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>>>
>>> Hey,
>>>
>>> The change was introduced by this commit [1] and relates to this JIRAs [2][3]. The root cause is in [3].
>>>
>>> Imagine a scenario where you add JCache module to your together infinispan-embedded. If your classpath was constructed in such a way that infinispan-embedded was before infinispan-core (classpath is scanned from left to right in standalone apps), we could get a relocated (uber jars move some classes into other packages) logger. That caused class mismatch errors. It is worth to mention that it will happen to all relocated classes, logger was just an example. And we need to relocate them, since a user might want to use his own, newer version of DMR or any other library. So there's no perfect solution here.
>>>
>>> Now a lot of time passed since then and we changed quite a few things. So this topic probably needs to be revisited.
>>>
>>> So the first question that we should ask, shall we allow putting jcache and infinispan-embedded together on the classpath. If the answer is yes, I believe it should stay as it is (since the user always have a choice whether he wants to use jcache with or without uber jar). The same question needs to be asked for Spring modules as well as all cache stores. The behavior needs to be consistent across all those modules.
>>>
>>> If the answer is no (which is also valid because jcache is already present in embedded uber jar), we should migrate all JBoss Logging references to Infinispan Common Logging (as Tristan did here [4]) and we can make infinispan-core as a compile time dependency to jcache. Even though migrating to Infinispan logger is not necessary, this way we won't break users app which used infinispan-embedded + jcache approach. Of course the same applies to Spring and Cache stores modules.
>>>
>>> I think the latter approach deserves some exploration. I would vote for moving that way.
>>>
>>> Thanks,
>>> Sebastian
>>>
>>> [1] https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>>> [2] https://issues.jboss.org/browse/ISPN-6295
>>> [3] https://issues.jboss.org/browse/ISPN-6132
>>> [4] https://github.com/infinispan/infinispan/pull/4140/files
>>>
>>>
>>> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com> wrote:
>>> Hi all,
>>>
>>> Re: https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>>>
>>> St?phane makes a good point there, why did we make core provided dependency? It does feel a bit of a pain that anyone that depends on jcache embedded also needs to depend on core.
>>>
>>> Any more details behind this decision?
>>>
>>> Cheers,
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>> --
>>> SEBASTIAN ?ASKAWIEC
>>> INFINISPAN DEVELOPER
>>> Red Hat EMEA
>>>
>>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From galder at redhat.com  Wed Jun  7 11:48:44 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 7 Jun 2017 17:48:44 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <CAA9R9O4seehkxr=JxpeJ+h5nswT-TdUzzHKN8R+rHj+-7oj=zA@mail.gmail.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>
	<CAA9R9O4seehkxr=JxpeJ+h5nswT-TdUzzHKN8R+rHj+-7oj=zA@mail.gmail.com>
Message-ID: <9082710C-9758-40C5-A986-3610866DCBAC@redhat.com>

What has changed is that St?phane has made a very good point which I had not realised:

Making core provided dependency means a JCache dependency user needs to add core dependency on top of that, which reduces usability. Core jar is not a provided dependency of JCache, it's a normal dependency. I don't think provided dependencies should be used to get around uber jar dependency issues.

IMO, the bigger usability issue is the fact that uber jars are available as Maven dependencies. Uber jars should just not be distributed as Maven dependencies. They should just be put somewhere else but not in Maven. That'd way we'd avoid the problem in the first place.

In the mean time, I think we should:

* Move back to having normal dependencies for core in JCache (and Spring too, if it applies)
* Go through our examples and avoid using uber jar dependencies.

Then, explore the idea above of not having uber jars as Maven dependencies.

Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat

> On 7 Jun 2017, at 12:23, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> 
> We discussed it number of times, including (but probably not limited to):
> 	? http://lists.jboss.org/pipermail/infinispan-dev/2016-February/016414.html
> 	? http://lists.jboss.org/pipermail/infinispan-dev/2016-March/016490.html
> You might also want to look into the internal lists...
> 
> The biggest question - has anything changed? Do we have any other idea? 
> 
> On Wed, Jun 7, 2017 at 12:05 PM Galder Zamarre?o <galder at redhat.com> wrote:
> Moreover:
> 
> * The experience of Maven users should never be penalised by uber jars. Uber jar users should be a minority compared with Maven/Gradle...etc users that have dependency engines in place to select which components they want to depend on.
> 
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
> 
> > On 7 Jun 2017, at 12:02, Galder Zamarre?o <galder at redhat.com> wrote:
> >
> > As far as I see it:
> >
> > * infinispan-embedded should never be a dependency in a Maven project.
> >
> > * No uber jars should really be used as Maven dependencies because all the exclusion that fine grained dependencies allow you to do goes out of the window when all classes are inside a jar. This is not just theory, I've personally had such issues.
> >
> > * Uber jars are designed for Ant or other build tool users that don't have a dependency resolution engine in place.
> >
> > Cheers,
> >
> > p.s. I thought we had already discussed this before?
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> >> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> >>
> >> Hey,
> >>
> >> The change was introduced by this commit [1] and relates to this JIRAs [2][3]. The root cause is in [3].
> >>
> >> Imagine a scenario where you add JCache module to your together infinispan-embedded. If your classpath was constructed in such a way that infinispan-embedded was before infinispan-core (classpath is scanned from left to right in standalone apps), we could get a relocated (uber jars move some classes into other packages) logger. That caused class mismatch errors. It is worth to mention that it will happen to all relocated classes, logger was just an example. And we need to relocate them, since a user might want to use his own, newer version of DMR or any other library. So there's no perfect solution here.
> >>
> >> Now a lot of time passed since then and we changed quite a few things. So this topic probably needs to be revisited.
> >>
> >> So the first question that we should ask, shall we allow putting jcache and infinispan-embedded together on the classpath. If the answer is yes, I believe it should stay as it is (since the user always have a choice whether he wants to use jcache with or without uber jar). The same question needs to be asked for Spring modules as well as all cache stores. The behavior needs to be consistent across all those modules.
> >>
> >> If the answer is no (which is also valid because jcache is already present in embedded uber jar), we should migrate all JBoss Logging references to Infinispan Common Logging (as Tristan did here [4]) and we can make infinispan-core as a compile time dependency to jcache. Even though migrating to Infinispan logger is not necessary, this way we won't break users app which used infinispan-embedded + jcache approach. Of course the same applies to Spring and Cache stores modules.
> >>
> >> I think the latter approach deserves some exploration. I would vote for moving that way.
> >>
> >> Thanks,
> >> Sebastian
> >>
> >> [1] https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
> >> [2] https://issues.jboss.org/browse/ISPN-6295
> >> [3] https://issues.jboss.org/browse/ISPN-6132
> >> [4] https://github.com/infinispan/infinispan/pull/4140/files
> >>
> >>
> >> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com> wrote:
> >> Hi all,
> >>
> >> Re: https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
> >>
> >> St?phane makes a good point there, why did we make core provided dependency? It does feel a bit of a pain that anyone that depends on jcache embedded also needs to depend on core.
> >>
> >> Any more details behind this decision?
> >>
> >> Cheers,
> >> --
> >> Galder Zamarre?o
> >> Infinispan, Red Hat
> >>
> >> --
> >> SEBASTIAN ?ASKAWIEC
> >> INFINISPAN DEVELOPER
> >> Red Hat EMEA
> >>
> >
> 
> -- 
> SEBASTIAN ?ASKAWIEC
> INFINISPAN DEVELOPER
> Red Hat EMEA
> 



From dereed at redhat.com  Wed Jun  7 19:58:25 2017
From: dereed at redhat.com (Dennis Reed)
Date: Wed, 7 Jun 2017 18:58:25 -0500
Subject: [infinispan-dev] Why JCache embedded has core as provided
 dependency
In-Reply-To: <9082710C-9758-40C5-A986-3610866DCBAC@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>
	<CAA9R9O4seehkxr=JxpeJ+h5nswT-TdUzzHKN8R+rHj+-7oj=zA@mail.gmail.com>
	<9082710C-9758-40C5-A986-3610866DCBAC@redhat.com>
Message-ID: <ca9ac372-66d2-d7e2-b6b6-f01e75f4cf85@redhat.com>

Or take it one step further, and just get rid of the uber jars.

Because there is really only a single benefit to uber jars: save maybe a
minute the very first time you start a new project and are using a build
system other than maven or similar.  And there are a whole bunch of
problems they cause.

-Dennis


On 06/07/2017 10:48 AM, Galder Zamarre?o wrote:
> What has changed is that St?phane has made a very good point which I had not realised:
> 
> Making core provided dependency means a JCache dependency user needs to add core dependency on top of that, which reduces usability. Core jar is not a provided dependency of JCache, it's a normal dependency. I don't think provided dependencies should be used to get around uber jar dependency issues.
> 
> IMO, the bigger usability issue is the fact that uber jars are available as Maven dependencies. Uber jars should just not be distributed as Maven dependencies. They should just be put somewhere else but not in Maven. That'd way we'd avoid the problem in the first place.
> 
> In the mean time, I think we should:
> 
> * Move back to having normal dependencies for core in JCache (and Spring too, if it applies)
> * Go through our examples and avoid using uber jar dependencies.
> 
> Then, explore the idea above of not having uber jars as Maven dependencies.
> 
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
> 
>> On 7 Jun 2017, at 12:23, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>>
>> We discussed it number of times, including (but probably not limited to):
>> 	? http://lists.jboss.org/pipermail/infinispan-dev/2016-February/016414.html
>> 	? http://lists.jboss.org/pipermail/infinispan-dev/2016-March/016490.html
>> You might also want to look into the internal lists...
>>
>> The biggest question - has anything changed? Do we have any other idea? 
>>
>> On Wed, Jun 7, 2017 at 12:05 PM Galder Zamarre?o <galder at redhat.com> wrote:
>> Moreover:
>>
>> * The experience of Maven users should never be penalised by uber jars. Uber jar users should be a minority compared with Maven/Gradle...etc users that have dependency engines in place to select which components they want to depend on.
>>
>> Cheers,
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>> On 7 Jun 2017, at 12:02, Galder Zamarre?o <galder at redhat.com> wrote:
>>>
>>> As far as I see it:
>>>
>>> * infinispan-embedded should never be a dependency in a Maven project.
>>>
>>> * No uber jars should really be used as Maven dependencies because all the exclusion that fine grained dependencies allow you to do goes out of the window when all classes are inside a jar. This is not just theory, I've personally had such issues.
>>>
>>> * Uber jars are designed for Ant or other build tool users that don't have a dependency resolution engine in place.
>>>
>>> Cheers,
>>>
>>> p.s. I thought we had already discussed this before?
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>>> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>>>>
>>>> Hey,
>>>>
>>>> The change was introduced by this commit [1] and relates to this JIRAs [2][3]. The root cause is in [3].
>>>>
>>>> Imagine a scenario where you add JCache module to your together infinispan-embedded. If your classpath was constructed in such a way that infinispan-embedded was before infinispan-core (classpath is scanned from left to right in standalone apps), we could get a relocated (uber jars move some classes into other packages) logger. That caused class mismatch errors. It is worth to mention that it will happen to all relocated classes, logger was just an example. And we need to relocate them, since a user might want to use his own, newer version of DMR or any other library. So there's no perfect solution here.
>>>>
>>>> Now a lot of time passed since then and we changed quite a few things. So this topic probably needs to be revisited.
>>>>
>>>> So the first question that we should ask, shall we allow putting jcache and infinispan-embedded together on the classpath. If the answer is yes, I believe it should stay as it is (since the user always have a choice whether he wants to use jcache with or without uber jar). The same question needs to be asked for Spring modules as well as all cache stores. The behavior needs to be consistent across all those modules.
>>>>
>>>> If the answer is no (which is also valid because jcache is already present in embedded uber jar), we should migrate all JBoss Logging references to Infinispan Common Logging (as Tristan did here [4]) and we can make infinispan-core as a compile time dependency to jcache. Even though migrating to Infinispan logger is not necessary, this way we won't break users app which used infinispan-embedded + jcache approach. Of course the same applies to Spring and Cache stores modules.
>>>>
>>>> I think the latter approach deserves some exploration. I would vote for moving that way.
>>>>
>>>> Thanks,
>>>> Sebastian
>>>>
>>>> [1] https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>>>> [2] https://issues.jboss.org/browse/ISPN-6295
>>>> [3] https://issues.jboss.org/browse/ISPN-6132
>>>> [4] https://github.com/infinispan/infinispan/pull/4140/files
>>>>
>>>>
>>>> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com> wrote:
>>>> Hi all,
>>>>
>>>> Re: https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>>>>
>>>> St?phane makes a good point there, why did we make core provided dependency? It does feel a bit of a pain that anyone that depends on jcache embedded also needs to depend on core.
>>>>
>>>> Any more details behind this decision?
>>>>
>>>> Cheers,
>>>> --
>>>> Galder Zamarre?o
>>>> Infinispan, Red Hat
>>>>
>>>> --
>>>> SEBASTIAN ?ASKAWIEC
>>>> INFINISPAN DEVELOPER
>>>> Red Hat EMEA
>>>>
>>>
>>
>> -- 
>> SEBASTIAN ?ASKAWIEC
>> INFINISPAN DEVELOPER
>> Red Hat EMEA
>>
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From sanne at infinispan.org  Wed Jun  7 20:07:48 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 8 Jun 2017 01:07:48 +0100
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <9082710C-9758-40C5-A986-3610866DCBAC@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>
	<CAA9R9O4seehkxr=JxpeJ+h5nswT-TdUzzHKN8R+rHj+-7oj=zA@mail.gmail.com>
	<9082710C-9758-40C5-A986-3610866DCBAC@redhat.com>
Message-ID: <CAFm4XO3CcwnpuJXcGKNV+20BS8UV7e3+OFCkmmYT7Cxo9U49dQ@mail.gmail.com>

On 7 June 2017 at 16:48, Galder Zamarre?o <galder at redhat.com> wrote:
> What has changed is that St?phane has made a very good point which I had not realised:
>
> Making core provided dependency means a JCache dependency user needs to add core dependency on top of that, which reduces usability. Core jar is not a provided dependency of JCache, it's a normal dependency. I don't think provided dependencies should be used to get around uber jar dependency issues.
>
> IMO, the bigger usability issue is the fact that uber jars are available as Maven dependencies. Uber jars should just not be distributed as Maven dependencies. They should just be put somewhere else but not in Maven. That'd way we'd avoid the problem in the first place.
>
> In the mean time, I think we should:
>
> * Move back to having normal dependencies for core in JCache (and Spring too, if it applies)

+1

> * Go through our examples and avoid using uber jar dependencies.

+10

> Then, explore the idea above of not having uber jars as Maven dependencies.

+100

They are designed for developers on outdated platforms. Let's ship
them exclusively on floppy disks?

>
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 7 Jun 2017, at 12:23, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>>
>> We discussed it number of times, including (but probably not limited to):
>>       ? http://lists.jboss.org/pipermail/infinispan-dev/2016-February/016414.html
>>       ? http://lists.jboss.org/pipermail/infinispan-dev/2016-March/016490.html
>> You might also want to look into the internal lists...
>>
>> The biggest question - has anything changed? Do we have any other idea?
>>
>> On Wed, Jun 7, 2017 at 12:05 PM Galder Zamarre?o <galder at redhat.com> wrote:
>> Moreover:
>>
>> * The experience of Maven users should never be penalised by uber jars. Uber jar users should be a minority compared with Maven/Gradle...etc users that have dependency engines in place to select which components they want to depend on.
>>
>> Cheers,
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>> > On 7 Jun 2017, at 12:02, Galder Zamarre?o <galder at redhat.com> wrote:
>> >
>> > As far as I see it:
>> >
>> > * infinispan-embedded should never be a dependency in a Maven project.
>> >
>> > * No uber jars should really be used as Maven dependencies because all the exclusion that fine grained dependencies allow you to do goes out of the window when all classes are inside a jar. This is not just theory, I've personally had such issues.
>> >
>> > * Uber jars are designed for Ant or other build tool users that don't have a dependency resolution engine in place.
>> >
>> > Cheers,
>> >
>> > p.s. I thought we had already discussed this before?
>> > --
>> > Galder Zamarre?o
>> > Infinispan, Red Hat
>> >
>> >> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>> >>
>> >> Hey,
>> >>
>> >> The change was introduced by this commit [1] and relates to this JIRAs [2][3]. The root cause is in [3].
>> >>
>> >> Imagine a scenario where you add JCache module to your together infinispan-embedded. If your classpath was constructed in such a way that infinispan-embedded was before infinispan-core (classpath is scanned from left to right in standalone apps), we could get a relocated (uber jars move some classes into other packages) logger. That caused class mismatch errors. It is worth to mention that it will happen to all relocated classes, logger was just an example. And we need to relocate them, since a user might want to use his own, newer version of DMR or any other library. So there's no perfect solution here.
>> >>
>> >> Now a lot of time passed since then and we changed quite a few things. So this topic probably needs to be revisited.
>> >>
>> >> So the first question that we should ask, shall we allow putting jcache and infinispan-embedded together on the classpath. If the answer is yes, I believe it should stay as it is (since the user always have a choice whether he wants to use jcache with or without uber jar). The same question needs to be asked for Spring modules as well as all cache stores. The behavior needs to be consistent across all those modules.
>> >>
>> >> If the answer is no (which is also valid because jcache is already present in embedded uber jar), we should migrate all JBoss Logging references to Infinispan Common Logging (as Tristan did here [4]) and we can make infinispan-core as a compile time dependency to jcache. Even though migrating to Infinispan logger is not necessary, this way we won't break users app which used infinispan-embedded + jcache approach. Of course the same applies to Spring and Cache stores modules.
>> >>
>> >> I think the latter approach deserves some exploration. I would vote for moving that way.
>> >>
>> >> Thanks,
>> >> Sebastian
>> >>
>> >> [1] https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>> >> [2] https://issues.jboss.org/browse/ISPN-6295
>> >> [3] https://issues.jboss.org/browse/ISPN-6132
>> >> [4] https://github.com/infinispan/infinispan/pull/4140/files
>> >>
>> >>
>> >> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com> wrote:
>> >> Hi all,
>> >>
>> >> Re: https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>> >>
>> >> St?phane makes a good point there, why did we make core provided dependency? It does feel a bit of a pain that anyone that depends on jcache embedded also needs to depend on core.
>> >>
>> >> Any more details behind this decision?
>> >>
>> >> Cheers,
>> >> --
>> >> Galder Zamarre?o
>> >> Infinispan, Red Hat
>> >>
>> >> --
>> >> SEBASTIAN ?ASKAWIEC
>> >> INFINISPAN DEVELOPER
>> >> Red Hat EMEA
>> >>
>> >
>>
>> --
>> SEBASTIAN ?ASKAWIEC
>> INFINISPAN DEVELOPER
>> Red Hat EMEA
>>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From ttarrant at redhat.com  Thu Jun  8 04:05:08 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Thu, 8 Jun 2017 10:05:08 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
 dependency
In-Reply-To: <CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
Message-ID: <6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>

I think we should turn off maven deployment for uber jars.

Tristan

On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
> On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com 
> <mailto:galder at redhat.com>> wrote:
> 
>     As far as I see it:
> 
>     * infinispan-embedded should never be a dependency in a Maven project.
> 
>     * No uber jars should really be used as Maven dependencies because
>     all the exclusion that fine grained dependencies allow you to do
>     goes out of the window when all classes are inside a jar. This is
>     not just theory, I've personally had such issues.
> 
>     * Uber jars are designed for Ant or other build tool users that
>     don't have a dependency resolution engine in place.
> 
>     Cheers,
> 
>     p.s. I thought we had already discussed this before?
> 
> 
> 
> I totally agree. In addition, uberjars should not be an osgi bundle or a 
> jboss module, for similar reasons.
> 
> P.S: Even Ant has a dependency mgmt available, which is Ivy.
> 
> Cheers,
> Gustavo
> 
>     --
>     Galder Zamarre?o
>     Infinispan, Red Hat
> 
>      > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
>     <mailto:slaskawi at redhat.com>> wrote:
>      >
>      > Hey,
>      >
>      > The change was introduced by this commit [1] and relates to this
>     JIRAs [2][3]. The root cause is in [3].
>      >
>      > Imagine a scenario where you add JCache module to your together
>     infinispan-embedded. If your classpath was constructed in such a way
>     that infinispan-embedded was before infinispan-core (classpath is
>     scanned from left to right in standalone apps), we could get a
>     relocated (uber jars move some classes into other packages) logger.
>     That caused class mismatch errors. It is worth to mention that it
>     will happen to all relocated classes, logger was just an example.
>     And we need to relocate them, since a user might want to use his
>     own, newer version of DMR or any other library. So there's no
>     perfect solution here.
>      >
>      > Now a lot of time passed since then and we changed quite a few
>     things. So this topic probably needs to be revisited.
>      >
>      > So the first question that we should ask, shall we allow putting
>     jcache and infinispan-embedded together on the classpath. If the
>     answer is yes, I believe it should stay as it is (since the user
>     always have a choice whether he wants to use jcache with or without
>     uber jar). The same question needs to be asked for Spring modules as
>     well as all cache stores. The behavior needs to be consistent across
>     all those modules.
>      >
>      > If the answer is no (which is also valid because jcache is
>     already present in embedded uber jar), we should migrate all JBoss
>     Logging references to Infinispan Common Logging (as Tristan did here
>     [4]) and we can make infinispan-core as a compile time dependency to
>     jcache. Even though migrating to Infinispan logger is not necessary,
>     this way we won't break users app which used infinispan-embedded +
>     jcache approach. Of course the same applies to Spring and Cache
>     stores modules.
>      >
>      > I think the latter approach deserves some exploration. I would
>     vote for moving that way.
>      >
>      > Thanks,
>      > Sebastian
>      >
>      > [1]
>     https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>     <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
>      > [2] https://issues.jboss.org/browse/ISPN-6295
>     <https://issues.jboss.org/browse/ISPN-6295>
>      > [3] https://issues.jboss.org/browse/ISPN-6132
>     <https://issues.jboss.org/browse/ISPN-6132>
>      > [4] https://github.com/infinispan/infinispan/pull/4140/files
>     <https://github.com/infinispan/infinispan/pull/4140/files>
>      >
>      >
>      > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
>     <galder at redhat.com <mailto:galder at redhat.com>> wrote:
>      > Hi all,
>      >
>      > Re:
>     https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>     <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
>      >
>      > St?phane makes a good point there, why did we make core provided
>     dependency? It does feel a bit of a pain that anyone that depends on
>     jcache embedded also needs to depend on core.
>      >
>      > Any more details behind this decision?
>      >
>      > Cheers,
>      > --
>      > Galder Zamarre?o
>      > Infinispan, Red Hat
>      >
>      > --
>      > SEBASTIAN ?ASKAWIEC
>      > INFINISPAN DEVELOPER
>      > Red Hat EMEA
>      >
> 
> 
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From gustavo at infinispan.org  Thu Jun  8 07:25:07 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Thu, 8 Jun 2017 12:25:07 +0100
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <CAFm4XO3CcwnpuJXcGKNV+20BS8UV7e3+OFCkmmYT7Cxo9U49dQ@mail.gmail.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<60101217-83A6-4DDB-A51C-54675DECB6A8@redhat.com>
	<CAA9R9O4seehkxr=JxpeJ+h5nswT-TdUzzHKN8R+rHj+-7oj=zA@mail.gmail.com>
	<9082710C-9758-40C5-A986-3610866DCBAC@redhat.com>
	<CAFm4XO3CcwnpuJXcGKNV+20BS8UV7e3+OFCkmmYT7Cxo9U49dQ@mail.gmail.com>
Message-ID: <CAH8Ud1Fze+nOinR+6CO185yOJyzwqyw1LGOuMoQ7G3bO2zUnmA@mail.gmail.com>

On Thu, Jun 8, 2017 at 1:07 AM, Sanne Grinovero <sanne at infinispan.org>
wrote:

> On 7 June 2017 at 16:48, Galder Zamarre?o <galder at redhat.com> wrote:
> > What has changed is that St?phane has made a very good point which I had
> not realised:
> >
> > Making core provided dependency means a JCache dependency user needs to
> add core dependency on top of that, which reduces usability. Core jar is
> not a provided dependency of JCache, it's a normal dependency. I don't
> think provided dependencies should be used to get around uber jar
> dependency issues.
> >
> > IMO, the bigger usability issue is the fact that uber jars are available
> as Maven dependencies. Uber jars should just not be distributed as Maven
> dependencies. They should just be put somewhere else but not in Maven.
> That'd way we'd avoid the problem in the first place.
> >
> > In the mean time, I think we should:
> >
> > * Move back to having normal dependencies for core in JCache (and Spring
> too, if it applies)
>
> +1
>
> > * Go through our examples and avoid using uber jar dependencies.
>
> +10
>
> > Then, explore the idea above of not having uber jars as Maven
> dependencies.
>
> +100
>
> They are designed for developers on outdated platforms. Let's ship
> them exclusively on floppy disks?
>

We should also stop depending on 3rd party uber-jars, like we do with
netty-all, due to reasons described on [1]

[1] https://github.com/netty/netty/issues/4671

Gustavo


>
> >
> > Cheers,
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> >> On 7 Jun 2017, at 12:23, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
> >>
> >> We discussed it number of times, including (but probably not limited
> to):
> >>       ? http://lists.jboss.org/pipermail/infinispan-dev/2016-
> February/016414.html
> >>       ? http://lists.jboss.org/pipermail/infinispan-dev/2016-
> March/016490.html
> >> You might also want to look into the internal lists...
> >>
> >> The biggest question - has anything changed? Do we have any other idea?
> >>
> >> On Wed, Jun 7, 2017 at 12:05 PM Galder Zamarre?o <galder at redhat.com>
> wrote:
> >> Moreover:
> >>
> >> * The experience of Maven users should never be penalised by uber jars.
> Uber jar users should be a minority compared with Maven/Gradle...etc users
> that have dependency engines in place to select which components they want
> to depend on.
> >>
> >> Cheers,
> >> --
> >> Galder Zamarre?o
> >> Infinispan, Red Hat
> >>
> >> > On 7 Jun 2017, at 12:02, Galder Zamarre?o <galder at redhat.com> wrote:
> >> >
> >> > As far as I see it:
> >> >
> >> > * infinispan-embedded should never be a dependency in a Maven project.
> >> >
> >> > * No uber jars should really be used as Maven dependencies because
> all the exclusion that fine grained dependencies allow you to do goes out
> of the window when all classes are inside a jar. This is not just theory,
> I've personally had such issues.
> >> >
> >> > * Uber jars are designed for Ant or other build tool users that don't
> have a dependency resolution engine in place.
> >> >
> >> > Cheers,
> >> >
> >> > p.s. I thought we had already discussed this before?
> >> > --
> >> > Galder Zamarre?o
> >> > Infinispan, Red Hat
> >> >
> >> >> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
> >> >>
> >> >> Hey,
> >> >>
> >> >> The change was introduced by this commit [1] and relates to this
> JIRAs [2][3]. The root cause is in [3].
> >> >>
> >> >> Imagine a scenario where you add JCache module to your together
> infinispan-embedded. If your classpath was constructed in such a way that
> infinispan-embedded was before infinispan-core (classpath is scanned from
> left to right in standalone apps), we could get a relocated (uber jars move
> some classes into other packages) logger. That caused class mismatch
> errors. It is worth to mention that it will happen to all relocated
> classes, logger was just an example. And we need to relocate them, since a
> user might want to use his own, newer version of DMR or any other library.
> So there's no perfect solution here.
> >> >>
> >> >> Now a lot of time passed since then and we changed quite a few
> things. So this topic probably needs to be revisited.
> >> >>
> >> >> So the first question that we should ask, shall we allow putting
> jcache and infinispan-embedded together on the classpath. If the answer is
> yes, I believe it should stay as it is (since the user always have a choice
> whether he wants to use jcache with or without uber jar). The same question
> needs to be asked for Spring modules as well as all cache stores. The
> behavior needs to be consistent across all those modules.
> >> >>
> >> >> If the answer is no (which is also valid because jcache is already
> present in embedded uber jar), we should migrate all JBoss Logging
> references to Infinispan Common Logging (as Tristan did here [4]) and we
> can make infinispan-core as a compile time dependency to jcache. Even
> though migrating to Infinispan logger is not necessary, this way we won't
> break users app which used infinispan-embedded + jcache approach. Of course
> the same applies to Spring and Cache stores modules.
> >> >>
> >> >> I think the latter approach deserves some exploration. I would vote
> for moving that way.
> >> >>
> >> >> Thanks,
> >> >> Sebastian
> >> >>
> >> >> [1] https://github.com/infinispan/infinispan/commit/
> 720f158cce38d86b292e1ce77b75509342007739
> >> >> [2] https://issues.jboss.org/browse/ISPN-6295
> >> >> [3] https://issues.jboss.org/browse/ISPN-6132
> >> >> [4] https://github.com/infinispan/infinispan/pull/4140/files
> >> >>
> >> >>
> >> >> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o <galder at redhat.com>
> wrote:
> >> >> Hi all,
> >> >>
> >> >> Re: https://github.com/spring-projects/spring-boot/pull/
> 9417#discussion_r120375579
> >> >>
> >> >> St?phane makes a good point there, why did we make core provided
> dependency? It does feel a bit of a pain that anyone that depends on jcache
> embedded also needs to depend on core.
> >> >>
> >> >> Any more details behind this decision?
> >> >>
> >> >> Cheers,
> >> >> --
> >> >> Galder Zamarre?o
> >> >> Infinispan, Red Hat
> >> >>
> >> >> --
> >> >> SEBASTIAN ?ASKAWIEC
> >> >> INFINISPAN DEVELOPER
> >> >> Red Hat EMEA
> >> >>
> >> >
> >>
> >> --
> >> SEBASTIAN ?ASKAWIEC
> >> INFINISPAN DEVELOPER
> >> Red Hat EMEA
> >>
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170608/0ab5a982/attachment.html 

From afield at redhat.com  Thu Jun  8 14:04:33 2017
From: afield at redhat.com (Alan Field)
Date: Thu, 8 Jun 2017 14:04:33 -0400 (EDT)
Subject: [infinispan-dev] Why JCache embedded has core as provided
 dependency
In-Reply-To: <6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
Message-ID: <977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>

Wasn't the ability to add a single dependency to a project to start using Infinispan the whole purpose for the uber jars? I'm not trying to make an argument for keeping them, because I know they have caused many issues. I just think that if we are going to remove them from Maven, then there should be a way to achieve the same easy developer on boarding that uber jars were supposed to provide. Whether this is Maven project templates, or something else doesn't matter.

Thanks,
Alan

----- Original Message -----
> From: "Tristan Tarrant" <ttarrant at redhat.com>
> To: infinispan-dev at lists.jboss.org
> Sent: Thursday, June 8, 2017 4:05:08 AM
> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided dependency
> 
> I think we should turn off maven deployment for uber jars.
> 
> Tristan
> 
> On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
> > On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
> > <mailto:galder at redhat.com>> wrote:
> > 
> >     As far as I see it:
> > 
> >     * infinispan-embedded should never be a dependency in a Maven project.
> > 
> >     * No uber jars should really be used as Maven dependencies because
> >     all the exclusion that fine grained dependencies allow you to do
> >     goes out of the window when all classes are inside a jar. This is
> >     not just theory, I've personally had such issues.
> > 
> >     * Uber jars are designed for Ant or other build tool users that
> >     don't have a dependency resolution engine in place.
> > 
> >     Cheers,
> > 
> >     p.s. I thought we had already discussed this before?
> > 
> > 
> > 
> > I totally agree. In addition, uberjars should not be an osgi bundle or a
> > jboss module, for similar reasons.
> > 
> > P.S: Even Ant has a dependency mgmt available, which is Ivy.
> > 
> > Cheers,
> > Gustavo
> > 
> >     --
> >     Galder Zamarre?o
> >     Infinispan, Red Hat
> > 
> >      > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
> >     <mailto:slaskawi at redhat.com>> wrote:
> >      >
> >      > Hey,
> >      >
> >      > The change was introduced by this commit [1] and relates to this
> >     JIRAs [2][3]. The root cause is in [3].
> >      >
> >      > Imagine a scenario where you add JCache module to your together
> >     infinispan-embedded. If your classpath was constructed in such a way
> >     that infinispan-embedded was before infinispan-core (classpath is
> >     scanned from left to right in standalone apps), we could get a
> >     relocated (uber jars move some classes into other packages) logger.
> >     That caused class mismatch errors. It is worth to mention that it
> >     will happen to all relocated classes, logger was just an example.
> >     And we need to relocate them, since a user might want to use his
> >     own, newer version of DMR or any other library. So there's no
> >     perfect solution here.
> >      >
> >      > Now a lot of time passed since then and we changed quite a few
> >     things. So this topic probably needs to be revisited.
> >      >
> >      > So the first question that we should ask, shall we allow putting
> >     jcache and infinispan-embedded together on the classpath. If the
> >     answer is yes, I believe it should stay as it is (since the user
> >     always have a choice whether he wants to use jcache with or without
> >     uber jar). The same question needs to be asked for Spring modules as
> >     well as all cache stores. The behavior needs to be consistent across
> >     all those modules.
> >      >
> >      > If the answer is no (which is also valid because jcache is
> >     already present in embedded uber jar), we should migrate all JBoss
> >     Logging references to Infinispan Common Logging (as Tristan did here
> >     [4]) and we can make infinispan-core as a compile time dependency to
> >     jcache. Even though migrating to Infinispan logger is not necessary,
> >     this way we won't break users app which used infinispan-embedded +
> >     jcache approach. Of course the same applies to Spring and Cache
> >     stores modules.
> >      >
> >      > I think the latter approach deserves some exploration. I would
> >     vote for moving that way.
> >      >
> >      > Thanks,
> >      > Sebastian
> >      >
> >      > [1]
> >     https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
> >     <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
> >      > [2] https://issues.jboss.org/browse/ISPN-6295
> >     <https://issues.jboss.org/browse/ISPN-6295>
> >      > [3] https://issues.jboss.org/browse/ISPN-6132
> >     <https://issues.jboss.org/browse/ISPN-6132>
> >      > [4] https://github.com/infinispan/infinispan/pull/4140/files
> >     <https://github.com/infinispan/infinispan/pull/4140/files>
> >      >
> >      >
> >      > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
> >     <galder at redhat.com <mailto:galder at redhat.com>> wrote:
> >      > Hi all,
> >      >
> >      > Re:
> >     https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
> >     <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
> >      >
> >      > St?phane makes a good point there, why did we make core provided
> >     dependency? It does feel a bit of a pain that anyone that depends on
> >     jcache embedded also needs to depend on core.
> >      >
> >      > Any more details behind this decision?
> >      >
> >      > Cheers,
> >      > --
> >      > Galder Zamarre?o
> >      > Infinispan, Red Hat
> >      >
> >      > --
> >      > SEBASTIAN ?ASKAWIEC
> >      > INFINISPAN DEVELOPER
> >      > Red Hat EMEA
> >      >
> > 
> > 
> >     _______________________________________________
> >     infinispan-dev mailing list
> >     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
> >     https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
> > 
> > 
> > 
> > 
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > 
> 
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From slaskawi at redhat.com  Fri Jun  9 02:37:13 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Fri, 09 Jun 2017 06:37:13 +0000
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
Message-ID: <CAA9R9O7f0uTiAJbWgjK6PdUikqmZ87FPniE280Z4Mu8y-YbkSQ@mail.gmail.com>

I agree with Alan here. Maven Central is a free "download area", so I
wouldn't give it up for free. BTW, what is the point of creating and not
shipping them?

I would lean towards to removing them completely or limiting the number of
use cases to the minimum e.g. we shouldn't support using
infinispan-embedded and jcache; if jcache is essential it should be inside
infinispan-embedded; the same for Spring integration modules - either we
should put them in uber jars or say that you can use Spring integration
with small jars.

On Fri, Jun 9, 2017 at 5:05 AM Alan Field <afield at redhat.com> wrote:

> Wasn't the ability to add a single dependency to a project to start using
> Infinispan the whole purpose for the uber jars? I'm not trying to make an
> argument for keeping them, because I know they have caused many issues. I
> just think that if we are going to remove them from Maven, then there
> should be a way to achieve the same easy developer on boarding that uber
> jars were supposed to provide. Whether this is Maven project templates, or
> something else doesn't matter.
>
> Thanks,
> Alan
>
> ----- Original Message -----
> > From: "Tristan Tarrant" <ttarrant at redhat.com>
> > To: infinispan-dev at lists.jboss.org
> > Sent: Thursday, June 8, 2017 4:05:08 AM
> > Subject: Re: [infinispan-dev] Why JCache embedded has core as provided
> dependency
> >
> > I think we should turn off maven deployment for uber jars.
> >
> > Tristan
> >
> > On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
> > > On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
> > > <mailto:galder at redhat.com>> wrote:
> > >
> > >     As far as I see it:
> > >
> > >     * infinispan-embedded should never be a dependency in a Maven
> project.
> > >
> > >     * No uber jars should really be used as Maven dependencies because
> > >     all the exclusion that fine grained dependencies allow you to do
> > >     goes out of the window when all classes are inside a jar. This is
> > >     not just theory, I've personally had such issues.
> > >
> > >     * Uber jars are designed for Ant or other build tool users that
> > >     don't have a dependency resolution engine in place.
> > >
> > >     Cheers,
> > >
> > >     p.s. I thought we had already discussed this before?
> > >
> > >
> > >
> > > I totally agree. In addition, uberjars should not be an osgi bundle or
> a
> > > jboss module, for similar reasons.
> > >
> > > P.S: Even Ant has a dependency mgmt available, which is Ivy.
> > >
> > > Cheers,
> > > Gustavo
> > >
> > >     --
> > >     Galder Zamarre?o
> > >     Infinispan, Red Hat
> > >
> > >      > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <
> slaskawi at redhat.com
> > >     <mailto:slaskawi at redhat.com>> wrote:
> > >      >
> > >      > Hey,
> > >      >
> > >      > The change was introduced by this commit [1] and relates to this
> > >     JIRAs [2][3]. The root cause is in [3].
> > >      >
> > >      > Imagine a scenario where you add JCache module to your together
> > >     infinispan-embedded. If your classpath was constructed in such a
> way
> > >     that infinispan-embedded was before infinispan-core (classpath is
> > >     scanned from left to right in standalone apps), we could get a
> > >     relocated (uber jars move some classes into other packages) logger.
> > >     That caused class mismatch errors. It is worth to mention that it
> > >     will happen to all relocated classes, logger was just an example.
> > >     And we need to relocate them, since a user might want to use his
> > >     own, newer version of DMR or any other library. So there's no
> > >     perfect solution here.
> > >      >
> > >      > Now a lot of time passed since then and we changed quite a few
> > >     things. So this topic probably needs to be revisited.
> > >      >
> > >      > So the first question that we should ask, shall we allow putting
> > >     jcache and infinispan-embedded together on the classpath. If the
> > >     answer is yes, I believe it should stay as it is (since the user
> > >     always have a choice whether he wants to use jcache with or without
> > >     uber jar). The same question needs to be asked for Spring modules
> as
> > >     well as all cache stores. The behavior needs to be consistent
> across
> > >     all those modules.
> > >      >
> > >      > If the answer is no (which is also valid because jcache is
> > >     already present in embedded uber jar), we should migrate all JBoss
> > >     Logging references to Infinispan Common Logging (as Tristan did
> here
> > >     [4]) and we can make infinispan-core as a compile time dependency
> to
> > >     jcache. Even though migrating to Infinispan logger is not
> necessary,
> > >     this way we won't break users app which used infinispan-embedded +
> > >     jcache approach. Of course the same applies to Spring and Cache
> > >     stores modules.
> > >      >
> > >      > I think the latter approach deserves some exploration. I would
> > >     vote for moving that way.
> > >      >
> > >      > Thanks,
> > >      > Sebastian
> > >      >
> > >      > [1]
> > >
> https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
> > >     <
> https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
> >
> > >      > [2] https://issues.jboss.org/browse/ISPN-6295
> > >     <https://issues.jboss.org/browse/ISPN-6295>
> > >      > [3] https://issues.jboss.org/browse/ISPN-6132
> > >     <https://issues.jboss.org/browse/ISPN-6132>
> > >      > [4] https://github.com/infinispan/infinispan/pull/4140/files
> > >     <https://github.com/infinispan/infinispan/pull/4140/files>
> > >      >
> > >      >
> > >      > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
> > >     <galder at redhat.com <mailto:galder at redhat.com>> wrote:
> > >      > Hi all,
> > >      >
> > >      > Re:
> > >
> https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
> > >     <
> https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
> >
> > >      >
> > >      > St?phane makes a good point there, why did we make core provided
> > >     dependency? It does feel a bit of a pain that anyone that depends
> on
> > >     jcache embedded also needs to depend on core.
> > >      >
> > >      > Any more details behind this decision?
> > >      >
> > >      > Cheers,
> > >      > --
> > >      > Galder Zamarre?o
> > >      > Infinispan, Red Hat
> > >      >
> > >      > --
> > >      > SEBASTIAN ?ASKAWIEC
> > >      > INFINISPAN DEVELOPER
> > >      > Red Hat EMEA
> > >      >
> > >
> > >
> > >     _______________________________________________
> > >     infinispan-dev mailing list
> > >     infinispan-dev at lists.jboss.org <mailto:
> infinispan-dev at lists.jboss.org>
> > >     https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > infinispan-dev mailing list
> > > infinispan-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >
> >
> > --
> > Tristan Tarrant
> > Infinispan Lead
> > JBoss, a division of Red Hat
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170609/d6dc73ed/attachment.html 

From rvansa at redhat.com  Fri Jun  9 03:29:44 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Fri, 9 Jun 2017 09:29:44 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
 dependency
In-Reply-To: <977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
Message-ID: <1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>

Katia has recently pointed out some usability flaws, and we have 
discussed a central point class that would allow you to explore the API: 
instead of *knowing* about org.infinispan.query.Search or 
org.infinispan.counters. EmbeddedCounterManagerFactory you'd just call

Infinispan ispn = Infinispan.newInstance();
ispn.search().someQueryMethod(...);
ispn.counters().someCounterMethod(...);
ispn.cacheManager().getCache(...);

An umbrella module that would contain this 'discovery API' would need 
all the dependencies, so that would be a perfect replacement for the 
embedded maven artifact. Shouldn't be that much of a work to hack this 
together - how do you think that should be called? infinispan-api (but 
it would be nicer to reserve this if we ever manage to create the 
'public API' module, with interfaces only), infinispan-facade, 
infinispan-surface? We could even use infinispan-embedded, but that 
would cause some confusion if we distributed infinispan-embedded uberjar 
and infinispan-embedded umbrella artifact.

Radim

On 06/08/2017 08:04 PM, Alan Field wrote:
> Wasn't the ability to add a single dependency to a project to start using Infinispan the whole purpose for the uber jars? I'm not trying to make an argument for keeping them, because I know they have caused many issues. I just think that if we are going to remove them from Maven, then there should be a way to achieve the same easy developer on boarding that uber jars were supposed to provide. Whether this is Maven project templates, or something else doesn't matter.
>
> Thanks,
> Alan
>
> ----- Original Message -----
>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>> To: infinispan-dev at lists.jboss.org
>> Sent: Thursday, June 8, 2017 4:05:08 AM
>> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided dependency
>>
>> I think we should turn off maven deployment for uber jars.
>>
>> Tristan
>>
>> On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
>>> On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
>>> <mailto:galder at redhat.com>> wrote:
>>>
>>>      As far as I see it:
>>>
>>>      * infinispan-embedded should never be a dependency in a Maven project.
>>>
>>>      * No uber jars should really be used as Maven dependencies because
>>>      all the exclusion that fine grained dependencies allow you to do
>>>      goes out of the window when all classes are inside a jar. This is
>>>      not just theory, I've personally had such issues.
>>>
>>>      * Uber jars are designed for Ant or other build tool users that
>>>      don't have a dependency resolution engine in place.
>>>
>>>      Cheers,
>>>
>>>      p.s. I thought we had already discussed this before?
>>>
>>>
>>>
>>> I totally agree. In addition, uberjars should not be an osgi bundle or a
>>> jboss module, for similar reasons.
>>>
>>> P.S: Even Ant has a dependency mgmt available, which is Ivy.
>>>
>>> Cheers,
>>> Gustavo
>>>
>>>      --
>>>      Galder Zamarre?o
>>>      Infinispan, Red Hat
>>>
>>>       > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
>>>      <mailto:slaskawi at redhat.com>> wrote:
>>>       >
>>>       > Hey,
>>>       >
>>>       > The change was introduced by this commit [1] and relates to this
>>>      JIRAs [2][3]. The root cause is in [3].
>>>       >
>>>       > Imagine a scenario where you add JCache module to your together
>>>      infinispan-embedded. If your classpath was constructed in such a way
>>>      that infinispan-embedded was before infinispan-core (classpath is
>>>      scanned from left to right in standalone apps), we could get a
>>>      relocated (uber jars move some classes into other packages) logger.
>>>      That caused class mismatch errors. It is worth to mention that it
>>>      will happen to all relocated classes, logger was just an example.
>>>      And we need to relocate them, since a user might want to use his
>>>      own, newer version of DMR or any other library. So there's no
>>>      perfect solution here.
>>>       >
>>>       > Now a lot of time passed since then and we changed quite a few
>>>      things. So this topic probably needs to be revisited.
>>>       >
>>>       > So the first question that we should ask, shall we allow putting
>>>      jcache and infinispan-embedded together on the classpath. If the
>>>      answer is yes, I believe it should stay as it is (since the user
>>>      always have a choice whether he wants to use jcache with or without
>>>      uber jar). The same question needs to be asked for Spring modules as
>>>      well as all cache stores. The behavior needs to be consistent across
>>>      all those modules.
>>>       >
>>>       > If the answer is no (which is also valid because jcache is
>>>      already present in embedded uber jar), we should migrate all JBoss
>>>      Logging references to Infinispan Common Logging (as Tristan did here
>>>      [4]) and we can make infinispan-core as a compile time dependency to
>>>      jcache. Even though migrating to Infinispan logger is not necessary,
>>>      this way we won't break users app which used infinispan-embedded +
>>>      jcache approach. Of course the same applies to Spring and Cache
>>>      stores modules.
>>>       >
>>>       > I think the latter approach deserves some exploration. I would
>>>      vote for moving that way.
>>>       >
>>>       > Thanks,
>>>       > Sebastian
>>>       >
>>>       > [1]
>>>      https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>>>      <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
>>>       > [2] https://issues.jboss.org/browse/ISPN-6295
>>>      <https://issues.jboss.org/browse/ISPN-6295>
>>>       > [3] https://issues.jboss.org/browse/ISPN-6132
>>>      <https://issues.jboss.org/browse/ISPN-6132>
>>>       > [4] https://github.com/infinispan/infinispan/pull/4140/files
>>>      <https://github.com/infinispan/infinispan/pull/4140/files>
>>>       >
>>>       >
>>>       > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
>>>      <galder at redhat.com <mailto:galder at redhat.com>> wrote:
>>>       > Hi all,
>>>       >
>>>       > Re:
>>>      https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>>>      <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
>>>       >
>>>       > St?phane makes a good point there, why did we make core provided
>>>      dependency? It does feel a bit of a pain that anyone that depends on
>>>      jcache embedded also needs to depend on core.
>>>       >
>>>       > Any more details behind this decision?
>>>       >
>>>       > Cheers,
>>>       > --
>>>       > Galder Zamarre?o
>>>       > Infinispan, Red Hat
>>>       >
>>>       > --
>>>       > SEBASTIAN ?ASKAWIEC
>>>       > INFINISPAN DEVELOPER
>>>       > Red Hat EMEA
>>>       >
>>>
>>>
>>>      _______________________________________________
>>>      infinispan-dev mailing list
>>>      infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>      <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> --
>> Tristan Tarrant
>> Infinispan Lead
>> JBoss, a division of Red Hat
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From ttarrant at redhat.com  Fri Jun  9 09:10:25 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 9 Jun 2017 15:10:25 +0200
Subject: [infinispan-dev] Test. Don't read.
Message-ID: <d07b1e84-6acc-ac4c-71a2-bf04eb02fef0@redhat.com>

I told you not to read it.

From ttarrant at redhat.com  Fri Jun  9 10:09:06 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 9 Jun 2017 16:09:06 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
 dependency
In-Reply-To: <CAA9R9O7f0uTiAJbWgjK6PdUikqmZ87FPniE280Z4Mu8y-YbkSQ@mail.gmail.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
	<CAA9R9O7f0uTiAJbWgjK6PdUikqmZ87FPniE280Z4Mu8y-YbkSQ@mail.gmail.com>
Message-ID: <6d73b980-7e21-f6f1-b5bf-1c7d1d0fdd21@redhat.com>

They would be shipped in the zip distributions.

Tristan

On 6/9/17 8:37 AM, Sebastian Laskawiec wrote:
> I agree with Alan here. Maven Central is a free "download area", so I 
> wouldn't give it up for free. BTW, what is the point of creating and not 
> shipping them?
> 
> I would lean towards to removing them completely or limiting the number 
> of use cases to the minimum e.g. we shouldn't support using 
> infinispan-embedded and jcache; if jcache is essential it should be 
> inside infinispan-embedded; the same for Spring integration modules - 
> either we should put them in uber jars or say that you can use Spring 
> integration with small jars.
> 
> On Fri, Jun 9, 2017 at 5:05 AM Alan Field <afield at redhat.com 
> <mailto:afield at redhat.com>> wrote:
> 
>     Wasn't the ability to add a single dependency to a project to start
>     using Infinispan the whole purpose for the uber jars? I'm not trying
>     to make an argument for keeping them, because I know they have
>     caused many issues. I just think that if we are going to remove them
>     from Maven, then there should be a way to achieve the same easy
>     developer on boarding that uber jars were supposed to provide.
>     Whether this is Maven project templates, or something else doesn't
>     matter.
> 
>     Thanks,
>     Alan
> 
>     ----- Original Message -----
>      > From: "Tristan Tarrant" <ttarrant at redhat.com
>     <mailto:ttarrant at redhat.com>>
>      > To: infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>      > Sent: Thursday, June 8, 2017 4:05:08 AM
>      > Subject: Re: [infinispan-dev] Why JCache embedded has core as
>     provided dependency
>      >
>      > I think we should turn off maven deployment for uber jars.
>      >
>      > Tristan
>      >
>      > On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
>      > > On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o
>     <galder at redhat.com <mailto:galder at redhat.com>
>      > > <mailto:galder at redhat.com <mailto:galder at redhat.com>>> wrote:
>      > >
>      > >     As far as I see it:
>      > >
>      > >     * infinispan-embedded should never be a dependency in a
>     Maven project.
>      > >
>      > >     * No uber jars should really be used as Maven dependencies
>     because
>      > >     all the exclusion that fine grained dependencies allow you
>     to do
>      > >     goes out of the window when all classes are inside a jar.
>     This is
>      > >     not just theory, I've personally had such issues.
>      > >
>      > >     * Uber jars are designed for Ant or other build tool users that
>      > >     don't have a dependency resolution engine in place.
>      > >
>      > >     Cheers,
>      > >
>      > >     p.s. I thought we had already discussed this before?
>      > >
>      > >
>      > >
>      > > I totally agree. In addition, uberjars should not be an osgi
>     bundle or a
>      > > jboss module, for similar reasons.
>      > >
>      > > P.S: Even Ant has a dependency mgmt available, which is Ivy.
>      > >
>      > > Cheers,
>      > > Gustavo
>      > >
>      > >     --
>      > >     Galder Zamarre?o
>      > >     Infinispan, Red Hat
>      > >
>      > >      > On 7 Jun 2017, at 11:50, Sebastian Laskawiec
>     <slaskawi at redhat.com <mailto:slaskawi at redhat.com>
>      > >     <mailto:slaskawi at redhat.com <mailto:slaskawi at redhat.com>>>
>     wrote:
>      > >      >
>      > >      > Hey,
>      > >      >
>      > >      > The change was introduced by this commit [1] and relates
>     to this
>      > >     JIRAs [2][3]. The root cause is in [3].
>      > >      >
>      > >      > Imagine a scenario where you add JCache module to your
>     together
>      > >     infinispan-embedded. If your classpath was constructed in
>     such a way
>      > >     that infinispan-embedded was before infinispan-core
>     (classpath is
>      > >     scanned from left to right in standalone apps), we could get a
>      > >     relocated (uber jars move some classes into other packages)
>     logger.
>      > >     That caused class mismatch errors. It is worth to mention
>     that it
>      > >     will happen to all relocated classes, logger was just an
>     example.
>      > >     And we need to relocate them, since a user might want to
>     use his
>      > >     own, newer version of DMR or any other library. So there's no
>      > >     perfect solution here.
>      > >      >
>      > >      > Now a lot of time passed since then and we changed quite
>     a few
>      > >     things. So this topic probably needs to be revisited.
>      > >      >
>      > >      > So the first question that we should ask, shall we allow
>     putting
>      > >     jcache and infinispan-embedded together on the classpath.
>     If the
>      > >     answer is yes, I believe it should stay as it is (since the
>     user
>      > >     always have a choice whether he wants to use jcache with or
>     without
>      > >     uber jar). The same question needs to be asked for Spring
>     modules as
>      > >     well as all cache stores. The behavior needs to be
>     consistent across
>      > >     all those modules.
>      > >      >
>      > >      > If the answer is no (which is also valid because jcache is
>      > >     already present in embedded uber jar), we should migrate
>     all JBoss
>      > >     Logging references to Infinispan Common Logging (as Tristan
>     did here
>      > >     [4]) and we can make infinispan-core as a compile time
>     dependency to
>      > >     jcache. Even though migrating to Infinispan logger is not
>     necessary,
>      > >     this way we won't break users app which used
>     infinispan-embedded +
>      > >     jcache approach. Of course the same applies to Spring and Cache
>      > >     stores modules.
>      > >      >
>      > >      > I think the latter approach deserves some exploration. I
>     would
>      > >     vote for moving that way.
>      > >      >
>      > >      > Thanks,
>      > >      > Sebastian
>      > >      >
>      > >      > [1]
>      > >
>     https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>      > >   
>       <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
>      > >      > [2] https://issues.jboss.org/browse/ISPN-6295
>      > >     <https://issues.jboss.org/browse/ISPN-6295>
>      > >      > [3] https://issues.jboss.org/browse/ISPN-6132
>      > >     <https://issues.jboss.org/browse/ISPN-6132>
>      > >      > [4] https://github.com/infinispan/infinispan/pull/4140/files
>      > >     <https://github.com/infinispan/infinispan/pull/4140/files>
>      > >      >
>      > >      >
>      > >      > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
>      > >     <galder at redhat.com <mailto:galder at redhat.com>
>     <mailto:galder at redhat.com <mailto:galder at redhat.com>>> wrote:
>      > >      > Hi all,
>      > >      >
>      > >      > Re:
>      > >
>     https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>      > >   
>       <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
>      > >      >
>      > >      > St?phane makes a good point there, why did we make core
>     provided
>      > >     dependency? It does feel a bit of a pain that anyone that
>     depends on
>      > >     jcache embedded also needs to depend on core.
>      > >      >
>      > >      > Any more details behind this decision?
>      > >      >
>      > >      > Cheers,
>      > >      > --
>      > >      > Galder Zamarre?o
>      > >      > Infinispan, Red Hat
>      > >      >
>      > >      > --
>      > >      > SEBASTIAN ?ASKAWIEC
>      > >      > INFINISPAN DEVELOPER
>      > >      > Red Hat EMEA
>      > >      >
>      > >
>      > >
>      > >     _______________________________________________
>      > >     infinispan-dev mailing list
>      > > infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>     <mailto:infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>>
>      > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>      > >     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>      > >
>      > >
>      > >
>      > >
>      > > _______________________________________________
>      > > infinispan-dev mailing list
>      > > infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>      > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>      > >
>      >
>      > --
>      > Tristan Tarrant
>      > Infinispan Lead
>      > JBoss, a division of Red Hat
>      > _______________________________________________
>      > infinispan-dev mailing list
>      > infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>      > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> -- 
> 
> SEBASTIAN?ASKAWIEC
> 
> INFINISPAN DEVELOPER
> 
> Red HatEMEA <https://www.redhat.com/>
> 
> <https://red.ht/sig>
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From ttarrant at redhat.com  Fri Jun  9 11:06:47 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 9 Jun 2017 17:06:47 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
 dependency
In-Reply-To: <1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
	<1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>
Message-ID: <377a8dea-3ea5-6650-40f8-fb1877569d89@redhat.com>

On 6/9/17 9:29 AM, Radim Vansa wrote:
> An umbrella module that would contain this 'discovery API' would need
> all the dependencies, so that would be a perfect replacement for the
> embedded maven artifact. Shouldn't be that much of a work to hack this
> together - how do you think that should be called? infinispan-api (but
> it would be nicer to reserve this if we ever manage to create the
> 'public API' module, with interfaces only), infinispan-facade,
> infinispan-surface? We could even use infinispan-embedded, but that
> would cause some confusion if we distributed infinispan-embedded uberjar
> and infinispan-embedded umbrella artifact.
org.infinispan:infinispan

Nothing else.

Tristan

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From afield at redhat.com  Fri Jun  9 11:13:05 2017
From: afield at redhat.com (Alan Field)
Date: Fri, 9 Jun 2017 11:13:05 -0400 (EDT)
Subject: [infinispan-dev] Why JCache embedded has core as provided
 dependency
In-Reply-To: <1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
	<1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>
Message-ID: <1310904907.20805805.1497021185755.JavaMail.zimbra@redhat.com>

I really like this idea. It is similar to one of the solutions hinted at in the Netty issue that Gustavo pointed at. [1] The suggestion there was to replace their current uber jar that contains all of the shaded dependencies with a JAR that just depended on everything. Then you would get the benefit of being able to use a single Maven dependency to pull in all of the JAR files without the issues that shading brings. 

Thanks,
Alan 

[1] https://github.com/netty/netty/issues/4671

----- Original Message -----
> From: "Radim Vansa" <rvansa at redhat.com>
> To: infinispan-dev at lists.jboss.org
> Sent: Friday, June 9, 2017 3:29:44 AM
> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided dependency
> 
> Katia has recently pointed out some usability flaws, and we have
> discussed a central point class that would allow you to explore the API:
> instead of *knowing* about org.infinispan.query.Search or
> org.infinispan.counters. EmbeddedCounterManagerFactory you'd just call
> 
> Infinispan ispn = Infinispan.newInstance();
> ispn.search().someQueryMethod(...);
> ispn.counters().someCounterMethod(...);
> ispn.cacheManager().getCache(...);
> 
> An umbrella module that would contain this 'discovery API' would need
> all the dependencies, so that would be a perfect replacement for the
> embedded maven artifact. Shouldn't be that much of a work to hack this
> together - how do you think that should be called? infinispan-api (but
> it would be nicer to reserve this if we ever manage to create the
> 'public API' module, with interfaces only), infinispan-facade,
> infinispan-surface? We could even use infinispan-embedded, but that
> would cause some confusion if we distributed infinispan-embedded uberjar
> and infinispan-embedded umbrella artifact.
> 
> Radim
> 
> On 06/08/2017 08:04 PM, Alan Field wrote:
> > Wasn't the ability to add a single dependency to a project to start using
> > Infinispan the whole purpose for the uber jars? I'm not trying to make an
> > argument for keeping them, because I know they have caused many issues. I
> > just think that if we are going to remove them from Maven, then there
> > should be a way to achieve the same easy developer on boarding that uber
> > jars were supposed to provide. Whether this is Maven project templates, or
> > something else doesn't matter.
> >
> > Thanks,
> > Alan
> >
> > ----- Original Message -----
> >> From: "Tristan Tarrant" <ttarrant at redhat.com>
> >> To: infinispan-dev at lists.jboss.org
> >> Sent: Thursday, June 8, 2017 4:05:08 AM
> >> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided
> >> dependency
> >>
> >> I think we should turn off maven deployment for uber jars.
> >>
> >> Tristan
> >>
> >> On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
> >>> On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
> >>> <mailto:galder at redhat.com>> wrote:
> >>>
> >>>      As far as I see it:
> >>>
> >>>      * infinispan-embedded should never be a dependency in a Maven
> >>>      project.
> >>>
> >>>      * No uber jars should really be used as Maven dependencies because
> >>>      all the exclusion that fine grained dependencies allow you to do
> >>>      goes out of the window when all classes are inside a jar. This is
> >>>      not just theory, I've personally had such issues.
> >>>
> >>>      * Uber jars are designed for Ant or other build tool users that
> >>>      don't have a dependency resolution engine in place.
> >>>
> >>>      Cheers,
> >>>
> >>>      p.s. I thought we had already discussed this before?
> >>>
> >>>
> >>>
> >>> I totally agree. In addition, uberjars should not be an osgi bundle or a
> >>> jboss module, for similar reasons.
> >>>
> >>> P.S: Even Ant has a dependency mgmt available, which is Ivy.
> >>>
> >>> Cheers,
> >>> Gustavo
> >>>
> >>>      --
> >>>      Galder Zamarre?o
> >>>      Infinispan, Red Hat
> >>>
> >>>       > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
> >>>      <mailto:slaskawi at redhat.com>> wrote:
> >>>       >
> >>>       > Hey,
> >>>       >
> >>>       > The change was introduced by this commit [1] and relates to this
> >>>      JIRAs [2][3]. The root cause is in [3].
> >>>       >
> >>>       > Imagine a scenario where you add JCache module to your together
> >>>      infinispan-embedded. If your classpath was constructed in such a way
> >>>      that infinispan-embedded was before infinispan-core (classpath is
> >>>      scanned from left to right in standalone apps), we could get a
> >>>      relocated (uber jars move some classes into other packages) logger.
> >>>      That caused class mismatch errors. It is worth to mention that it
> >>>      will happen to all relocated classes, logger was just an example.
> >>>      And we need to relocate them, since a user might want to use his
> >>>      own, newer version of DMR or any other library. So there's no
> >>>      perfect solution here.
> >>>       >
> >>>       > Now a lot of time passed since then and we changed quite a few
> >>>      things. So this topic probably needs to be revisited.
> >>>       >
> >>>       > So the first question that we should ask, shall we allow putting
> >>>      jcache and infinispan-embedded together on the classpath. If the
> >>>      answer is yes, I believe it should stay as it is (since the user
> >>>      always have a choice whether he wants to use jcache with or without
> >>>      uber jar). The same question needs to be asked for Spring modules as
> >>>      well as all cache stores. The behavior needs to be consistent across
> >>>      all those modules.
> >>>       >
> >>>       > If the answer is no (which is also valid because jcache is
> >>>      already present in embedded uber jar), we should migrate all JBoss
> >>>      Logging references to Infinispan Common Logging (as Tristan did here
> >>>      [4]) and we can make infinispan-core as a compile time dependency to
> >>>      jcache. Even though migrating to Infinispan logger is not necessary,
> >>>      this way we won't break users app which used infinispan-embedded +
> >>>      jcache approach. Of course the same applies to Spring and Cache
> >>>      stores modules.
> >>>       >
> >>>       > I think the latter approach deserves some exploration. I would
> >>>      vote for moving that way.
> >>>       >
> >>>       > Thanks,
> >>>       > Sebastian
> >>>       >
> >>>       > [1]
> >>>      https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
> >>>      <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
> >>>       > [2] https://issues.jboss.org/browse/ISPN-6295
> >>>      <https://issues.jboss.org/browse/ISPN-6295>
> >>>       > [3] https://issues.jboss.org/browse/ISPN-6132
> >>>      <https://issues.jboss.org/browse/ISPN-6132>
> >>>       > [4] https://github.com/infinispan/infinispan/pull/4140/files
> >>>      <https://github.com/infinispan/infinispan/pull/4140/files>
> >>>       >
> >>>       >
> >>>       > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
> >>>      <galder at redhat.com <mailto:galder at redhat.com>> wrote:
> >>>       > Hi all,
> >>>       >
> >>>       > Re:
> >>>      https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
> >>>      <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
> >>>       >
> >>>       > St?phane makes a good point there, why did we make core provided
> >>>      dependency? It does feel a bit of a pain that anyone that depends on
> >>>      jcache embedded also needs to depend on core.
> >>>       >
> >>>       > Any more details behind this decision?
> >>>       >
> >>>       > Cheers,
> >>>       > --
> >>>       > Galder Zamarre?o
> >>>       > Infinispan, Red Hat
> >>>       >
> >>>       > --
> >>>       > SEBASTIAN ?ASKAWIEC
> >>>       > INFINISPAN DEVELOPER
> >>>       > Red Hat EMEA
> >>>       >
> >>>
> >>>
> >>>      _______________________________________________
> >>>      infinispan-dev mailing list
> >>>      infinispan-dev at lists.jboss.org
> >>>      <mailto:infinispan-dev at lists.jboss.org>
> >>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>      <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >> --
> >> Tristan Tarrant
> >> Infinispan Lead
> >> JBoss, a division of Red Hat
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From dereed at redhat.com  Fri Jun  9 13:05:55 2017
From: dereed at redhat.com (Dennis Reed)
Date: Fri, 9 Jun 2017 12:05:55 -0500
Subject: [infinispan-dev] Why JCache embedded has core as provided
 dependency
In-Reply-To: <977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
Message-ID: <88228f9b-fcd7-30d5-1848-7537039c484b@redhat.com>

Uber jars were never needed for maven.

Their only purpose is to be able to add a single dependency in other
build systems that don't already have automatic dependency management.

-Dennis


On 06/08/2017 01:04 PM, Alan Field wrote:
> Wasn't the ability to add a single dependency to a project to start using Infinispan the whole purpose for the uber jars? I'm not trying to make an argument for keeping them, because I know they have caused many issues. I just think that if we are going to remove them from Maven, then there should be a way to achieve the same easy developer on boarding that uber jars were supposed to provide. Whether this is Maven project templates, or something else doesn't matter.
> 
> Thanks,
> Alan
> 
> ----- Original Message -----
>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>> To: infinispan-dev at lists.jboss.org
>> Sent: Thursday, June 8, 2017 4:05:08 AM
>> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided dependency
>>
>> I think we should turn off maven deployment for uber jars.
>>
>> Tristan
>>
>> On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
>>> On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
>>> <mailto:galder at redhat.com>> wrote:
>>>
>>>     As far as I see it:
>>>
>>>     * infinispan-embedded should never be a dependency in a Maven project.
>>>
>>>     * No uber jars should really be used as Maven dependencies because
>>>     all the exclusion that fine grained dependencies allow you to do
>>>     goes out of the window when all classes are inside a jar. This is
>>>     not just theory, I've personally had such issues.
>>>
>>>     * Uber jars are designed for Ant or other build tool users that
>>>     don't have a dependency resolution engine in place.
>>>
>>>     Cheers,
>>>
>>>     p.s. I thought we had already discussed this before?
>>>
>>>
>>>
>>> I totally agree. In addition, uberjars should not be an osgi bundle or a
>>> jboss module, for similar reasons.
>>>
>>> P.S: Even Ant has a dependency mgmt available, which is Ivy.
>>>
>>> Cheers,
>>> Gustavo
>>>
>>>     --
>>>     Galder Zamarre?o
>>>     Infinispan, Red Hat
>>>
>>>      > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
>>>     <mailto:slaskawi at redhat.com>> wrote:
>>>      >
>>>      > Hey,
>>>      >
>>>      > The change was introduced by this commit [1] and relates to this
>>>     JIRAs [2][3]. The root cause is in [3].
>>>      >
>>>      > Imagine a scenario where you add JCache module to your together
>>>     infinispan-embedded. If your classpath was constructed in such a way
>>>     that infinispan-embedded was before infinispan-core (classpath is
>>>     scanned from left to right in standalone apps), we could get a
>>>     relocated (uber jars move some classes into other packages) logger.
>>>     That caused class mismatch errors. It is worth to mention that it
>>>     will happen to all relocated classes, logger was just an example.
>>>     And we need to relocate them, since a user might want to use his
>>>     own, newer version of DMR or any other library. So there's no
>>>     perfect solution here.
>>>      >
>>>      > Now a lot of time passed since then and we changed quite a few
>>>     things. So this topic probably needs to be revisited.
>>>      >
>>>      > So the first question that we should ask, shall we allow putting
>>>     jcache and infinispan-embedded together on the classpath. If the
>>>     answer is yes, I believe it should stay as it is (since the user
>>>     always have a choice whether he wants to use jcache with or without
>>>     uber jar). The same question needs to be asked for Spring modules as
>>>     well as all cache stores. The behavior needs to be consistent across
>>>     all those modules.
>>>      >
>>>      > If the answer is no (which is also valid because jcache is
>>>     already present in embedded uber jar), we should migrate all JBoss
>>>     Logging references to Infinispan Common Logging (as Tristan did here
>>>     [4]) and we can make infinispan-core as a compile time dependency to
>>>     jcache. Even though migrating to Infinispan logger is not necessary,
>>>     this way we won't break users app which used infinispan-embedded +
>>>     jcache approach. Of course the same applies to Spring and Cache
>>>     stores modules.
>>>      >
>>>      > I think the latter approach deserves some exploration. I would
>>>     vote for moving that way.
>>>      >
>>>      > Thanks,
>>>      > Sebastian
>>>      >
>>>      > [1]
>>>     https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>>>     <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
>>>      > [2] https://issues.jboss.org/browse/ISPN-6295
>>>     <https://issues.jboss.org/browse/ISPN-6295>
>>>      > [3] https://issues.jboss.org/browse/ISPN-6132
>>>     <https://issues.jboss.org/browse/ISPN-6132>
>>>      > [4] https://github.com/infinispan/infinispan/pull/4140/files
>>>     <https://github.com/infinispan/infinispan/pull/4140/files>
>>>      >
>>>      >
>>>      > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
>>>     <galder at redhat.com <mailto:galder at redhat.com>> wrote:
>>>      > Hi all,
>>>      >
>>>      > Re:
>>>     https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>>>     <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
>>>      >
>>>      > St?phane makes a good point there, why did we make core provided
>>>     dependency? It does feel a bit of a pain that anyone that depends on
>>>     jcache embedded also needs to depend on core.
>>>      >
>>>      > Any more details behind this decision?
>>>      >
>>>      > Cheers,
>>>      > --
>>>      > Galder Zamarre?o
>>>      > Infinispan, Red Hat
>>>      >
>>>      > --
>>>      > SEBASTIAN ?ASKAWIEC
>>>      > INFINISPAN DEVELOPER
>>>      > Red Hat EMEA
>>>      >
>>>
>>>
>>>     _______________________________________________
>>>     infinispan-dev mailing list
>>>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> --
>> Tristan Tarrant
>> Infinispan Lead
>> JBoss, a division of Red Hat
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From sanne at infinispan.org  Fri Jun  9 13:31:08 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 9 Jun 2017 18:31:08 +0100
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
	<1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>
Message-ID: <CAFm4XO0-d5f4M-JO4Z+PM=bNKu04QCfCSwc4QTuSkrfRQFo8rA@mail.gmail.com>

On 9 June 2017 at 08:29, Radim Vansa <rvansa at redhat.com> wrote:
> Katia has recently pointed out some usability flaws, and we have
> discussed a central point class that would allow you to explore the API:
> instead of *knowing* about org.infinispan.query.Search or
> org.infinispan.counters. EmbeddedCounterManagerFactory you'd just call
>
> Infinispan ispn = Infinispan.newInstance();
> ispn.search().someQueryMethod(...);
> ispn.counters().someCounterMethod(...);
> ispn.cacheManager().getCache(...);
>
> An umbrella module that would contain this 'discovery API' would need
> all the dependencies, so that would be a perfect replacement for the
> embedded maven artifact. Shouldn't be that much of a work to hack this
> together - how do you think that should be called? infinispan-api (but
> it would be nicer to reserve this if we ever manage to create the
> 'public API' module, with interfaces only), infinispan-facade,
> infinispan-surface? We could even use infinispan-embedded, but that
> would cause some confusion if we distributed infinispan-embedded uberjar
> and infinispan-embedded umbrella artifact.

+1 but.. hadn't we already proposed this? I still agree though ;)

>
> Radim
>
> On 06/08/2017 08:04 PM, Alan Field wrote:
>> Wasn't the ability to add a single dependency to a project to start using Infinispan the whole purpose for the uber jars? I'm not trying to make an argument for keeping them, because I know they have caused many issues. I just think that if we are going to remove them from Maven, then there should be a way to achieve the same easy developer on boarding that uber jars were supposed to provide. Whether this is Maven project templates, or something else doesn't matter.
>>
>> Thanks,
>> Alan
>>
>> ----- Original Message -----
>>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>>> To: infinispan-dev at lists.jboss.org
>>> Sent: Thursday, June 8, 2017 4:05:08 AM
>>> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided dependency
>>>
>>> I think we should turn off maven deployment for uber jars.
>>>
>>> Tristan
>>>
>>> On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
>>>> On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
>>>> <mailto:galder at redhat.com>> wrote:
>>>>
>>>>      As far as I see it:
>>>>
>>>>      * infinispan-embedded should never be a dependency in a Maven project.
>>>>
>>>>      * No uber jars should really be used as Maven dependencies because
>>>>      all the exclusion that fine grained dependencies allow you to do
>>>>      goes out of the window when all classes are inside a jar. This is
>>>>      not just theory, I've personally had such issues.
>>>>
>>>>      * Uber jars are designed for Ant or other build tool users that
>>>>      don't have a dependency resolution engine in place.
>>>>
>>>>      Cheers,
>>>>
>>>>      p.s. I thought we had already discussed this before?
>>>>
>>>>
>>>>
>>>> I totally agree. In addition, uberjars should not be an osgi bundle or a
>>>> jboss module, for similar reasons.
>>>>
>>>> P.S: Even Ant has a dependency mgmt available, which is Ivy.
>>>>
>>>> Cheers,
>>>> Gustavo
>>>>
>>>>      --
>>>>      Galder Zamarre?o
>>>>      Infinispan, Red Hat
>>>>
>>>>       > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
>>>>      <mailto:slaskawi at redhat.com>> wrote:
>>>>       >
>>>>       > Hey,
>>>>       >
>>>>       > The change was introduced by this commit [1] and relates to this
>>>>      JIRAs [2][3]. The root cause is in [3].
>>>>       >
>>>>       > Imagine a scenario where you add JCache module to your together
>>>>      infinispan-embedded. If your classpath was constructed in such a way
>>>>      that infinispan-embedded was before infinispan-core (classpath is
>>>>      scanned from left to right in standalone apps), we could get a
>>>>      relocated (uber jars move some classes into other packages) logger.
>>>>      That caused class mismatch errors. It is worth to mention that it
>>>>      will happen to all relocated classes, logger was just an example.
>>>>      And we need to relocate them, since a user might want to use his
>>>>      own, newer version of DMR or any other library. So there's no
>>>>      perfect solution here.
>>>>       >
>>>>       > Now a lot of time passed since then and we changed quite a few
>>>>      things. So this topic probably needs to be revisited.
>>>>       >
>>>>       > So the first question that we should ask, shall we allow putting
>>>>      jcache and infinispan-embedded together on the classpath. If the
>>>>      answer is yes, I believe it should stay as it is (since the user
>>>>      always have a choice whether he wants to use jcache with or without
>>>>      uber jar). The same question needs to be asked for Spring modules as
>>>>      well as all cache stores. The behavior needs to be consistent across
>>>>      all those modules.
>>>>       >
>>>>       > If the answer is no (which is also valid because jcache is
>>>>      already present in embedded uber jar), we should migrate all JBoss
>>>>      Logging references to Infinispan Common Logging (as Tristan did here
>>>>      [4]) and we can make infinispan-core as a compile time dependency to
>>>>      jcache. Even though migrating to Infinispan logger is not necessary,
>>>>      this way we won't break users app which used infinispan-embedded +
>>>>      jcache approach. Of course the same applies to Spring and Cache
>>>>      stores modules.
>>>>       >
>>>>       > I think the latter approach deserves some exploration. I would
>>>>      vote for moving that way.
>>>>       >
>>>>       > Thanks,
>>>>       > Sebastian
>>>>       >
>>>>       > [1]
>>>>      https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>>>>      <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
>>>>       > [2] https://issues.jboss.org/browse/ISPN-6295
>>>>      <https://issues.jboss.org/browse/ISPN-6295>
>>>>       > [3] https://issues.jboss.org/browse/ISPN-6132
>>>>      <https://issues.jboss.org/browse/ISPN-6132>
>>>>       > [4] https://github.com/infinispan/infinispan/pull/4140/files
>>>>      <https://github.com/infinispan/infinispan/pull/4140/files>
>>>>       >
>>>>       >
>>>>       > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
>>>>      <galder at redhat.com <mailto:galder at redhat.com>> wrote:
>>>>       > Hi all,
>>>>       >
>>>>       > Re:
>>>>      https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>>>>      <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
>>>>       >
>>>>       > St?phane makes a good point there, why did we make core provided
>>>>      dependency? It does feel a bit of a pain that anyone that depends on
>>>>      jcache embedded also needs to depend on core.
>>>>       >
>>>>       > Any more details behind this decision?
>>>>       >
>>>>       > Cheers,
>>>>       > --
>>>>       > Galder Zamarre?o
>>>>       > Infinispan, Red Hat
>>>>       >
>>>>       > --
>>>>       > SEBASTIAN ?ASKAWIEC
>>>>       > INFINISPAN DEVELOPER
>>>>       > Red Hat EMEA
>>>>       >
>>>>
>>>>
>>>>      _______________________________________________
>>>>      infinispan-dev mailing list
>>>>      infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>      <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>> --
>>> Tristan Tarrant
>>> Infinispan Lead
>>> JBoss, a division of Red Hat
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From afield at redhat.com  Fri Jun  9 16:27:34 2017
From: afield at redhat.com (Alan Field)
Date: Fri, 9 Jun 2017 16:27:34 -0400 (EDT)
Subject: [infinispan-dev] Test. Don't read.
In-Reply-To: <d07b1e84-6acc-ac4c-71a2-bf04eb02fef0@redhat.com>
References: <d07b1e84-6acc-ac4c-71a2-bf04eb02fef0@redhat.com>
Message-ID: <547864316.20914231.1497040054129.JavaMail.zimbra@redhat.com>

I read it, and now I am infected!

----- Original Message -----
> From: "Tristan Tarrant" <ttarrant at redhat.com>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Friday, June 9, 2017 9:10:25 AM
> Subject: [infinispan-dev] Test. Don't read.
> 
> I told you not to read it.
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From rvansa at redhat.com  Fri Jun  9 17:02:21 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Fri, 9 Jun 2017 23:02:21 +0200
Subject: [infinispan-dev] Moving functional API to core
Message-ID: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>

Hi guys,

when the functional API has been outline, the interfaces were put into 
infinispan-commons to make it possible to share these between remote 
clients and embedded use case. However, it seems that reusing this as-is 
impossible or at least impractical as we cannot send the lambdas in a 
language neutral way. In the future, we may implement a way to share 
functions between client and a server but that will most likely result 
in an interface accepting something else than Function<ReadWriteEntry, 
R>. Also, it's rather weird to have two EntryVersion interfaces.

Therefore I suggest moving org.infinispan.commons.api.functional to 
infinispan-core, package org.infinispan.api.functional

You might say that the server-side code would use the interfaces, but 
once it's running on server, it should depend on core (or core-api) - 
commons is what is shared with the client, and if the client will in 
future register a new function on the server, the user code should 
depend on core-api as well (client-hotrod itself does not have to).

If you wonder what led me to this is that I've tried to add 
SerializableFunction overloads to the FunctionalMap and found out that 
SerializableFunction et all are only in infinispan-core (for good).

Please let me know if you have objections/if there something I have missed.

Radim

-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From rvansa at redhat.com  Fri Jun  9 17:04:00 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Fri, 9 Jun 2017 23:04:00 +0200
Subject: [infinispan-dev] Test. Don't read.
In-Reply-To: <d07b1e84-6acc-ac4c-71a2-bf04eb02fef0@redhat.com>
References: <d07b1e84-6acc-ac4c-71a2-bf04eb02fef0@redhat.com>
Message-ID: <984f45b2-6936-3f2c-5b62-70ca385880d5@redhat.com>

I did not, I swear to Ctulhu!

On 06/09/2017 03:10 PM, Tristan Tarrant wrote:
> I told you not to read it.
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From sanne at infinispan.org  Sun Jun 11 14:35:51 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Sun, 11 Jun 2017 19:35:51 +0100
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <1310904907.20805805.1497021185755.JavaMail.zimbra@redhat.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
	<1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>
	<1310904907.20805805.1497021185755.JavaMail.zimbra@redhat.com>
Message-ID: <CAFm4XO2J5g8+8ZZwCxQZN+zfXrYzCeDw_C28z7vW_CVHNeuo5A@mail.gmail.com>

On 9 June 2017 at 16:13, Alan Field <afield at redhat.com> wrote:
> I really like this idea. It is similar to one of the solutions hinted at in the Netty issue that Gustavo pointed at. [1] The suggestion there was to replace their current uber jar that contains all of the shaded dependencies with a JAR that just depended on everything. Then you would get the benefit of being able to use a single Maven dependency to pull in all of the JAR files without the issues that shading brings.

The proposal is to bundle APIs only, but we'll always have tons of
optional (and provided) extension points, e.g. we can't include all
CacheStore(s), not least as many live in separate repositories.

Similarly, many "experimental modules" might provide additional APIs
which will need to be accessed like the current Query API: *decoupled*
- at least until there's an agreement for inclusion in such an
umbrella API.
This is unavoidable as someone might start working on such a module
without telling us.. incidentally to foster a community of extensions
I'd rather have our own API showcase how it works.

Also some Java EE APIs - e.g. the JTA API - should never be included;
that's "best practice" in community standards - ugly and we could
complain that Maven could deal better with it, containers could also
deal better with it, but that's how it is.

So it's an improvement in the API but it's not sorting out the
problems for people not familiar with dependency management. And I
believe that's ok! Just making sure we're agreeing on the goals and
non-goals.

Considering it's a very minor improvement in usability I'm not sold
about this direction: you'll have ONE MORE JAR and one more layer of
silly indirection to maintain. You might state it's trivial but I
don't believe that, as the definition of which components to include
is necessarily going to be fuild, makign this "API" more prone to wild
changes.

The one liner to get a reference to the SearchFactory is no big deal
and could be solved better by having CDI and Spring extensions - while
maintaining good decoupling - but if it's the path to convince you all
to take out the uber jars then by all means do it right now ;)

Thanks,
Sanne

>
> Thanks,
> Alan
>
> [1] https://github.com/netty/netty/issues/4671
>
> ----- Original Message -----
>> From: "Radim Vansa" <rvansa at redhat.com>
>> To: infinispan-dev at lists.jboss.org
>> Sent: Friday, June 9, 2017 3:29:44 AM
>> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided dependency
>>
>> Katia has recently pointed out some usability flaws, and we have
>> discussed a central point class that would allow you to explore the API:
>> instead of *knowing* about org.infinispan.query.Search or
>> org.infinispan.counters. EmbeddedCounterManagerFactory you'd just call
>>
>> Infinispan ispn = Infinispan.newInstance();
>> ispn.search().someQueryMethod(...);
>> ispn.counters().someCounterMethod(...);
>> ispn.cacheManager().getCache(...);
>>
>> An umbrella module that would contain this 'discovery API' would need
>> all the dependencies, so that would be a perfect replacement for the
>> embedded maven artifact. Shouldn't be that much of a work to hack this
>> together - how do you think that should be called? infinispan-api (but
>> it would be nicer to reserve this if we ever manage to create the
>> 'public API' module, with interfaces only), infinispan-facade,
>> infinispan-surface? We could even use infinispan-embedded, but that
>> would cause some confusion if we distributed infinispan-embedded uberjar
>> and infinispan-embedded umbrella artifact.
>>
>> Radim
>>
>> On 06/08/2017 08:04 PM, Alan Field wrote:
>> > Wasn't the ability to add a single dependency to a project to start using
>> > Infinispan the whole purpose for the uber jars? I'm not trying to make an
>> > argument for keeping them, because I know they have caused many issues. I
>> > just think that if we are going to remove them from Maven, then there
>> > should be a way to achieve the same easy developer on boarding that uber
>> > jars were supposed to provide. Whether this is Maven project templates, or
>> > something else doesn't matter.
>> >
>> > Thanks,
>> > Alan
>> >
>> > ----- Original Message -----
>> >> From: "Tristan Tarrant" <ttarrant at redhat.com>
>> >> To: infinispan-dev at lists.jboss.org
>> >> Sent: Thursday, June 8, 2017 4:05:08 AM
>> >> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided
>> >> dependency
>> >>
>> >> I think we should turn off maven deployment for uber jars.
>> >>
>> >> Tristan
>> >>
>> >> On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
>> >>> On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
>> >>> <mailto:galder at redhat.com>> wrote:
>> >>>
>> >>>      As far as I see it:
>> >>>
>> >>>      * infinispan-embedded should never be a dependency in a Maven
>> >>>      project.
>> >>>
>> >>>      * No uber jars should really be used as Maven dependencies because
>> >>>      all the exclusion that fine grained dependencies allow you to do
>> >>>      goes out of the window when all classes are inside a jar. This is
>> >>>      not just theory, I've personally had such issues.
>> >>>
>> >>>      * Uber jars are designed for Ant or other build tool users that
>> >>>      don't have a dependency resolution engine in place.
>> >>>
>> >>>      Cheers,
>> >>>
>> >>>      p.s. I thought we had already discussed this before?
>> >>>
>> >>>
>> >>>
>> >>> I totally agree. In addition, uberjars should not be an osgi bundle or a
>> >>> jboss module, for similar reasons.
>> >>>
>> >>> P.S: Even Ant has a dependency mgmt available, which is Ivy.
>> >>>
>> >>> Cheers,
>> >>> Gustavo
>> >>>
>> >>>      --
>> >>>      Galder Zamarre?o
>> >>>      Infinispan, Red Hat
>> >>>
>> >>>       > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
>> >>>      <mailto:slaskawi at redhat.com>> wrote:
>> >>>       >
>> >>>       > Hey,
>> >>>       >
>> >>>       > The change was introduced by this commit [1] and relates to this
>> >>>      JIRAs [2][3]. The root cause is in [3].
>> >>>       >
>> >>>       > Imagine a scenario where you add JCache module to your together
>> >>>      infinispan-embedded. If your classpath was constructed in such a way
>> >>>      that infinispan-embedded was before infinispan-core (classpath is
>> >>>      scanned from left to right in standalone apps), we could get a
>> >>>      relocated (uber jars move some classes into other packages) logger.
>> >>>      That caused class mismatch errors. It is worth to mention that it
>> >>>      will happen to all relocated classes, logger was just an example.
>> >>>      And we need to relocate them, since a user might want to use his
>> >>>      own, newer version of DMR or any other library. So there's no
>> >>>      perfect solution here.
>> >>>       >
>> >>>       > Now a lot of time passed since then and we changed quite a few
>> >>>      things. So this topic probably needs to be revisited.
>> >>>       >
>> >>>       > So the first question that we should ask, shall we allow putting
>> >>>      jcache and infinispan-embedded together on the classpath. If the
>> >>>      answer is yes, I believe it should stay as it is (since the user
>> >>>      always have a choice whether he wants to use jcache with or without
>> >>>      uber jar). The same question needs to be asked for Spring modules as
>> >>>      well as all cache stores. The behavior needs to be consistent across
>> >>>      all those modules.
>> >>>       >
>> >>>       > If the answer is no (which is also valid because jcache is
>> >>>      already present in embedded uber jar), we should migrate all JBoss
>> >>>      Logging references to Infinispan Common Logging (as Tristan did here
>> >>>      [4]) and we can make infinispan-core as a compile time dependency to
>> >>>      jcache. Even though migrating to Infinispan logger is not necessary,
>> >>>      this way we won't break users app which used infinispan-embedded +
>> >>>      jcache approach. Of course the same applies to Spring and Cache
>> >>>      stores modules.
>> >>>       >
>> >>>       > I think the latter approach deserves some exploration. I would
>> >>>      vote for moving that way.
>> >>>       >
>> >>>       > Thanks,
>> >>>       > Sebastian
>> >>>       >
>> >>>       > [1]
>> >>>      https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>> >>>      <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
>> >>>       > [2] https://issues.jboss.org/browse/ISPN-6295
>> >>>      <https://issues.jboss.org/browse/ISPN-6295>
>> >>>       > [3] https://issues.jboss.org/browse/ISPN-6132
>> >>>      <https://issues.jboss.org/browse/ISPN-6132>
>> >>>       > [4] https://github.com/infinispan/infinispan/pull/4140/files
>> >>>      <https://github.com/infinispan/infinispan/pull/4140/files>
>> >>>       >
>> >>>       >
>> >>>       > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
>> >>>      <galder at redhat.com <mailto:galder at redhat.com>> wrote:
>> >>>       > Hi all,
>> >>>       >
>> >>>       > Re:
>> >>>      https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>> >>>      <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
>> >>>       >
>> >>>       > St?phane makes a good point there, why did we make core provided
>> >>>      dependency? It does feel a bit of a pain that anyone that depends on
>> >>>      jcache embedded also needs to depend on core.
>> >>>       >
>> >>>       > Any more details behind this decision?
>> >>>       >
>> >>>       > Cheers,
>> >>>       > --
>> >>>       > Galder Zamarre?o
>> >>>       > Infinispan, Red Hat
>> >>>       >
>> >>>       > --
>> >>>       > SEBASTIAN ?ASKAWIEC
>> >>>       > INFINISPAN DEVELOPER
>> >>>       > Red Hat EMEA
>> >>>       >
>> >>>
>> >>>
>> >>>      _______________________________________________
>> >>>      infinispan-dev mailing list
>> >>>      infinispan-dev at lists.jboss.org
>> >>>      <mailto:infinispan-dev at lists.jboss.org>
>> >>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>>      <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> infinispan-dev mailing list
>> >>> infinispan-dev at lists.jboss.org
>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>>
>> >> --
>> >> Tristan Tarrant
>> >> Infinispan Lead
>> >> JBoss, a division of Red Hat
>> >> _______________________________________________
>> >> infinispan-dev mailing list
>> >> infinispan-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From galder at redhat.com  Mon Jun 12 09:27:00 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Mon, 12 Jun 2017 15:27:00 +0200
Subject: [infinispan-dev] Moving functional API to core
In-Reply-To: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
References: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
Message-ID: <C7EBC6A8-DCBA-477B-AF14-D43858729DF2@redhat.com>

Sounds good to me. 

Remember that functional API is marked as experimental, so it's fine to do things like this.

Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat

> On 9 Jun 2017, at 23:02, Radim Vansa <rvansa at redhat.com> wrote:
> 
> Hi guys,
> 
> when the functional API has been outline, the interfaces were put into 
> infinispan-commons to make it possible to share these between remote 
> clients and embedded use case. However, it seems that reusing this as-is 
> impossible or at least impractical as we cannot send the lambdas in a 
> language neutral way. In the future, we may implement a way to share 
> functions between client and a server but that will most likely result 
> in an interface accepting something else than Function<ReadWriteEntry, 
> R>. Also, it's rather weird to have two EntryVersion interfaces.
> 
> Therefore I suggest moving org.infinispan.commons.api.functional to 
> infinispan-core, package org.infinispan.api.functional
> 
> You might say that the server-side code would use the interfaces, but 
> once it's running on server, it should depend on core (or core-api) - 
> commons is what is shared with the client, and if the client will in 
> future register a new function on the server, the user code should 
> depend on core-api as well (client-hotrod itself does not have to).
> 
> If you wonder what led me to this is that I've tried to add 
> SerializableFunction overloads to the FunctionalMap and found out that 
> SerializableFunction et all are only in infinispan-core (for good).
> 
> Please let me know if you have objections/if there something I have missed.
> 
> Radim
> 
> -- 
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From dan.berindei at gmail.com  Mon Jun 12 10:05:15 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 12 Jun 2017 17:05:15 +0300
Subject: [infinispan-dev] Moving functional API to core
In-Reply-To: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
References: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
Message-ID: <CA+nfvwSELXecmR_iAWurCvd2y9457yJ5G2zb2T2GpqEL-p=nMw@mail.gmail.com>

+1

On Sat, Jun 10, 2017 at 12:02 AM, Radim Vansa <rvansa at redhat.com> wrote:
> Hi guys,
>
> when the functional API has been outline, the interfaces were put into
> infinispan-commons to make it possible to share these between remote
> clients and embedded use case. However, it seems that reusing this as-is
> impossible or at least impractical as we cannot send the lambdas in a
> language neutral way. In the future, we may implement a way to share
> functions between client and a server but that will most likely result
> in an interface accepting something else than Function<ReadWriteEntry,
> R>. Also, it's rather weird to have two EntryVersion interfaces.
>
> Therefore I suggest moving org.infinispan.commons.api.functional to
> infinispan-core, package org.infinispan.api.functional
>
> You might say that the server-side code would use the interfaces, but
> once it's running on server, it should depend on core (or core-api) -
> commons is what is shared with the client, and if the client will in
> future register a new function on the server, the user code should
> depend on core-api as well (client-hotrod itself does not have to).
>
> If you wonder what led me to this is that I've tried to add
> SerializableFunction overloads to the FunctionalMap and found out that
> SerializableFunction et all are only in infinispan-core (for good).
>
> Please let me know if you have objections/if there something I have missed.
>
> Radim
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From mudokonman at gmail.com  Mon Jun 12 10:52:59 2017
From: mudokonman at gmail.com (William Burns)
Date: Mon, 12 Jun 2017 14:52:59 +0000
Subject: [infinispan-dev] Moving functional API to core
In-Reply-To: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
References: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
Message-ID: <CA+YCuUPoyO6DV_02LvUQnb4c72twqDJ42EoFXBfgENcyTFftrw@mail.gmail.com>

On Sat, Jun 10, 2017 at 12:56 AM Radim Vansa <rvansa at redhat.com> wrote:

> Hi guys,
>
> when the functional API has been outline, the interfaces were put into
> infinispan-commons to make it possible to share these between remote
> clients and embedded use case. However, it seems that reusing this as-is
> impossible or at least impractical as we cannot send the lambdas in a
> language neutral way. In the future, we may implement a way to share
> functions between client and a server but that will most likely result
> in an interface accepting something else than Function<ReadWriteEntry,
> R>. Also, it's rather weird to have two EntryVersion interfaces.
>
> Therefore I suggest moving org.infinispan.commons.api.functional to
> infinispan-core, package org.infinispan.api.functional
>
> You might say that the server-side code would use the interfaces, but
> once it's running on server, it should depend on core (or core-api) -
> commons is what is shared with the client, and if the client will in
> future register a new function on the server, the user code should
> depend on core-api as well (client-hotrod itself does not have to).
>
> If you wonder what led me to this is that I've tried to add
> SerializableFunction overloads to the FunctionalMap and found out that
> SerializableFunction et all are only in infinispan-core (for good).
>

We could move these into commons in a major version if we need to as well.
I never thought about using them in the client code as we never planned on
supporting serialized lambdas there, but if it makes other things easier I
am for it.

Also there is nothing stopping us from having these in commons right now,
there is nothing special about the interfaces, they can just be copied over.


>
> Please let me know if you have objections/if there something I have missed.
>
> Radim
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170612/b09e8e32/attachment-0001.html 

From vrigamon at redhat.com  Mon Jun 12 13:08:29 2017
From: vrigamon at redhat.com (Vittorio Rigamonti)
Date: Mon, 12 Jun 2017 19:08:29 +0200
Subject: [infinispan-dev] Moving functional API to core
In-Reply-To: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
References: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
Message-ID: <CADE_1G-WaFNjbhoH0Wddez3-aCFDUMf5hn3oMp4=m1dMjWzVcQ@mail.gmail.com>

+1

On Fri, Jun 9, 2017 at 11:02 PM, Radim Vansa <rvansa at redhat.com> wrote:

> Hi guys,
>
> when the functional API has been outline, the interfaces were put into
> infinispan-commons to make it possible to share these between remote
> clients and embedded use case. However, it seems that reusing this as-is
> impossible or at least impractical as we cannot send the lambdas in a
> language neutral way. In the future, we may implement a way to share
> functions between client and a server but that will most likely result
> in an interface accepting something else than Function<ReadWriteEntry,
> R>. Also, it's rather weird to have two EntryVersion interfaces.
>
> Therefore I suggest moving org.infinispan.commons.api.functional to
> infinispan-core, package org.infinispan.api.functional
>
> You might say that the server-side code would use the interfaces, but
> once it's running on server, it should depend on core (or core-api) -
> commons is what is shared with the client, and if the client will in
> future register a new function on the server, the user code should
> depend on core-api as well (client-hotrod itself does not have to).
>
> If you wonder what led me to this is that I've tried to add
> SerializableFunction overloads to the FunctionalMap and found out that
> SerializableFunction et all are only in infinispan-core (for good).
>
> Please let me know if you have objections/if there something I have missed.
>
> Radim
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>



-- 

Vittorio Rigamonti

Senior Software Engineer

Red Hat

<https://www.redhat.com>

Milan, Italy

vrigamon at redhat.com

irc: rigazilla
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170612/43f71785/attachment.html 

From rvansa at redhat.com  Tue Jun 13 03:44:44 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 13 Jun 2017 09:44:44 +0200
Subject: [infinispan-dev] Moving functional API to core
In-Reply-To: <CA+YCuUPoyO6DV_02LvUQnb4c72twqDJ42EoFXBfgENcyTFftrw@mail.gmail.com>
References: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
	<CA+YCuUPoyO6DV_02LvUQnb4c72twqDJ42EoFXBfgENcyTFftrw@mail.gmail.com>
Message-ID: <7b1542e2-dbc6-697f-f5df-d808bcee7478@redhat.com>

On 06/12/2017 04:52 PM, William Burns wrote:
>
>
> On Sat, Jun 10, 2017 at 12:56 AM Radim Vansa <rvansa at redhat.com 
> <mailto:rvansa at redhat.com>> wrote:
>
>     Hi guys,
>
>     when the functional API has been outline, the interfaces were put into
>     infinispan-commons to make it possible to share these between remote
>     clients and embedded use case. However, it seems that reusing this
>     as-is
>     impossible or at least impractical as we cannot send the lambdas in a
>     language neutral way. In the future, we may implement a way to share
>     functions between client and a server but that will most likely result
>     in an interface accepting something else than Function<ReadWriteEntry,
>     R>. Also, it's rather weird to have two EntryVersion interfaces.
>
>     Therefore I suggest moving org.infinispan.commons.api.functional to
>     infinispan-core, package org.infinispan.api.functional
>
>     You might say that the server-side code would use the interfaces, but
>     once it's running on server, it should depend on core (or core-api) -
>     commons is what is shared with the client, and if the client will in
>     future register a new function on the server, the user code should
>     depend on core-api as well (client-hotrod itself does not have to).
>
>     If you wonder what led me to this is that I've tried to add
>     SerializableFunction overloads to the FunctionalMap and found out that
>     SerializableFunction et all are only in infinispan-core (for good).
>
>
> We could move these into commons in a major version if we need to as 
> well. I never thought about using them in the client code as we never 
> planned on supporting serialized lambdas there, but if it makes other 
> things easier I am for it.
>
> Also there is nothing stopping us from having these in commons right 
> now, there is nothing special about the interfaces, they can just be 
> copied over.

-1 These can't be simply copied, because two modules cannot share a 
package name (org.infinispan.util), therefore we would have to move the 
SerializableFunction to org.infinispan.commons.util.function. But as you 
say; we can't/don't want to support lambdas in any remote client 
operations and therefore these would be superfluous in commons.

We have to think about a pattern for the building-blocks (counters, 
locks, multimaps...): in embedded case we want to expose API using 
lambdas, in remote client this should be named filter, script or Ickle 
query. Obvious solution is having BaseFeature -> EmbeddedFeature, 
RemoteFeature that would expose the functional operations symmetrically 
but with different API, but it seems to me a bit inelegant.

Note that embedded/remote building blocks will have different 
properties/behaviour anyway - e.g. for embedded it could be useful to 
execute an action once the 'owning node' crashes (e.g. release a lock) 
while it does not make much sense with remote client.

Radim

>
>     Please let me know if you have objections/if there something I
>     have missed.
>
>     Radim
>
>     --
>     Radim Vansa <rvansa at redhat.com <mailto:rvansa at redhat.com>>
>     JBoss Performance Team
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From mudokonman at gmail.com  Tue Jun 13 09:07:16 2017
From: mudokonman at gmail.com (William Burns)
Date: Tue, 13 Jun 2017 13:07:16 +0000
Subject: [infinispan-dev] Moving functional API to core
In-Reply-To: <7b1542e2-dbc6-697f-f5df-d808bcee7478@redhat.com>
References: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
	<CA+YCuUPoyO6DV_02LvUQnb4c72twqDJ42EoFXBfgENcyTFftrw@mail.gmail.com>
	<7b1542e2-dbc6-697f-f5df-d808bcee7478@redhat.com>
Message-ID: <CA+YCuUMj1U-AW-S7ZEV9PJSppWmP3p40NuRCA9yDa4ky_Yt=TQ@mail.gmail.com>

On Tue, Jun 13, 2017, 3:54 AM Radim Vansa <rvansa at redhat.com> wrote:

> On 06/12/2017 04:52 PM, William Burns wrote:
> >
> >
> > On Sat, Jun 10, 2017 at 12:56 AM Radim Vansa <rvansa at redhat.com
> > <mailto:rvansa at redhat.com>> wrote:
> >
> >     Hi guys,
> >
> >     when the functional API has been outline, the interfaces were put
> into
> >     infinispan-commons to make it possible to share these between remote
> >     clients and embedded use case. However, it seems that reusing this
> >     as-is
> >     impossible or at least impractical as we cannot send the lambdas in a
> >     language neutral way. In the future, we may implement a way to share
> >     functions between client and a server but that will most likely
> result
> >     in an interface accepting something else than
> Function<ReadWriteEntry,
> >     R>. Also, it's rather weird to have two EntryVersion interfaces.
> >
> >     Therefore I suggest moving org.infinispan.commons.api.functional to
> >     infinispan-core, package org.infinispan.api.functional
> >
> >     You might say that the server-side code would use the interfaces, but
> >     once it's running on server, it should depend on core (or core-api) -
> >     commons is what is shared with the client, and if the client will in
> >     future register a new function on the server, the user code should
> >     depend on core-api as well (client-hotrod itself does not have to).
> >
> >     If you wonder what led me to this is that I've tried to add
> >     SerializableFunction overloads to the FunctionalMap and found out
> that
> >     SerializableFunction et all are only in infinispan-core (for good).
> >
> >
> > We could move these into commons in a major version if we need to as
> > well. I never thought about using them in the client code as we never
> > planned on supporting serialized lambdas there, but if it makes other
> > things easier I am for it.
> >
> > Also there is nothing stopping us from having these in commons right
> > now, there is nothing special about the interfaces, they can just be
> > copied over.
>
> -1 These can't be simply copied, because two modules cannot share a
> package name (org.infinispan.util), therefore we would have to move the
> SerializableFunction to org.infinispan.commons.util.function.


I never said they had to be on the same package :-P

But as you
> say; we can't/don't want to support lambdas in any remote client
> operations and therefore these would be superfluous in commons.


> We have to think about a pattern for the building-blocks (counters,
> locks, multimaps...): in embedded case we want to expose API using
> lambdas, in remote client this should be named filter, script or Ickle
> query. Obvious solution is having BaseFeature -> EmbeddedFeature,
> RemoteFeature that would expose the functional operations symmetrically
> but with different API, but it seems to me a bit inelegant.
>

This is always our problem, we never have a solution and then client API
falls behind.

Also even though we wouldn't serialize lambdas with client, doesn't mean we
can't use lambdas with the client. Just means the operation would have
slower performance, since it would be evaluated in the client.

I personally welcome the BaseFeature you mentioned because we need that
asap so that we can create these API while maintaining some type of
semblance between them.


> Note that embedded/remote building blocks will have different
> properties/behaviour anyway - e.g. for embedded it could be useful to
> execute an action once the 'owning node' crashes (e.g. release a lock)
> while it does not make much sense with remote client.
>
> Radim
>
> >
> >     Please let me know if you have objections/if there something I
> >     have missed.
> >
> >     Radim
> >
> >     --
> >     Radim Vansa <rvansa at redhat.com <mailto:rvansa at redhat.com>>
> >     JBoss Performance Team
> >
> >     _______________________________________________
> >     infinispan-dev mailing list
> >     infinispan-dev at lists.jboss.org <mailto:
> infinispan-dev at lists.jboss.org>
> >     https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170613/1ed141e5/attachment.html 

From slaskawi at redhat.com  Tue Jun 13 09:10:40 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Tue, 13 Jun 2017 13:10:40 +0000
Subject: [infinispan-dev] REST content directory
Message-ID: <CAA9R9O7=Lv+hxLxqxWY4oo=si0UN9j9XQ=F+dg+_hczq+8y44g@mail.gmail.com>

Hey,

A while ago I noticed that we have static directory with some information
about REST interface in the feature pack [1]. The directory is put in the
server distribution ($ISPN_HOME/rest).

Our REST server doesn't host those files when accessing /rest context. We
host [2] in this scenario. Those two web pages differ from each other and I
must admit, that [1] is much better.

I would like to propose removing [1] from the feature pack and serving it
in the /rest context. Would you agree?

Thanks,
Sebastian

[1]
https://github.com/infinispan/infinispan/tree/master/server/integration/feature-pack/src/main/resources/content/rest
[2]
https://github.com/infinispan/infinispan/tree/master/server/rest/src/main/resources
-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170613/18052674/attachment-0001.html 

From rvansa at redhat.com  Tue Jun 13 10:11:37 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 13 Jun 2017 16:11:37 +0200
Subject: [infinispan-dev] Moving functional API to core
In-Reply-To: <CA+YCuUMj1U-AW-S7ZEV9PJSppWmP3p40NuRCA9yDa4ky_Yt=TQ@mail.gmail.com>
References: <87fd2086-2921-cdf1-6ec5-e5177bfe9579@redhat.com>
	<CA+YCuUPoyO6DV_02LvUQnb4c72twqDJ42EoFXBfgENcyTFftrw@mail.gmail.com>
	<7b1542e2-dbc6-697f-f5df-d808bcee7478@redhat.com>
	<CA+YCuUMj1U-AW-S7ZEV9PJSppWmP3p40NuRCA9yDa4ky_Yt=TQ@mail.gmail.com>
Message-ID: <317037cf-67ac-b266-9338-c6aba1dd1008@redhat.com>

On 06/13/2017 03:07 PM, William Burns wrote:
>
>
> On Tue, Jun 13, 2017, 3:54 AM Radim Vansa <rvansa at redhat.com 
> <mailto:rvansa at redhat.com>> wrote:
>
>     On 06/12/2017 04:52 PM, William Burns wrote:
>     >
>     >
>     > On Sat, Jun 10, 2017 at 12:56 AM Radim Vansa <rvansa at redhat.com
>     <mailto:rvansa at redhat.com>
>     > <mailto:rvansa at redhat.com <mailto:rvansa at redhat.com>>> wrote:
>     >
>     >     Hi guys,
>     >
>     >     when the functional API has been outline, the interfaces
>     were put into
>     >     infinispan-commons to make it possible to share these
>     between remote
>     >     clients and embedded use case. However, it seems that
>     reusing this
>     >     as-is
>     >     impossible or at least impractical as we cannot send the
>     lambdas in a
>     >     language neutral way. In the future, we may implement a way
>     to share
>     >     functions between client and a server but that will most
>     likely result
>     >     in an interface accepting something else than
>     Function<ReadWriteEntry,
>     >     R>. Also, it's rather weird to have two EntryVersion interfaces.
>     >
>     >     Therefore I suggest moving
>     org.infinispan.commons.api.functional to
>     >     infinispan-core, package org.infinispan.api.functional
>     >
>     >     You might say that the server-side code would use the
>     interfaces, but
>     >     once it's running on server, it should depend on core (or
>     core-api) -
>     >     commons is what is shared with the client, and if the client
>     will in
>     >     future register a new function on the server, the user code
>     should
>     >     depend on core-api as well (client-hotrod itself does not
>     have to).
>     >
>     >     If you wonder what led me to this is that I've tried to add
>     >     SerializableFunction overloads to the FunctionalMap and
>     found out that
>     >     SerializableFunction et all are only in infinispan-core (for
>     good).
>     >
>     >
>     > We could move these into commons in a major version if we need to as
>     > well. I never thought about using them in the client code as we
>     never
>     > planned on supporting serialized lambdas there, but if it makes
>     other
>     > things easier I am for it.
>     >
>     > Also there is nothing stopping us from having these in commons right
>     > now, there is nothing special about the interfaces, they can just be
>     > copied over.
>
>     -1 These can't be simply copied, because two modules cannot share a
>     package name (org.infinispan.util), therefore we would have to
>     move the
>     SerializableFunction to org.infinispan.commons.util.function.
>
>
> I never said they had to be on the same package :-P
>
>     But as you
>     say; we can't/don't want to support lambdas in any remote client
>     operations and therefore these would be superfluous in commons.
>
>
>     We have to think about a pattern for the building-blocks (counters,
>     locks, multimaps...): in embedded case we want to expose API using
>     lambdas, in remote client this should be named filter, script or Ickle
>     query. Obvious solution is having BaseFeature -> EmbeddedFeature,
>     RemoteFeature that would expose the functional operations
>     symmetrically
>     but with different API, but it seems to me a bit inelegant.
>
>
> This is always our problem, we never have a solution and then client 
> API falls behind.

Speaking about clients falling behind, do we have the remote counters 
somewhere on the roadmap? I think that OGM's need of sequences was one 
of the primary motivations, but OGM is now focusing more on the Hot Rod 
integration.

>
> Also even though we wouldn't serialize lambdas with client, doesn't 
> mean we can't use lambdas with the client. Just means the operation 
> would have slower performance, since it would be evaluated in the client.

That defies the point completely. I wouldn't trick the user into 
thinking that the operation happens in place unless we have a plan to 
fix that **soon**.

>
> I personally welcome the BaseFeature you mentioned because we need 
> that asap so that we can create these API while maintaining some type 
> of semblance between them.
>
>
>     Note that embedded/remote building blocks will have different
>     properties/behaviour anyway - e.g. for embedded it could be useful to
>     execute an action once the 'owning node' crashes (e.g. release a lock)
>     while it does not make much sense with remote client.
>
>     Radim
>
>     >
>     >     Please let me know if you have objections/if there something I
>     >     have missed.
>     >
>     >     Radim
>     >
>     >     --
>     >     Radim Vansa <rvansa at redhat.com <mailto:rvansa at redhat.com>
>     <mailto:rvansa at redhat.com <mailto:rvansa at redhat.com>>>
>     >     JBoss Performance Team
>     >
>     >     _______________________________________________
>     >     infinispan-dev mailing list
>     > infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>     <mailto:infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>>
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     >
>     >
>     >
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>     --
>     Radim Vansa <rvansa at redhat.com <mailto:rvansa at redhat.com>>
>     JBoss Performance Team
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From afield at redhat.com  Tue Jun 13 10:19:46 2017
From: afield at redhat.com (Alan Field)
Date: Tue, 13 Jun 2017 10:19:46 -0400 (EDT)
Subject: [infinispan-dev] REST content directory
In-Reply-To: <CAA9R9O7=Lv+hxLxqxWY4oo=si0UN9j9XQ=F+dg+_hczq+8y44g@mail.gmail.com>
References: <CAA9R9O7=Lv+hxLxqxWY4oo=si0UN9j9XQ=F+dg+_hczq+8y44g@mail.gmail.com>
Message-ID: <1703116948.22553927.1497363586567.JavaMail.zimbra@redhat.com>

Hey Sebastian, 

I completely agree that hosting [1] in the /rest context is superior to [2]. I think that is what you are proposing, right? :-) 

Thanks, 
Alan 

----- Original Message -----

> From: "Sebastian Laskawiec" <slaskawi at redhat.com>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Tuesday, June 13, 2017 9:10:40 AM
> Subject: [infinispan-dev] REST content directory

> Hey,

> A while ago I noticed that we have static directory with some information
> about REST interface in the feature pack [1]. The directory is put in the
> server distribution ($ISPN_HOME/rest).

> Our REST server doesn't host those files when accessing /rest context. We
> host [2] in this scenario. Those two web pages differ from each other and I
> must admit, that [1] is much better.

> I would like to propose removing [1] from the feature pack and serving it in
> the /rest context. Would you agree?

> Thanks,
> Sebastian

> [1]
> https://github.com/infinispan/infinispan/tree/master/server/integration/feature-pack/src/main/resources/content/rest
> [2]
> https://github.com/infinispan/infinispan/tree/master/server/rest/src/main/resources
> --

> SEBASTIAN ?ASKAWIEC

> INFINISPAN DEVELOPER

> Red Hat EMEA

> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170613/6a8ba675/attachment.html 

From slaskawi at redhat.com  Tue Jun 13 10:34:29 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Tue, 13 Jun 2017 14:34:29 +0000
Subject: [infinispan-dev] REST content directory
In-Reply-To: <1703116948.22553927.1497363586567.JavaMail.zimbra@redhat.com>
References: <CAA9R9O7=Lv+hxLxqxWY4oo=si0UN9j9XQ=F+dg+_hczq+8y44g@mail.gmail.com>
	<1703116948.22553927.1497363586567.JavaMail.zimbra@redhat.com>
Message-ID: <CAA9R9O5PVXp5L42kWRLSBz5L3Ta8iXagh3tN+NV-EKEhS28oMg@mail.gmail.com>

Yes, exactly. Sorry if I didn't make it clear enough...

On Tue, Jun 13, 2017 at 4:31 PM Alan Field <afield at redhat.com> wrote:

> Hey Sebastian,
>
> I completely agree that hosting [1] in the /rest context is superior to
> [2]. I think that is what you are proposing, right? :-)
>
> Thanks,
> Alan
>
> ------------------------------
>
> *From: *"Sebastian Laskawiec" <slaskawi at redhat.com>
> *To: *"infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> *Sent: *Tuesday, June 13, 2017 9:10:40 AM
> *Subject: *[infinispan-dev] REST content directory
>
>
>
> Hey,
>
> A while ago I noticed that we have static directory with some information
> about REST interface in the feature pack [1]. The directory is put in the
> server distribution ($ISPN_HOME/rest).
>
> Our REST server doesn't host those files when accessing /rest context. We
> host [2] in this scenario. Those two web pages differ from each other and I
> must admit, that [1] is much better.
>
> I would like to propose removing [1] from the feature pack and serving it
> in the /rest context. Would you agree?
>
> Thanks,
> Sebastian
>
> [1]
> https://github.com/infinispan/infinispan/tree/master/server/integration/feature-pack/src/main/resources/content/rest
> [2]
> https://github.com/infinispan/infinispan/tree/master/server/rest/src/main/resources
> --
>
> SEBASTIAN ?ASKAWIEC
>
> INFINISPAN DEVELOPER
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170613/503cca94/attachment.html 

From sanne at infinispan.org  Tue Jun 13 14:15:03 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 13 Jun 2017 19:15:03 +0100
Subject: [infinispan-dev] Hibernate Search v 5.8 and plans for beyond!
Message-ID: <CAFm4XO11cGjoQvxVieggB92RB5q=_hE06Y2sp6JM_w7QpqHBCg@mail.gmail.com>

TLDR: the new Search SPI requires using the "free form" types - but
don't expect to be able to take advantage of it just yet.

Hi all,

we just released Hibernate Search 5.8.0.Beta3 and it's meant to be our
last beta!

Progress and pace are looking very good.

The details of the release can be found on our blog [1] as usual, but
I'd like to call out on several changes relevant for Infinispan.

First and foremost, know that we'll be working on 6.0 as next big
thing; at that stage the API will see a significant redesign, focused
on making Lucene an optional implementation detail.

This makes this 5.8 significant for Infinispan as it's the last one
you'll have until you're ready to break API.

We've been working hard on addressing the needs of Infinispan, but in
several cases we had to make - and will still make - dramatic changes
in the SPI.

Some benefits relevant to you all:
 - the "ram" directory changed name as you suggested just recently
 - statistics now include index size (how large is it to store it)
 - a proper API to define Analyzers programmatically, you could now
add "out of the box" default definitions for Infinispan server.
 - the same for Normalizers (a new concept, see the blog [1])
 - tons of other improvements and bugfixes ;) such as:
   -- better performance by finally being able to use JDK8 features
   -- Java 9 compatibility
   -- a critical performance improvement in Spatial bridges
   -- better Spring and CDI improvements


# A new type system is coming !

And this is the main point.. however, it's not ready yet as it's a huge change.

The new type system - also named "free form" in all the meetings in
which we wished we had it - needs to start from the SPI and that's
partially included in this Beta.
(We postponed this for way too long by trying to match a good time to
break SPIs.. we'll just break it or it will never happen)

I will soon send a PR to update Infinispan to *work fine* with the
current SPI changes, but I will refrain from refactoring anything yet,
so that we can validate it all in small steps.

Still, many other existing SPI methods have been deprecated:

As soon as that's merged, I'll need help to migrate away from all
those deprecated methods as I simply can't make progress on the Search
side without starting to delete a good amount of these deprecated
methods.
So please know this: most of those deprecated methods will be removed
from Hibernate Search soon; we can of course keep some of them around
if we identify strong needs, but identifying these points can only be
done by getting started.

Please note that you can't yet plug-in a new type system to benefit
REST, Protobuf or other systems:
 - several components still make assumptions about this being backed
by annotated POJOs
 - there's no SPI yet to plugin alternative metadata

There's also no API to extract indexed values from an alternative
system, although I'm sure you could keep using the current hacks for
this to work out: it's not possible to get this in 5.8 so it will have
to be 6+.

Thanks,
Sanne

1 - http://in.relation.to/2017/06/13/hibernate-search-5-8-0-Beta3/

From slaskawi at redhat.com  Wed Jun 14 02:46:45 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 14 Jun 2017 06:46:45 +0000
Subject: [infinispan-dev] Using load balancers for Infinispan in
	Kubernetes
In-Reply-To: <CAA9R9O4CrfxHsr0pOGJmq6LBSeF0s2CWiXTTZT1WRu4JXx_uYA@mail.gmail.com>
References: <CAA9R9O5Z7OW994bewSHRBHLd4gmztkOKnTOoxV22kk15PwwAVA@mail.gmail.com>
	<CAFm4XO22YOCZY8VfEarv8us-=aPo1Tw5hG7AaMq0vCQfAxtiQg@mail.gmail.com>
	<CAA9R9O4a2TkR3PFcrUep=_UKM0bfnZ4dc=Fhmn1sZDLCKWy_9g@mail.gmail.com>
	<1E9DF83C-0BEC-4F7B-B91C-F1B4313F1E9C@redhat.com>
	<CAFm4XO3V-f0937Vf8cSfP0KuDEYCPUWRRpvH9b=+xYAuxDaCfw@mail.gmail.com>
	<CAA9R9O4CrfxHsr0pOGJmq6LBSeF0s2CWiXTTZT1WRu4JXx_uYA@mail.gmail.com>
Message-ID: <CAA9R9O6UjUmUMipaFzf1nABp_AX8TxUUuO9z-p886VC14t2kpg@mail.gmail.com>

Hey guys,

I'm receiving more and more queries about this solution so I marked [1] for
9.1.0.Beta1 milestone.

Please let me know if that's NOT ok [3] :)

Thanks,
Sebastian

[1] https://github.com/infinispan/infinispan/pull/5164
[2] https://github.com/infinispan/infinispan/milestone/78
[3] Just checking Sanne's and Will's spam filters :)

On Thu, Jun 1, 2017 at 9:09 AM Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> I'm calm!! I'm calm!!
> [image: pasted1]
> Being serious now, no offence taken and thanks for clarification Sanne.
> Next time I'll just publish some percentage difference which will probably
> express better what I want to tell. And if you see anything wrong with the
> test methodology that could influence the relation between those tests,
> just let me know.
>
> Don't you think guys we should have a reference OpenShift/Kubernetes
> environment for testing (minikube/minishift are great but it's always "it
> works on my machine" type of thing)? I expect there will be more and more
> discussions around cloud topics. If you are interested in this, please let
> me know off-list. I will try to arrange something.
>
> Thanks,
> Sebastian
>
> On Thu, Jun 1, 2017 at 4:49 AM Sanne Grinovero <sanne at infinispan.org>
> wrote:
>
>> On 31 May 2017 at 17:48, Galder Zamarre?o <galder at redhat.com> wrote:
>> > Cool down peoples!
>> >
>> > http://www.quickmeme.com/meme/35ovcy
>> >
>> > Sebastian, don't think Sanne was being rude, he's just blunt and we
>> need his bluntness :)
>> >
>> > Sanne, be nice to Sebastian and get him a beer next time around ;)
>>
>> Hey, he started it! His email was formatted with HTML !? ;)
>>
>> But seriously, I didn't mean to be rude or disrespectful; if it comes
>> across like that I'm sorry. FWIW the answers seemed cool to me too.
>>
>> Let me actually clarify that I love the attitude of Sebastian to try
>> the various approaches and get with some measurements to help with
>> important design decisions. It's good that we spend some time
>> evaluating the alternatives, and it's equally good that we debate the
>> trade-offs here.
>>
>> As warned in my email I'm "nit-picking" on the benchmark methodology,
>> and probably more than the usual, because I care!
>>
>> I am highlighting what I believe to be useful advice though: the
>> absolute metrics of such tests need not be taken as primary
>> (exclusive?) decision factor. Which doesn't mean that performing such
>> tests is not useful, they certainly provide a lot to think about.
>> Yet the interpretation of such results need to not be generalised, and
>> the interpretation process is more important than the absolute
>> ballpark figures they provide; for example it's paramount to figure
>> out which factors of the test could theoretically invert the results.
>> Using them to identify a binary faster/slow yes/no to proof/disproof a
>> design decision is a dangerous fallacy .. and I'm not picking on
>> Sebastian specifically, just reminding about it as we've all been
>> guilty of it: confirmation bias, etc..
>>
>> The best advice I've ever had myself in performance analysis is to not
>> try to figure out which implementation is faster "on my machine", but
>> to understand why it's producing a specific result, and what is
>> preventing it to produce an higher figure.
>> Once you know that, it's very valuable information as it will tell you
>> either what needs fixing with a benchmark, or what needs to be done to
>> improve the performance of your implementation ;)
>> So that's why I personally don't publish figures often, but hey I
>> still run such tests too and spend a lot of time analysing them, to
>> eventually share what I figure out in the process...
>>
>> Thanks,
>> Sanne
>>
>>
>>
>> >
>> > Peace out! :)
>> > --
>> > Galder Zamarre?o
>> > Infinispan, Red Hat
>> >
>> >> On 31 May 2017, at 09:38, Sebastian Laskawiec <slaskawi at redhat.com>
>> wrote:
>> >>
>> >> Hey Sanne,
>> >>
>> >> Comments inlined.
>> >>
>> >> Thanks,
>> >> Sebastian
>> >>
>> >> On Tue, May 30, 2017 at 5:58 PM Sanne Grinovero <sanne at infinispan.org>
>> wrote:
>> >> Hi Sebastian,
>> >>
>> >> the "intelligent routing" of Hot Rod being one of - if not the main -
>> reason to use Hot Rod, I wonder if we shouldn't rather suggest people to
>> stick with HTTP (REST) in such architectures.
>> >>
>> >> Several people have suggested in the past the need to have an HTTP
>> smart load balancer which would be able to route the external REST requests
>> to the right node. Essentially have people use REST over the wider network,
>> up to reaching the Infinispan cluster where the service endpoint (the load
>> balancer) can convert them to optimised Hot Rod calls, or just leave them
>> in the same format but routing them with the same intelligence to the right
>> nodes.
>> >>
>> >> I realise my proposal requires some work on several fronts, at very
>> least we would need:
>> >>  - feature parity Hot Rod / REST so that people can actually use it
>> >>  - a REST load balancer
>> >>
>> >> But I think the output of such a direction would be far more reusable,
>> as both these points are high on the wish list anyway.
>> >>
>> >> Unfortunately I'm not convinced into this idea. Let me elaborate...
>> >>
>> >> It goes without saying that HTTP payload is simply larger and require
>> much more processing. That alone makes it slower than Hot Rod (I believe
>> Martin could provide you some numbers on that). The second arguments is
>> that switching/routing inside Kubernetes is bloody fast (since it's based
>> on iptables) and some cloud vendors optimize it even further (e.g. Google
>> Andromeda [1][2], I would be surprised if AWS didn't have anything
>> similar). During the work on this prototype I wrote a simple async binary
>> proxy [3] and measured GCP load balancer vs my proxy performance. They were
>> twice as fast [4][5]. You may argue whether I could write a better proxy.
>> Probably I could, but the bottom line is that another performance hit is
>> inevitable. They are really fast and they operate on their own
>> infrastructure (load balancers is something that is provided by the cloud
>> vendor to Kubernetes, not the other way around).
>> >>
>> >> So with all that in mind, are we going to get better results comparing
>> to my proposal for Hot Rod? I dare to doubt, even with HTTP/2 support
>> (which comes really soon I hope). The second question is whether this new
>> "REST load balancer" will work better than a standard load balancer using
>> round robin strategy? Again I dare to doubt, even if you you're faster at
>> routing request to proper node, you introduce another layer of latency.
>> >>
>> >> Of course the priority of this is up to Tristan but I definitely
>> wouldn't place it high on todo list. And before even looking at it I would
>> recommend taking Netty HTTP Proxy, putting it in the middle between real
>> load balancer and Infinispan app and measure performance with and without
>> it. Another test could be with 1 and 10 replicas to check the performance
>> penalty of hitting 100% and 10% requests into proper node.
>> >>
>> >> [1]
>> https://cloudplatform.googleblog.com/2014/08/containers-vms-kubernetes-and-vmware.html
>> >> [2]
>> https://cloudplatform.googleblog.com/2014/04/enter-andromeda-zone-google-cloud-platforms-latest-networking-stack.html
>> >> [3]
>> https://github.com/slaskawi/external-ip-proxy/blob/Benchmark_with_proxy/Proxy/Proxy.go
>> >> [4]
>> https://github.com/slaskawi/external-ip-proxy/blob/master/benchmark/results%20with%20proxy.txt
>> >> [5]
>> https://github.com/slaskawi/external-ip-proxy/blob/master/benchmark/results%20with%20loadbalancer.txt
>> >>
>> >> Not least having a "REST load balancer" would allow to deploy
>> Infinispan as an HTTP cache; just honouring the HTTP caching protocols and
>> existing standards would allow people to use any client to their liking,
>> >>
>> >> Could you please give me an example how this could work? The only way
>> that I know is to plug a cache into reverse proxy. NGNIX supports pluggable
>> Redis for example [6].
>> >>
>> >> [6] https://www.nginx.com/resources/wiki/modules/redis/
>> >>
>> >> without us having to maintain Hot Rod clients and support it on many
>> exotic platforms - we would still have Hot Rod clients but we'd be able to
>> pick a smaller set of strategical platforms (e.g. Windows doesn't have to
>> be in that list).
>> >>
>> >> As I mentioned before, I really doubts HTTP will be faster then Hot
>> Rod in any scenario.
>> >>
>> >> Such a load balancer could be written in Java (recent WildFly versions
>> are able to do this efficiently) or it could be written in another
>> language, all it takes is to integrate an Hot Rod client - or just the
>> intelligence of it- as an extension into an existing load balancer of our
>> choice.
>> >>
>> >> As I mentioned before, with custom load balancer you're introducing
>> another layer of latency. It's not a free ride.
>> >>
>> >> Allow me a bit more nit-picking on your benchmarks ;)
>> >> As you pointed out yourself there are several flaws in your setup:
>> "didn't tune", "running in a VM", "benchmarked on a mac mini", ...if you
>> know it's a flawed setup I'd rather not publish figures, especially not
>> suggest to make decisions based on such results.
>> >>
>> >> Why not? Infinispan is a public project and anyone can benchmark it
>> using JMH and taking decisions based on figures is always better than on
>> intuition. Even though there were multiple unknown factors involved in this
>> benchmark (this is why I pointed them out and asked to take the results
>> with a grain of salt), the test conditions for all scenarios were the same.
>> For me this is sufficient to give a general recommendation as I did. BTW,
>> this recommendation fits exactly my expectations (communication inside Kube
>> the fastest, LB per Pod a bit slower and no advanced routing the slowest).
>> Finally, the recommendation is based on a POC which by definition means it
>> doesn't fit all scenarios. You should always measure your system!
>> >>
>> >> So unless you can prove the benchmark results are fundamentally wrong
>> and I have drawn wrong conclusions (e.g. a simple client is the fastest
>> solution whereas inside Kubernetes communication is the slowest), please
>> don't use "naaah, that's wrong" argument. It's rude.
>> >>
>> >> At this level of design need to focus on getting the architecture
>> right; it should be self-speaking that your proposal of actually using
>> intelligent routing in some way should be better than not using it.
>> >>
>> >> My benchmark confirmed this. But as always I would be happy to discuss
>> some alternatives. But before trying to convince me to "REST Router",
>> please prove that introducing a load balancer (or just a simple async proxy
>> for start) gives similar or better performance than a simple load balancer
>> with round robin strategy.
>> >>
>> >> Once we'll have an agreement on a sound architecture, then we'll be
>> able to make the implementation efficient.
>> >>
>> >> Thanks,
>> >> Sanne
>> >>
>> >>
>> >>
>> >>
>> >> On 30 May 2017 at 13:43, Sebastian Laskawiec <slaskawi at redhat.com>
>> wrote:
>> >> Hey guys!
>> >>
>> >> Over past few weeks I've been working on accessing Infinispan cluster
>> deployed inside Kubernetes from the outside world. The POC diagram looks
>> like the following:
>> >>
>> >> <pasted1.png>
>> >>
>> >> As a reminder, the easiest (though not the most effective) way to do
>> it is to expose a load balancer Service (or a Node Port Service) and access
>> it using a client with basic intelligence (so that it doesn't try to update
>> server list based on topology information). As you might expect, this won't
>> give you much performance but at least you could access the cluster.
>> Another approach is to use TLS/SNI but again, the performance would be even
>> worse.
>> >>
>> >> During the research I tried to answer this problem and created
>> "External IP Controller" [1] (and corresponding Pull Request for mapping
>> internal/external addresses [2]). The main idea is to create a controller
>> deployed inside Kubernetes which will create (and destroy if not needed) a
>> load balancer per Infinispan Pod. Additionally the controller exposes
>> mapping between internal and external addresses which allows the client to
>> properly update server list as well as consistent hash information. A full
>> working example is located here [3].
>> >>
>> >> The biggest question is whether it's worth it? The short answer is
>> yes. Here are some benchmark results of performing 10k puts and 10k
>> puts&gets (please take them with a big grain of salt, I didn't optimize any
>> server settings):
>> >>       ? Benchmark app deployed inside Kuberenetes and using internal
>> addresses (baseline):
>> >>               ? 10k puts: 674.244 ?  16.654
>> >>               ? 10k puts&gets: 1288.437 ? 136.207
>> >>       ? Benchamrking app deployed in a VM outside of Kubernetes with
>> basic intelligence:
>> >>               ? 10k puts: 1465.567 ? 176.349
>> >>               ? 10k puts&gets: 2684.984 ? 114.993
>> >>       ? Benchamrking app deployed in a VM outside of Kubernetes with
>> address mapping and topology aware hashing:
>> >>               ? 10k puts: 1052.891 ?  31.218
>> >>               ? 10k puts&gets: 2465.586 ?  85.034
>> >> Note that benchmarking Infinispan from a VM might be very misleading
>> since it depends on data center configuration. Benchmarks above definitely
>> contain some delay between Google Compute Engine VM and a Kubernetes
>> cluster deployed in Google Container Engine. How big is the delay? Hard to
>> tell. What counts is the difference between client using basic intelligence
>> and topology aware intelligence. And as you can see it's not that small.
>> >>
>> >> So the bottom line - if you can, deploy your application along with
>> Infinispan cluster inside Kubernetes. That's the fastest configuration
>> since only iptables are involved. Otherwise use a load balancer per pod
>> with External IP Controller. If you don't care about performance, just use
>> basic client intelligence and expose everything using single load balancer.
>> >>
>> >> Thanks,
>> >> Sebastian
>> >>
>> >> [1] https://github.com/slaskawi/external-ip-proxy
>> >> [2] https://github.com/infinispan/infinispan/pull/5164
>> >> [3]
>> https://github.com/slaskawi/external-ip-proxy/tree/master/benchmark
>> >>
>> >> _______________________________________________
>> >> infinispan-dev mailing list
>> >> infinispan-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>
>> >> _______________________________________________
>> >> infinispan-dev mailing list
>> >> infinispan-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >> --
>> >> SEBASTIAN ?ASKAWIEC
>> >> INFINISPAN DEVELOPER
>> >> Red Hat EMEA
>> >>
>> >> _______________________________________________
>> >> infinispan-dev mailing list
>> >> infinispan-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
>
> SEBASTIAN ?ASKAWIEC
>
> INFINISPAN DEVELOPER
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig>
>
-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170614/4dabbc07/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pasted1
Type: image/png
Size: 25404 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170614/4dabbc07/attachment-0001.png 

From slaskawi at redhat.com  Wed Jun 14 03:14:02 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 14 Jun 2017 07:14:02 +0000
Subject: [infinispan-dev] Hibernate Search v 5.8 and plans for beyond!
In-Reply-To: <CAFm4XO11cGjoQvxVieggB92RB5q=_hE06Y2sp6JM_w7QpqHBCg@mail.gmail.com>
References: <CAFm4XO11cGjoQvxVieggB92RB5q=_hE06Y2sp6JM_w7QpqHBCg@mail.gmail.com>
Message-ID: <CAA9R9O4knaVXhN4wd=SnjvZ2Z4DroEmUbiOyJvH+XTsXzpxbhA@mail.gmail.com>

Hey Sanne,

Looks good. Could you tell me where I can find some information about the
CDI and Spring improvements? Maybe there's something we could integrate
into our extensions...

Thanks,
Sebastian

On Tue, Jun 13, 2017 at 8:34 PM Sanne Grinovero <sanne at infinispan.org>
wrote:

> TLDR: the new Search SPI requires using the "free form" types - but
> don't expect to be able to take advantage of it just yet.
>
> Hi all,
>
> we just released Hibernate Search 5.8.0.Beta3 and it's meant to be our
> last beta!
>
> Progress and pace are looking very good.
>
> The details of the release can be found on our blog [1] as usual, but
> I'd like to call out on several changes relevant for Infinispan.
>
> First and foremost, know that we'll be working on 6.0 as next big
> thing; at that stage the API will see a significant redesign, focused
> on making Lucene an optional implementation detail.
>
> This makes this 5.8 significant for Infinispan as it's the last one
> you'll have until you're ready to break API.
>
> We've been working hard on addressing the needs of Infinispan, but in
> several cases we had to make - and will still make - dramatic changes
> in the SPI.
>
> Some benefits relevant to you all:
>  - the "ram" directory changed name as you suggested just recently
>  - statistics now include index size (how large is it to store it)
>  - a proper API to define Analyzers programmatically, you could now
> add "out of the box" default definitions for Infinispan server.
>  - the same for Normalizers (a new concept, see the blog [1])
>  - tons of other improvements and bugfixes ;) such as:
>    -- better performance by finally being able to use JDK8 features
>    -- Java 9 compatibility
>    -- a critical performance improvement in Spatial bridges
>    -- better Spring and CDI improvements
>
>
> # A new type system is coming !
>
> And this is the main point.. however, it's not ready yet as it's a huge
> change.
>
> The new type system - also named "free form" in all the meetings in
> which we wished we had it - needs to start from the SPI and that's
> partially included in this Beta.
> (We postponed this for way too long by trying to match a good time to
> break SPIs.. we'll just break it or it will never happen)
>
> I will soon send a PR to update Infinispan to *work fine* with the
> current SPI changes, but I will refrain from refactoring anything yet,
> so that we can validate it all in small steps.
>
> Still, many other existing SPI methods have been deprecated:
>
> As soon as that's merged, I'll need help to migrate away from all
> those deprecated methods as I simply can't make progress on the Search
> side without starting to delete a good amount of these deprecated
> methods.
> So please know this: most of those deprecated methods will be removed
> from Hibernate Search soon; we can of course keep some of them around
> if we identify strong needs, but identifying these points can only be
> done by getting started.
>
> Please note that you can't yet plug-in a new type system to benefit
> REST, Protobuf or other systems:
>  - several components still make assumptions about this being backed
> by annotated POJOs
>  - there's no SPI yet to plugin alternative metadata
>
> There's also no API to extract indexed values from an alternative
> system, although I'm sure you could keep using the current hacks for
> this to work out: it's not possible to get this in 5.8 so it will have
> to be 6+.
>
> Thanks,
> Sanne
>
> 1 - http://in.relation.to/2017/06/13/hibernate-search-5-8-0-Beta3/
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170614/897dfae7/attachment.html 

From sanne at infinispan.org  Wed Jun 14 05:58:55 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 14 Jun 2017 10:58:55 +0100
Subject: [infinispan-dev] Hibernate Search v 5.8 and plans for beyond!
In-Reply-To: <CAA9R9O4knaVXhN4wd=SnjvZ2Z4DroEmUbiOyJvH+XTsXzpxbhA@mail.gmail.com>
References: <CAFm4XO11cGjoQvxVieggB92RB5q=_hE06Y2sp6JM_w7QpqHBCg@mail.gmail.com>
	<CAA9R9O4knaVXhN4wd=SnjvZ2Z4DroEmUbiOyJvH+XTsXzpxbhA@mail.gmail.com>
Message-ID: <CAFm4XO0GCav4AFDkz_URi5NibzX5mhThJx1HrcHYnHo2LU_twQ@mail.gmail.com>

On 14 June 2017 at 08:14, Sebastian Laskawiec <slaskawi at redhat.com> wrote:

> Hey Sanne,
>
> Looks good. Could you tell me where I can find some information about the
> CDI and Spring improvements? Maybe there's something we could integrate
> into our extensions...
>

?Yes! It would be great to have Infinispan's extensions set it up, as we
don't ship any such component. So this is left as an "exercise to the user"
for now.. Infinispan could do better ;)

There are pointers in the release blog, I linked to it in my previous email:
? - http://in.relation.to/2017/06/13/hibernate-search-5-8-0-Beta3/

Essentially our testsuite contains fully working examples.

Thanks,
Sanne




>
> Thanks,
> Sebastian
>
> On Tue, Jun 13, 2017 at 8:34 PM Sanne Grinovero <sanne at infinispan.org>
> wrote:
>
>> TLDR: the new Search SPI requires using the "free form" types - but
>> don't expect to be able to take advantage of it just yet.
>>
>> Hi all,
>>
>> we just released Hibernate Search 5.8.0.Beta3 and it's meant to be our
>> last beta!
>>
>> Progress and pace are looking very good.
>>
>> The details of the release can be found on our blog [1] as usual, but
>> I'd like to call out on several changes relevant for Infinispan.
>>
>> First and foremost, know that we'll be working on 6.0 as next big
>> thing; at that stage the API will see a significant redesign, focused
>> on making Lucene an optional implementation detail.
>>
>> This makes this 5.8 significant for Infinispan as it's the last one
>> you'll have until you're ready to break API.
>>
>> We've been working hard on addressing the needs of Infinispan, but in
>> several cases we had to make - and will still make - dramatic changes
>> in the SPI.
>>
>> Some benefits relevant to you all:
>>  - the "ram" directory changed name as you suggested just recently
>>  - statistics now include index size (how large is it to store it)
>>  - a proper API to define Analyzers programmatically, you could now
>> add "out of the box" default definitions for Infinispan server.
>>  - the same for Normalizers (a new concept, see the blog [1])
>>  - tons of other improvements and bugfixes ;) such as:
>>    -- better performance by finally being able to use JDK8 features
>>    -- Java 9 compatibility
>>    -- a critical performance improvement in Spatial bridges
>>    -- better Spring and CDI improvements
>>
>>
>> # A new type system is coming !
>>
>> And this is the main point.. however, it's not ready yet as it's a huge
>> change.
>>
>> The new type system - also named "free form" in all the meetings in
>> which we wished we had it - needs to start from the SPI and that's
>> partially included in this Beta.
>> (We postponed this for way too long by trying to match a good time to
>> break SPIs.. we'll just break it or it will never happen)
>>
>> I will soon send a PR to update Infinispan to *work fine* with the
>> current SPI changes, but I will refrain from refactoring anything yet,
>> so that we can validate it all in small steps.
>>
>> Still, many other existing SPI methods have been deprecated:
>>
>> As soon as that's merged, I'll need help to migrate away from all
>> those deprecated methods as I simply can't make progress on the Search
>> side without starting to delete a good amount of these deprecated
>> methods.
>> So please know this: most of those deprecated methods will be removed
>> from Hibernate Search soon; we can of course keep some of them around
>> if we identify strong needs, but identifying these points can only be
>> done by getting started.
>>
>> Please note that you can't yet plug-in a new type system to benefit
>> REST, Protobuf or other systems:
>>  - several components still make assumptions about this being backed
>> by annotated POJOs
>>  - there's no SPI yet to plugin alternative metadata
>>
>> There's also no API to extract indexed values from an alternative
>> system, although I'm sure you could keep using the current hacks for
>> this to work out: it's not possible to get this in 5.8 so it will have
>> to be 6+.
>>
>> Thanks,
>> Sanne
>>
>> 1 - http://in.relation.to/2017/06/13/hibernate-search-5-8-0-Beta3/
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> --
>
> SEBASTIAN ?ASKAWIEC
>
> INFINISPAN DEVELOPER
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170614/3c4615bc/attachment.html 

From slaskawi at redhat.com  Wed Jun 14 07:15:11 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 14 Jun 2017 11:15:11 +0000
Subject: [infinispan-dev] KUBE_PING changes
Message-ID: <CAA9R9O7U3w-if7XyLc3bRPM_GVyzr9ywChm+222EptrNSbA8Tg@mail.gmail.com>

Hey guys,

Just a heads up, I've just created a PR that upgrades KUBE_PING to
1.0.0.Beta1 [1]. As you probably seen in [2], 1.0.0.Beta1 was completely
rewritten and might behave slightly differently.

Here is a summary of changes:

   - The latest KUBE_PING doesn't require embedded HTTP server for
   discovery. Thus it is no longer required to expose port 8888 in Pods.
   - The number of dependencies has been decreased. Currently we only
   require JGroups and OAuth library.
   - The new KUBE_PING works only with JGroups 4. There will be no JGroups
   3 support.
   - Some of the environmental variables were shortened and we removed
   `OPENSHIFT` prefix. So if you use `OPENSHIFT_KUBE_PING_NAMESPACE`, you will
   need to change it to `KUBERNETES_NAMESPACE`. Please refer to [3] for more
   information.

I also switched default branch in Kubernetes Ping repository to master [4].

Thanks,
Sebastian

[1] https://github.com/infinispan/infinispan/pull/5201
[2] http://belaban.blogspot.ch/2017/05/running-infinispan-cluster-with.html
[3]
https://github.com/jgroups-extras/jgroups-kubernetes/blob/master/README.adoc
[4] https://github.com/jgroups-extras/jgroups-kubernetes
-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170614/8d9c3d5a/attachment-0001.html 

From galder at redhat.com  Wed Jun 14 08:58:38 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 14 Jun 2017 14:58:38 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <CAA9R9O7f0uTiAJbWgjK6PdUikqmZ87FPniE280Z4Mu8y-YbkSQ@mail.gmail.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
	<CAA9R9O7f0uTiAJbWgjK6PdUikqmZ87FPniE280Z4Mu8y-YbkSQ@mail.gmail.com>
Message-ID: <23341D75-AB71-48D4-907F-BD56C5FD15AC@redhat.com>


--
Galder Zamarre?o
Infinispan, Red Hat

> On 9 Jun 2017, at 08:37, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> 
> I agree with Alan here. Maven Central is a free "download area", so I wouldn't give it up for free. BTW, what is the point of creating and not shipping them?

I never said we wouldn't ship them. 

I'm happy with uber jars being produced but they should just not be available as a Maven dependency IMO.

So, you'd ship them just like the other non-Maven stuff we produce, direct downloads.

> 
> I would lean towards to removing them completely or limiting the number of use cases to the minimum e.g. we shouldn't support using infinispan-embedded and jcache; if jcache is essential it should be inside infinispan-embedded; the same for Spring integration modules - either we should put them in uber jars or say that you can use Spring integration with small jars.

How do you plan to stop that happening?

> 
> On Fri, Jun 9, 2017 at 5:05 AM Alan Field <afield at redhat.com> wrote:
> Wasn't the ability to add a single dependency to a project to start using Infinispan the whole purpose for the uber jars? I'm not trying to make an argument for keeping them, because I know they have caused many issues. I just think that if we are going to remove them from Maven, then there should be a way to achieve the same easy developer on boarding that uber jars were supposed to provide. Whether this is Maven project templates, or something else doesn't matter.
> 
> Thanks,
> Alan
> 
> ----- Original Message -----
> > From: "Tristan Tarrant" <ttarrant at redhat.com>
> > To: infinispan-dev at lists.jboss.org
> > Sent: Thursday, June 8, 2017 4:05:08 AM
> > Subject: Re: [infinispan-dev] Why JCache embedded has core as provided dependency
> >
> > I think we should turn off maven deployment for uber jars.
> >
> > Tristan
> >
> > On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
> > > On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
> > > <mailto:galder at redhat.com>> wrote:
> > >
> > >     As far as I see it:
> > >
> > >     * infinispan-embedded should never be a dependency in a Maven project.
> > >
> > >     * No uber jars should really be used as Maven dependencies because
> > >     all the exclusion that fine grained dependencies allow you to do
> > >     goes out of the window when all classes are inside a jar. This is
> > >     not just theory, I've personally had such issues.
> > >
> > >     * Uber jars are designed for Ant or other build tool users that
> > >     don't have a dependency resolution engine in place.
> > >
> > >     Cheers,
> > >
> > >     p.s. I thought we had already discussed this before?
> > >
> > >
> > >
> > > I totally agree. In addition, uberjars should not be an osgi bundle or a
> > > jboss module, for similar reasons.
> > >
> > > P.S: Even Ant has a dependency mgmt available, which is Ivy.
> > >
> > > Cheers,
> > > Gustavo
> > >
> > >     --
> > >     Galder Zamarre?o
> > >     Infinispan, Red Hat
> > >
> > >      > On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
> > >     <mailto:slaskawi at redhat.com>> wrote:
> > >      >
> > >      > Hey,
> > >      >
> > >      > The change was introduced by this commit [1] and relates to this
> > >     JIRAs [2][3]. The root cause is in [3].
> > >      >
> > >      > Imagine a scenario where you add JCache module to your together
> > >     infinispan-embedded. If your classpath was constructed in such a way
> > >     that infinispan-embedded was before infinispan-core (classpath is
> > >     scanned from left to right in standalone apps), we could get a
> > >     relocated (uber jars move some classes into other packages) logger.
> > >     That caused class mismatch errors. It is worth to mention that it
> > >     will happen to all relocated classes, logger was just an example.
> > >     And we need to relocate them, since a user might want to use his
> > >     own, newer version of DMR or any other library. So there's no
> > >     perfect solution here.
> > >      >
> > >      > Now a lot of time passed since then and we changed quite a few
> > >     things. So this topic probably needs to be revisited.
> > >      >
> > >      > So the first question that we should ask, shall we allow putting
> > >     jcache and infinispan-embedded together on the classpath. If the
> > >     answer is yes, I believe it should stay as it is (since the user
> > >     always have a choice whether he wants to use jcache with or without
> > >     uber jar). The same question needs to be asked for Spring modules as
> > >     well as all cache stores. The behavior needs to be consistent across
> > >     all those modules.
> > >      >
> > >      > If the answer is no (which is also valid because jcache is
> > >     already present in embedded uber jar), we should migrate all JBoss
> > >     Logging references to Infinispan Common Logging (as Tristan did here
> > >     [4]) and we can make infinispan-core as a compile time dependency to
> > >     jcache. Even though migrating to Infinispan logger is not necessary,
> > >     this way we won't break users app which used infinispan-embedded +
> > >     jcache approach. Of course the same applies to Spring and Cache
> > >     stores modules.
> > >      >
> > >      > I think the latter approach deserves some exploration. I would
> > >     vote for moving that way.
> > >      >
> > >      > Thanks,
> > >      > Sebastian
> > >      >
> > >      > [1]
> > >     https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
> > >     <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
> > >      > [2] https://issues.jboss.org/browse/ISPN-6295
> > >     <https://issues.jboss.org/browse/ISPN-6295>
> > >      > [3] https://issues.jboss.org/browse/ISPN-6132
> > >     <https://issues.jboss.org/browse/ISPN-6132>
> > >      > [4] https://github.com/infinispan/infinispan/pull/4140/files
> > >     <https://github.com/infinispan/infinispan/pull/4140/files>
> > >      >
> > >      >
> > >      > On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
> > >     <galder at redhat.com <mailto:galder at redhat.com>> wrote:
> > >      > Hi all,
> > >      >
> > >      > Re:
> > >     https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
> > >     <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
> > >      >
> > >      > St?phane makes a good point there, why did we make core provided
> > >     dependency? It does feel a bit of a pain that anyone that depends on
> > >     jcache embedded also needs to depend on core.
> > >      >
> > >      > Any more details behind this decision?
> > >      >
> > >      > Cheers,
> > >      > --
> > >      > Galder Zamarre?o
> > >      > Infinispan, Red Hat
> > >      >
> > >      > --
> > >      > SEBASTIAN ?ASKAWIEC
> > >      > INFINISPAN DEVELOPER
> > >      > Red Hat EMEA
> > >      >
> > >
> > >
> > >     _______________________________________________
> > >     infinispan-dev mailing list
> > >     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
> > >     https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > infinispan-dev mailing list
> > > infinispan-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >
> >
> > --
> > Tristan Tarrant
> > Infinispan Lead
> > JBoss, a division of Red Hat
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> -- 
> SEBASTIAN ?ASKAWIEC
> INFINISPAN DEVELOPER
> Red Hat EMEA
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From galder at redhat.com  Wed Jun 14 09:34:51 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 14 Jun 2017 15:34:51 +0200
Subject: [infinispan-dev] Why JCache embedded has core as provided
	dependency
In-Reply-To: <CAFm4XO2J5g8+8ZZwCxQZN+zfXrYzCeDw_C28z7vW_CVHNeuo5A@mail.gmail.com>
References: <D3257029-77FF-44AC-AE03-304D6816FF06@redhat.com>
	<CAA9R9O5PSAwT-e-E45i7FYW7b+y=qSjOjPzJATPjV+N_kDdkWg@mail.gmail.com>
	<EE3B0D25-9C00-43A0-B25B-51DAB8FE5B6C@redhat.com>
	<CAH8Ud1GCCdNA_+dWMKt0BzwwObLHKC0t+zp3b2SUi6EKewT-2Q@mail.gmail.com>
	<6a90fcdc-7e69-70bd-7c4d-15f864bd632a@redhat.com>
	<977331077.20454597.1496945073697.JavaMail.zimbra@redhat.com>
	<1f6fd37f-3068-3c3a-b8bb-4dd16eaae4c2@redhat.com>
	<1310904907.20805805.1497021185755.JavaMail.zimbra@redhat.com>
	<CAFm4XO2J5g8+8ZZwCxQZN+zfXrYzCeDw_C28z7vW_CVHNeuo5A@mail.gmail.com>
Message-ID: <3189D9FF-51EC-4075-B983-6DD682637EB3@redhat.com>

I don't yet have an opinion on this dependency that depends on everything, but I've created these two JIRAs which most of us seem to agree on:

* Remove unnecessary provided dependencies which make depending on Infinispan harder. [1]

* Uber jars should not be pushed to Maven. Instead they should be treated just like other zip distros that are not pushed. [2]

I think we should fix [1] in 9.0.x and [2] can wait to 9.1.x.

Cheers,

[1] https://issues.jboss.org/browse/ISPN-7930
[2] https://issues.jboss.org/browse/ISPN-7931
--
Galder Zamarre?o
Infinispan, Red Hat

> On 11 Jun 2017, at 20:35, Sanne Grinovero <sanne at infinispan.org> wrote:
> 
> On 9 June 2017 at 16:13, Alan Field <afield at redhat.com> wrote:
>> I really like this idea. It is similar to one of the solutions hinted at in the Netty issue that Gustavo pointed at. [1] The suggestion there was to replace their current uber jar that contains all of the shaded dependencies with a JAR that just depended on everything. Then you would get the benefit of being able to use a single Maven dependency to pull in all of the JAR files without the issues that shading brings.
> 
> The proposal is to bundle APIs only, but we'll always have tons of
> optional (and provided) extension points, e.g. we can't include all
> CacheStore(s), not least as many live in separate repositories.
> 
> Similarly, many "experimental modules" might provide additional APIs
> which will need to be accessed like the current Query API: *decoupled*
> - at least until there's an agreement for inclusion in such an
> umbrella API.
> This is unavoidable as someone might start working on such a module
> without telling us.. incidentally to foster a community of extensions
> I'd rather have our own API showcase how it works.
> 
> Also some Java EE APIs - e.g. the JTA API - should never be included;
> that's "best practice" in community standards - ugly and we could
> complain that Maven could deal better with it, containers could also
> deal better with it, but that's how it is.
> 
> So it's an improvement in the API but it's not sorting out the
> problems for people not familiar with dependency management. And I
> believe that's ok! Just making sure we're agreeing on the goals and
> non-goals.
> 
> Considering it's a very minor improvement in usability I'm not sold
> about this direction: you'll have ONE MORE JAR and one more layer of
> silly indirection to maintain. You might state it's trivial but I
> don't believe that, as the definition of which components to include
> is necessarily going to be fuild, makign this "API" more prone to wild
> changes.
> 
> The one liner to get a reference to the SearchFactory is no big deal
> and could be solved better by having CDI and Spring extensions - while
> maintaining good decoupling - but if it's the path to convince you all
> to take out the uber jars then by all means do it right now ;)
> 
> Thanks,
> Sanne
> 
>> 
>> Thanks,
>> Alan
>> 
>> [1] https://github.com/netty/netty/issues/4671
>> 
>> ----- Original Message -----
>>> From: "Radim Vansa" <rvansa at redhat.com>
>>> To: infinispan-dev at lists.jboss.org
>>> Sent: Friday, June 9, 2017 3:29:44 AM
>>> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided dependency
>>> 
>>> Katia has recently pointed out some usability flaws, and we have
>>> discussed a central point class that would allow you to explore the API:
>>> instead of *knowing* about org.infinispan.query.Search or
>>> org.infinispan.counters. EmbeddedCounterManagerFactory you'd just call
>>> 
>>> Infinispan ispn = Infinispan.newInstance();
>>> ispn.search().someQueryMethod(...);
>>> ispn.counters().someCounterMethod(...);
>>> ispn.cacheManager().getCache(...);
>>> 
>>> An umbrella module that would contain this 'discovery API' would need
>>> all the dependencies, so that would be a perfect replacement for the
>>> embedded maven artifact. Shouldn't be that much of a work to hack this
>>> together - how do you think that should be called? infinispan-api (but
>>> it would be nicer to reserve this if we ever manage to create the
>>> 'public API' module, with interfaces only), infinispan-facade,
>>> infinispan-surface? We could even use infinispan-embedded, but that
>>> would cause some confusion if we distributed infinispan-embedded uberjar
>>> and infinispan-embedded umbrella artifact.
>>> 
>>> Radim
>>> 
>>> On 06/08/2017 08:04 PM, Alan Field wrote:
>>>> Wasn't the ability to add a single dependency to a project to start using
>>>> Infinispan the whole purpose for the uber jars? I'm not trying to make an
>>>> argument for keeping them, because I know they have caused many issues. I
>>>> just think that if we are going to remove them from Maven, then there
>>>> should be a way to achieve the same easy developer on boarding that uber
>>>> jars were supposed to provide. Whether this is Maven project templates, or
>>>> something else doesn't matter.
>>>> 
>>>> Thanks,
>>>> Alan
>>>> 
>>>> ----- Original Message -----
>>>>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>>>>> To: infinispan-dev at lists.jboss.org
>>>>> Sent: Thursday, June 8, 2017 4:05:08 AM
>>>>> Subject: Re: [infinispan-dev] Why JCache embedded has core as provided
>>>>> dependency
>>>>> 
>>>>> I think we should turn off maven deployment for uber jars.
>>>>> 
>>>>> Tristan
>>>>> 
>>>>> On 6/7/17 5:10 PM, Gustavo Fernandes wrote:
>>>>>> On Wed, Jun 7, 2017 at 11:02 AM, Galder Zamarre?o <galder at redhat.com
>>>>>> <mailto:galder at redhat.com>> wrote:
>>>>>> 
>>>>>>     As far as I see it:
>>>>>> 
>>>>>>     * infinispan-embedded should never be a dependency in a Maven
>>>>>>     project.
>>>>>> 
>>>>>>     * No uber jars should really be used as Maven dependencies because
>>>>>>     all the exclusion that fine grained dependencies allow you to do
>>>>>>     goes out of the window when all classes are inside a jar. This is
>>>>>>     not just theory, I've personally had such issues.
>>>>>> 
>>>>>>     * Uber jars are designed for Ant or other build tool users that
>>>>>>     don't have a dependency resolution engine in place.
>>>>>> 
>>>>>>     Cheers,
>>>>>> 
>>>>>>     p.s. I thought we had already discussed this before?
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> I totally agree. In addition, uberjars should not be an osgi bundle or a
>>>>>> jboss module, for similar reasons.
>>>>>> 
>>>>>> P.S: Even Ant has a dependency mgmt available, which is Ivy.
>>>>>> 
>>>>>> Cheers,
>>>>>> Gustavo
>>>>>> 
>>>>>>     --
>>>>>>     Galder Zamarre?o
>>>>>>     Infinispan, Red Hat
>>>>>> 
>>>>>>> On 7 Jun 2017, at 11:50, Sebastian Laskawiec <slaskawi at redhat.com
>>>>>>     <mailto:slaskawi at redhat.com>> wrote:
>>>>>>> 
>>>>>>> Hey,
>>>>>>> 
>>>>>>> The change was introduced by this commit [1] and relates to this
>>>>>>     JIRAs [2][3]. The root cause is in [3].
>>>>>>> 
>>>>>>> Imagine a scenario where you add JCache module to your together
>>>>>>     infinispan-embedded. If your classpath was constructed in such a way
>>>>>>     that infinispan-embedded was before infinispan-core (classpath is
>>>>>>     scanned from left to right in standalone apps), we could get a
>>>>>>     relocated (uber jars move some classes into other packages) logger.
>>>>>>     That caused class mismatch errors. It is worth to mention that it
>>>>>>     will happen to all relocated classes, logger was just an example.
>>>>>>     And we need to relocate them, since a user might want to use his
>>>>>>     own, newer version of DMR or any other library. So there's no
>>>>>>     perfect solution here.
>>>>>>> 
>>>>>>> Now a lot of time passed since then and we changed quite a few
>>>>>>     things. So this topic probably needs to be revisited.
>>>>>>> 
>>>>>>> So the first question that we should ask, shall we allow putting
>>>>>>     jcache and infinispan-embedded together on the classpath. If the
>>>>>>     answer is yes, I believe it should stay as it is (since the user
>>>>>>     always have a choice whether he wants to use jcache with or without
>>>>>>     uber jar). The same question needs to be asked for Spring modules as
>>>>>>     well as all cache stores. The behavior needs to be consistent across
>>>>>>     all those modules.
>>>>>>> 
>>>>>>> If the answer is no (which is also valid because jcache is
>>>>>>     already present in embedded uber jar), we should migrate all JBoss
>>>>>>     Logging references to Infinispan Common Logging (as Tristan did here
>>>>>>     [4]) and we can make infinispan-core as a compile time dependency to
>>>>>>     jcache. Even though migrating to Infinispan logger is not necessary,
>>>>>>     this way we won't break users app which used infinispan-embedded +
>>>>>>     jcache approach. Of course the same applies to Spring and Cache
>>>>>>     stores modules.
>>>>>>> 
>>>>>>> I think the latter approach deserves some exploration. I would
>>>>>>     vote for moving that way.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Sebastian
>>>>>>> 
>>>>>>> [1]
>>>>>>     https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739
>>>>>>     <https://github.com/infinispan/infinispan/commit/720f158cce38d86b292e1ce77b75509342007739>
>>>>>>> [2] https://issues.jboss.org/browse/ISPN-6295
>>>>>>     <https://issues.jboss.org/browse/ISPN-6295>
>>>>>>> [3] https://issues.jboss.org/browse/ISPN-6132
>>>>>>     <https://issues.jboss.org/browse/ISPN-6132>
>>>>>>> [4] https://github.com/infinispan/infinispan/pull/4140/files
>>>>>>     <https://github.com/infinispan/infinispan/pull/4140/files>
>>>>>>> 
>>>>>>> 
>>>>>>> On Wed, Jun 7, 2017 at 11:19 AM Galder Zamarre?o
>>>>>>     <galder at redhat.com <mailto:galder at redhat.com>> wrote:
>>>>>>> Hi all,
>>>>>>> 
>>>>>>> Re:
>>>>>>     https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579
>>>>>>     <https://github.com/spring-projects/spring-boot/pull/9417#discussion_r120375579>
>>>>>>> 
>>>>>>> St?phane makes a good point there, why did we make core provided
>>>>>>     dependency? It does feel a bit of a pain that anyone that depends on
>>>>>>     jcache embedded also needs to depend on core.
>>>>>>> 
>>>>>>> Any more details behind this decision?
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> --
>>>>>>> Galder Zamarre?o
>>>>>>> Infinispan, Red Hat
>>>>>>> 
>>>>>>> --
>>>>>>> SEBASTIAN ?ASKAWIEC
>>>>>>> INFINISPAN DEVELOPER
>>>>>>> Red Hat EMEA
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>     _______________________________________________
>>>>>>     infinispan-dev mailing list
>>>>>>     infinispan-dev at lists.jboss.org
>>>>>>     <mailto:infinispan-dev at lists.jboss.org>
>>>>>>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> 
>>>>> --
>>>>> Tristan Tarrant
>>>>> Infinispan Lead
>>>>> JBoss, a division of Red Hat
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> 
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From galder at redhat.com  Wed Jun 14 11:35:17 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 14 Jun 2017 17:35:17 +0200
Subject: [infinispan-dev] Quick fix for ISPN-7710
Message-ID: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>

Hi all,

I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.

One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.

So, in essence, adding the following to org/infinispan/main/module.xml:

      <module name="org.infinispan.remote-query.server" optional="true"/>

Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.

Cheers,

[1] https://github.com/infinispan-demos/swiss-transport-datagrid
[2] https://issues.jboss.org/browse/ISPN-7710
--
Galder Zamarre?o
Infinispan, Red Hat



From belaban at mailbox.org  Thu Jun 15 01:37:24 2017
From: belaban at mailbox.org (Bela Ban)
Date: Thu, 15 Jun 2017 07:37:24 +0200
Subject: [infinispan-dev] KUBE_PING changes
In-Reply-To: <CAA9R9O7U3w-if7XyLc3bRPM_GVyzr9ywChm+222EptrNSbA8Tg@mail.gmail.com>
References: <CAA9R9O7U3w-if7XyLc3bRPM_GVyzr9ywChm+222EptrNSbA8Tg@mail.gmail.com>
Message-ID: <6b3dc945-449b-1c9b-1384-4a4cdde83eca@mailbox.org>

Thanks, Sebastian!

I assume testing on GKE and minikube/openshift was successful?


On 14/06/17 13:15, Sebastian Laskawiec wrote:
> Hey guys,
>
> Just a heads up, I've just created a PR that upgrades KUBE_PING to
> 1.0.0.Beta1 [1]. As you probably seen in [2], 1.0.0.Beta1 was completely
> rewritten and might behave slightly differently.
>
> Here is a summary of changes:
>
>   * The latest KUBE_PING doesn't require embedded HTTP server for
>     discovery. Thus it is no longer required to expose port 8888 in Pods.
>   * The number of dependencies has been decreased. Currently we only
>     require JGroups and OAuth library.
>   * The new KUBE_PING works only with JGroups 4. There will be no
>     JGroups 3 support.
>   * Some of the environmental variables were shortened and we removed
>     `OPENSHIFT` prefix. So if you use `OPENSHIFT_KUBE_PING_NAMESPACE`,
>     you will need to change it to `KUBERNETES_NAMESPACE`. Please refer
>     to [3] for more information.
>
> I also switched default branch in Kubernetes Ping repository to master [4].
>
> Thanks,
> Sebastian
>
> [1] https://github.com/infinispan/infinispan/pull/5201
> [2] http://belaban.blogspot.ch/2017/05/running-infinispan-cluster-with.html
> [3] https://github.com/jgroups-extras/jgroups-kubernetes/blob/master/README.adoc
> [4] https://github.com/jgroups-extras/jgroups-kubernetes
> --
>
> SEBASTIAN ?ASKAWIEC
>
> INFINISPAN DEVELOPER
>
> Red Hat EMEA <https://www.redhat.com/>
>
> <https://red.ht/sig>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

-- 
Bela Ban | http://www.jgroups.org


From slaskawi at redhat.com  Thu Jun 15 02:21:39 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 15 Jun 2017 06:21:39 +0000
Subject: [infinispan-dev] KUBE_PING changes
In-Reply-To: <6b3dc945-449b-1c9b-1384-4a4cdde83eca@mailbox.org>
References: <CAA9R9O7U3w-if7XyLc3bRPM_GVyzr9ywChm+222EptrNSbA8Tg@mail.gmail.com>
	<6b3dc945-449b-1c9b-1384-4a4cdde83eca@mailbox.org>
Message-ID: <CAA9R9O6Vhrd2No5dQd6NawuuAx5VbFdkKsFOvNs5aT0qp9ftRQ@mail.gmail.com>

Yep, no problems found!!!

I had also impression that the new implementation is "faster". Though I
haven't measured it... it just my impression.

Awesome work Bela!

On Thu, Jun 15, 2017 at 7:42 AM Bela Ban <belaban at mailbox.org> wrote:

> Thanks, Sebastian!
>
> I assume testing on GKE and minikube/openshift was successful?
>
>
> On 14/06/17 13:15, Sebastian Laskawiec wrote:
> > Hey guys,
> >
> > Just a heads up, I've just created a PR that upgrades KUBE_PING to
> > 1.0.0.Beta1 [1]. As you probably seen in [2], 1.0.0.Beta1 was completely
> > rewritten and might behave slightly differently.
> >
> > Here is a summary of changes:
> >
> >   * The latest KUBE_PING doesn't require embedded HTTP server for
> >     discovery. Thus it is no longer required to expose port 8888 in Pods.
> >   * The number of dependencies has been decreased. Currently we only
> >     require JGroups and OAuth library.
> >   * The new KUBE_PING works only with JGroups 4. There will be no
> >     JGroups 3 support.
> >   * Some of the environmental variables were shortened and we removed
> >     `OPENSHIFT` prefix. So if you use `OPENSHIFT_KUBE_PING_NAMESPACE`,
> >     you will need to change it to `KUBERNETES_NAMESPACE`. Please refer
> >     to [3] for more information.
> >
> > I also switched default branch in Kubernetes Ping repository to master
> [4].
> >
> > Thanks,
> > Sebastian
> >
> > [1] https://github.com/infinispan/infinispan/pull/5201
> > [2]
> http://belaban.blogspot.ch/2017/05/running-infinispan-cluster-with.html
> > [3]
> https://github.com/jgroups-extras/jgroups-kubernetes/blob/master/README.adoc
> > [4] https://github.com/jgroups-extras/jgroups-kubernetes
> > --
> >
> > SEBASTIAN ?ASKAWIEC
> >
> > INFINISPAN DEVELOPER
> >
> > Red Hat EMEA <https://www.redhat.com/>
> >
> > <https://red.ht/sig>
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
>
> --
> Bela Ban | http://www.jgroups.org
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170615/566e2f69/attachment.html 

From anistor at redhat.com  Thu Jun 15 05:05:31 2017
From: anistor at redhat.com (Adrian Nistor)
Date: Thu, 15 Jun 2017 12:05:31 +0300
Subject: [infinispan-dev] Quick fix for ISPN-7710
In-Reply-To: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>
References: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>
Message-ID: <c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>

Hi Galder,

this fix is acceptable for now as it quickly enables users to use 
CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the 
long run we would want users to be able to specify a custom marshaller 
class that comes from a user supplied module or even a deployment - the 
general case.

With the introduction of encoders and deprecation of compat mode we 
still have the same class loading issue in the general case. So I 
propose to refine a bit our approach and instead of specifying just a 
class name we should use a naming schema like "moduleId:slot:className", 
giving users the ability to specify a class that comes from a different 
module or deployment. I'm currently experimenting with this. I'll come 
back with results soon.

There are also other code bits that need to be deployed in the server 
ASAP: protostream entity marshallers, lucene analyzers. I'm thinking 
these could all benefit from the same solution.

Btw, what is the relation between ISPN-7814 and ISPN-7710 ?

Adrian

On 06/14/2017 06:35 PM, Galder Zamarre?o wrote:
> Hi all,
>
> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>
> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>
> So, in essence, adding the following to org/infinispan/main/module.xml:
>
>        <module name="org.infinispan.remote-query.server" optional="true"/>
>
> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>
> Cheers,
>
> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
> [2] https://issues.jboss.org/browse/ISPN-7710
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From galder at redhat.com  Thu Jun 15 08:40:28 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Thu, 15 Jun 2017 14:40:28 +0200
Subject: [infinispan-dev] Important feedback for transcoding work - Re:
	Quick fix for ISPN-7710
In-Reply-To: <c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>
References: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>
	<c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>
Message-ID: <A6A6C7CA-E7DB-4B79-8450-41A8C23AF8BF@redhat.com>

@Gustavo, some important info for your transcoding work below:

--
Galder Zamarre?o
Infinispan, Red Hat

> On 15 Jun 2017, at 11:05, Adrian Nistor <anistor at redhat.com> wrote:
> 
> Hi Galder,
> 
> this fix is acceptable for now as it quickly enables users to use CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the long run we would want users to be able to specify a custom marshaller class that comes from a user supplied module or even a deployment - the general case.
> 
> With the introduction of encoders and deprecation of compat mode we still have the same class loading issue in the general case. So I propose to refine a bit our approach and instead of specifying just a class name we should use a naming schema like "moduleId:slot:className", giving users the ability to specify a class that comes from a different module or deployment. I'm currently experimenting with this. I'll come back with results soon.
> 
> There are also other code bits that need to be deployed in the server ASAP: protostream entity marshallers, lucene analyzers. I'm thinking these could all benefit from the same solution.

I was able to easily get around the issue of deploying protostream entity marshallers by simply adding a server tasks that did that:

https://github.com/infinispan-demos/swiss-transport-datagrid/blob/master/analytics/analytics-domain/src/main/java/delays/java/stream/proto/AddProtobufTask.java

In fact, that server tasks acts serves as a way to add domain pojos to the system... So when the server starts receiving data, it can deserialize it without problems.

However, there's a potential problem here you might want to consider in your work: If I deploy the add protobuf task, write data, then redeploy the add protobuf task, then retrieve some data, the system blows up because the classloader of the domain POJOs has changed. So you'd start seeing ClassCastException errors...

That's why I think that even though in the past we'd store objects in deserialized form, this could be problematic because you're committing to domain objects with a given classloader...

The more I think about it, the more I think we should keep data only in binary format in the server. IOW, we should never try to keep it in deserialized format. That way, no matter how many times the domain objects are redeployed, assuming no compile-binary changes, the lazy transcoding would work without problems.

> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?

The relationship between them is explained here:

https://github.com/infinispan-demos/swiss-transport-datagrid#infinispan-server-docker-image

I would strongly recommend that you give that demo repository a try, you might get new ideas on the work you're doing.

Cheers,

> 
> Adrian
> 
> On 06/14/2017 06:35 PM, Galder Zamarre?o wrote:
>> Hi all,
>> 
>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>> 
>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>> 
>> So, in essence, adding the following to org/infinispan/main/module.xml:
>> 
>>       <module name="org.infinispan.remote-query.server" optional="true"/>
>> 
>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>> 
>> Cheers,
>> 
>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>> [2] https://issues.jboss.org/browse/ISPN-7710
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 



From sanne at infinispan.org  Thu Jun 15 09:06:41 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 15 Jun 2017 14:06:41 +0100
Subject: [infinispan-dev] Quick fix for ISPN-7710
In-Reply-To: <c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>
References: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>
	<c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>
Message-ID: <CAFm4XO1jgz7fxip4NS-csXsYhT_a7Se3+X5NC1vdsSQ4vQ=keg@mail.gmail.com>

On 15 June 2017 at 10:05, Adrian Nistor <anistor at redhat.com> wrote:
> Hi Galder,
>
> this fix is acceptable for now as it quickly enables users to use
> CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the
> long run we would want users to be able to specify a custom marshaller
> class that comes from a user supplied module or even a deployment - the
> general case.
>
> With the introduction of encoders and deprecation of compat mode we
> still have the same class loading issue in the general case. So I
> propose to refine a bit our approach and instead of specifying just a
> class name we should use a naming schema like "moduleId:slot:className",
> giving users the ability to specify a class that comes from a different
> module or deployment. I'm currently experimenting with this. I'll come
> back with results soon.

Great idea!
Can you also figure out how to identify loading something from the
user deployment ?
Maybe we could apply a simple convention, as in when the module
identification is missing then we look into TCCL, while when it's
provided we look for the specific module (exclusively) ?

Thanks,
Sanne

>
> There are also other code bits that need to be deployed in the server
> ASAP: protostream entity marshallers, lucene analyzers. I'm thinking
> these could all benefit from the same solution.
>
> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?
>
> Adrian
>
> On 06/14/2017 06:35 PM, Galder Zamarre?o wrote:
>> Hi all,
>>
>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>
>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>
>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>
>>        <module name="org.infinispan.remote-query.server" optional="true"/>
>>
>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>
>> Cheers,
>>
>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>> [2] https://issues.jboss.org/browse/ISPN-7710
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From anistor at redhat.com  Thu Jun 15 09:15:23 2017
From: anistor at redhat.com (Adrian Nistor)
Date: Thu, 15 Jun 2017 16:15:23 +0300
Subject: [infinispan-dev] Quick fix for ISPN-7710
In-Reply-To: <CAFm4XO1jgz7fxip4NS-csXsYhT_a7Se3+X5NC1vdsSQ4vQ=keg@mail.gmail.com>
References: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>
	<c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>
	<CAFm4XO1jgz7fxip4NS-csXsYhT_a7Se3+X5NC1vdsSQ4vQ=keg@mail.gmail.com>
Message-ID: <32f7ef40-c384-0173-cd34-cbccdc957cff@redhat.com>

Yup. That's exactly what I'm proposing. But today I stopped working on 
it because of some PR reviewing for the release. I hope to restart it 
and have a PoC by tomorrow.

On 06/15/2017 04:06 PM, Sanne Grinovero wrote:
> On 15 June 2017 at 10:05, Adrian Nistor <anistor at redhat.com> wrote:
>> Hi Galder,
>>
>> this fix is acceptable for now as it quickly enables users to use
>> CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the
>> long run we would want users to be able to specify a custom marshaller
>> class that comes from a user supplied module or even a deployment - the
>> general case.
>>
>> With the introduction of encoders and deprecation of compat mode we
>> still have the same class loading issue in the general case. So I
>> propose to refine a bit our approach and instead of specifying just a
>> class name we should use a naming schema like "moduleId:slot:className",
>> giving users the ability to specify a class that comes from a different
>> module or deployment. I'm currently experimenting with this. I'll come
>> back with results soon.
> Great idea!
> Can you also figure out how to identify loading something from the
> user deployment ?
> Maybe we could apply a simple convention, as in when the module
> identification is missing then we look into TCCL, while when it's
> provided we look for the specific module (exclusively) ?
>
> Thanks,
> Sanne
>
>> There are also other code bits that need to be deployed in the server
>> ASAP: protostream entity marshallers, lucene analyzers. I'm thinking
>> these could all benefit from the same solution.
>>
>> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?
>>
>> Adrian
>>
>> On 06/14/2017 06:35 PM, Galder Zamarre?o wrote:
>>> Hi all,
>>>
>>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>>
>>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>>
>>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>>
>>>         <module name="org.infinispan.remote-query.server" optional="true"/>
>>>
>>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>>
>>> Cheers,
>>>
>>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>>> [2] https://issues.jboss.org/browse/ISPN-7710
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From anistor at redhat.com  Thu Jun 15 09:25:40 2017
From: anistor at redhat.com (Adrian Nistor)
Date: Thu, 15 Jun 2017 16:25:40 +0300
Subject: [infinispan-dev] Important feedback for transcoding work - Re:
 Quick fix for ISPN-7710
In-Reply-To: <A6A6C7CA-E7DB-4B79-8450-41A8C23AF8BF@redhat.com>
References: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>
	<c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>
	<A6A6C7CA-E7DB-4B79-8450-41A8C23AF8BF@redhat.com>
Message-ID: <1cbc3eea-7a52-b13d-1a2a-0c66da7dec55@redhat.com>

Galder, I've seen AddProtobufTask in March or April when you mentioned 
this issue on the devlist; that approach can work for protostream 
marshallers or any other code bits that the Cache does not depend on 
during startup, and which can be deployed anytime later. In this 
category we currently have : filters, converters. These are currently 
deployed with the help of a DeploymentUnitProcessor, but we could have 
done it using a ServerTask as well. Now that we took the route of DUP, I 
think we should continue in a consistent manner and use it for other 
'deployables' we identify from now on, ie. the protobuf entity marshallers.

As for encoders, lucene analyzers, compatibility marshaller, event 
marshaller - these are all needed during cache startup. We need to come 
up with something for these, so I propose to look them up using the 
"moduleId:slot:className" convention.


On 06/15/2017 03:40 PM, Galder Zamarre?o wrote:
> @Gustavo, some important info for your transcoding work below:
>
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 15 Jun 2017, at 11:05, Adrian Nistor <anistor at redhat.com> wrote:
>>
>> Hi Galder,
>>
>> this fix is acceptable for now as it quickly enables users to use CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the long run we would want users to be able to specify a custom marshaller class that comes from a user supplied module or even a deployment - the general case.
>>
>> With the introduction of encoders and deprecation of compat mode we still have the same class loading issue in the general case. So I propose to refine a bit our approach and instead of specifying just a class name we should use a naming schema like "moduleId:slot:className", giving users the ability to specify a class that comes from a different module or deployment. I'm currently experimenting with this. I'll come back with results soon.
>>
>> There are also other code bits that need to be deployed in the server ASAP: protostream entity marshallers, lucene analyzers. I'm thinking these could all benefit from the same solution.
> I was able to easily get around the issue of deploying protostream entity marshallers by simply adding a server tasks that did that:
>
> https://github.com/infinispan-demos/swiss-transport-datagrid/blob/master/analytics/analytics-domain/src/main/java/delays/java/stream/proto/AddProtobufTask.java
>
> In fact, that server tasks acts serves as a way to add domain pojos to the system... So when the server starts receiving data, it can deserialize it without problems.
>
> However, there's a potential problem here you might want to consider in your work: If I deploy the add protobuf task, write data, then redeploy the add protobuf task, then retrieve some data, the system blows up because the classloader of the domain POJOs has changed. So you'd start seeing ClassCastException errors...
>
> That's why I think that even though in the past we'd store objects in deserialized form, this could be problematic because you're committing to domain objects with a given classloader...
>
> The more I think about it, the more I think we should keep data only in binary format in the server. IOW, we should never try to keep it in deserialized format. That way, no matter how many times the domain objects are redeployed, assuming no compile-binary changes, the lazy transcoding would work without problems.
>
>> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?
> The relationship between them is explained here:
>
> https://github.com/infinispan-demos/swiss-transport-datagrid#infinispan-server-docker-image
>
> I would strongly recommend that you give that demo repository a try, you might get new ideas on the work you're doing.
>
> Cheers,
>
>> Adrian
>>
>> On 06/14/2017 06:35 PM, Galder Zamarre?o wrote:
>>> Hi all,
>>>
>>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>>
>>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>>
>>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>>
>>>        <module name="org.infinispan.remote-query.server" optional="true"/>
>>>
>>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>>
>>> Cheers,
>>>
>>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>>> [2] https://issues.jboss.org/browse/ISPN-7710
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>


From mudokonman at gmail.com  Thu Jun 15 17:20:11 2017
From: mudokonman at gmail.com (William Burns)
Date: Thu, 15 Jun 2017 21:20:11 +0000
Subject: [infinispan-dev] Reactive Streams + RxJava
Message-ID: <CA+YCuUMG1sv8NeUOWCSkzdQf0n7O34Rp3u_w21LiE6TG-xBywA@mail.gmail.com>

I was thinking more about [1] and I find that I was going to implement
basically reactive streams. What we have now in github is similar but it
uses a very crude method of blocking the thread to prevent back pressure.
This can then cause severe issues as many users have found out when they
don't close iterator.

Unfortunately reactive streams is just a spec. I am proposing to add RxJava
[2] as a dependency [2] in the core module to provide access to reactive
streams and the various conversion methods. This library adds a bunch of
support for built in back pressure, transformations and much more which
would reduce the amount of code I would need to write substantially.

In regards to timing, I am thinking this is too close for 9.1, so maybe 9.2
or higher.

What do you guys think?

[1] https://issues.jboss.org/browse/ISPN-7865
[2] https://github.com/ReactiveX/RxJava
[3] https://mvnrepository.com/artifact/io.reactivex.rxjava2/rxjava/2.1.0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170615/500a9468/attachment.html 

From galder at redhat.com  Fri Jun 16 06:07:57 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 16 Jun 2017 12:07:57 +0200
Subject: [infinispan-dev] Important feedback for transcoding work - Re:
	Quick fix for ISPN-7710
In-Reply-To: <1cbc3eea-7a52-b13d-1a2a-0c66da7dec55@redhat.com>
References: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>
	<c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>
	<A6A6C7CA-E7DB-4B79-8450-41A8C23AF8BF@redhat.com>
	<1cbc3eea-7a52-b13d-1a2a-0c66da7dec55@redhat.com>
Message-ID: <62BF467B-791C-4141-85C1-ACA9AC96AED5@redhat.com>


--
Galder Zamarre?o
Infinispan, Red Hat

> On 15 Jun 2017, at 15:25, Adrian Nistor <anistor at redhat.com> wrote:
> 
> Galder, I've seen AddProtobufTask in March or April when you mentioned this issue on the devlist; that approach can work for protostream marshallers or any other code bits that the Cache does not depend on during startup, and which can be deployed anytime later. In this category we currently have : filters, converters. These are currently deployed with the help of a DeploymentUnitProcessor, but we could have done it using a ServerTask as well.

^ I'm not sure we had ServerTasks in place when we had filters and converters... But if we had server tasks then, we should have used that path. My bad if we didn't do it :\

> Now that we took the route of DUP, I think we should continue in a consistent manner and use it for other 'deployables' we identify from now on, ie. the protobuf entity marshallers.

^ Having written DUPs, I consider them to be a royal PITA. So, I'm against that.

> As for encoders, lucene analyzers, compatibility marshaller, event marshaller - these are all needed during cache startup. We need to come up with something for these, so I propose to look them up using the "moduleId:slot:className" convention.

As I see it, encoders/compatibility-marshaller/event-marshaller might not be needed on startup. If data is kept in binary and only deserialized lazily when needed, you only need them when you're going to do what you need...

To be more precise, yday Gustavo and I had a discussion and you'd need maybe pre-registering the encoders when a server tasks is deployed so that the server task, when executed in other nodes, can find the encoder. E.g. this is some pseudo code:

on ServerTask initialization:
Encoder encoder = new ProtoStreamEncoder(customMarshallers)
cache.registerEncoder("blah", encoder);

on ServerTask call:
cache.withEncoder("blah").values().stream()....

^ Btw, I know that's not how withEncoder() works... This would be much flexible to use than forcing encoders to be defined on startup. The server task deployment could ship the marshallers for POJOs since it already has to ship the POJOs to be able to use them.

I'd like to hear which use cases would really require those components you mention to be available on startup... assuming we don't by default transform data into their deserialized format and we keep data in binary format instead...

Cheers,

> 
> 
> On 06/15/2017 03:40 PM, Galder Zamarre?o wrote:
>> @Gustavo, some important info for your transcoding work below:
>> 
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>> 
>>> On 15 Jun 2017, at 11:05, Adrian Nistor <anistor at redhat.com> wrote:
>>> 
>>> Hi Galder,
>>> 
>>> this fix is acceptable for now as it quickly enables users to use CompatibilityProtoStreamMarshaller (provided by Infinispan), but in the long run we would want users to be able to specify a custom marshaller class that comes from a user supplied module or even a deployment - the general case.
>>> 
>>> With the introduction of encoders and deprecation of compat mode we still have the same class loading issue in the general case. So I propose to refine a bit our approach and instead of specifying just a class name we should use a naming schema like "moduleId:slot:className", giving users the ability to specify a class that comes from a different module or deployment. I'm currently experimenting with this. I'll come back with results soon.
>>> 
>>> There are also other code bits that need to be deployed in the server ASAP: protostream entity marshallers, lucene analyzers. I'm thinking these could all benefit from the same solution.
>> I was able to easily get around the issue of deploying protostream entity marshallers by simply adding a server tasks that did that:
>> 
>> https://github.com/infinispan-demos/swiss-transport-datagrid/blob/master/analytics/analytics-domain/src/main/java/delays/java/stream/proto/AddProtobufTask.java
>> 
>> In fact, that server tasks acts serves as a way to add domain pojos to the system... So when the server starts receiving data, it can deserialize it without problems.
>> 
>> However, there's a potential problem here you might want to consider in your work: If I deploy the add protobuf task, write data, then redeploy the add protobuf task, then retrieve some data, the system blows up because the classloader of the domain POJOs has changed. So you'd start seeing ClassCastException errors...
>> 
>> That's why I think that even though in the past we'd store objects in deserialized form, this could be problematic because you're committing to domain objects with a given classloader...
>> 
>> The more I think about it, the more I think we should keep data only in binary format in the server. IOW, we should never try to keep it in deserialized format. That way, no matter how many times the domain objects are redeployed, assuming no compile-binary changes, the lazy transcoding would work without problems.
>> 
>>> Btw, what is the relation between ISPN-7814 and ISPN-7710 ?
>> The relationship between them is explained here:
>> 
>> https://github.com/infinispan-demos/swiss-transport-datagrid#infinispan-server-docker-image
>> 
>> I would strongly recommend that you give that demo repository a try, you might get new ideas on the work you're doing.
>> 
>> Cheers,
>> 
>>> Adrian
>>> 
>>> On 06/14/2017 06:35 PM, Galder Zamarre?o wrote:
>>>> Hi all,
>>>> 
>>>> I'm seeing more and more people trying to do stuff like I did in [1] WRT to running server tasks in server.
>>>> 
>>>> One of the blockers is [2]. I know we have transcoding coming up but I wondered if we could implement the quick hack of referencing remote-query.server module from root org.infinispan module.
>>>> 
>>>> So, in essence, adding the following to org/infinispan/main/module.xml:
>>>> 
>>>>       <module name="org.infinispan.remote-query.server" optional="true"/>
>>>> 
>>>> Once ISPN-7710 is in place, along with ISPN-7814, users can run the demos in [1] without a custom server build.
>>>> 
>>>> Cheers,
>>>> 
>>>> [1] https://github.com/infinispan-demos/swiss-transport-datagrid
>>>> [2] https://issues.jboss.org/browse/ISPN-7710
>>>> --
>>>> Galder Zamarre?o
>>>> Infinispan, Red Hat
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
> 



From rory.odonnell at oracle.com  Fri Jun 16 06:10:37 2017
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Fri, 16 Jun 2017 11:10:37 +0100
Subject: [infinispan-dev] JDK 9 EA Build 174 & JDK 8u152 b04 are available
	on jdk.java.net
Message-ID: <d21b7f85-e731-26c9-ad9d-57018ee0220e@oracle.com>

Hi Galder,

*JDK 9 Early Access*  build 174  is available at : - jdk.java.net/9/

A summary of all the changes in this build are listed here 
<http://download.java.net/java/jdk9/changes/jdk-9+174.html>.

Changes which were introduced since the last availability email that may 
be of interest :

  * b172 - JDK-8179014 : JFileChooser with Windows look and feel crashes
    on win 10
  * b173 - JDK-8180582 : the bind to rmiregistry is rejected by
    registryFilter even though registryFilter is set
  * b174 - JDK-8181702  : deprecate for removal the following tool
    modules:jdk.xml.bind and jdk.xml.ws

*JDK 9 Schedule Update*

  * The new GA date for JDK 9 is now set to 2017-09-21 [1]
  * We have updated the OpenJDK JDK 9 project page [2] with the new
    milestones and GA date.


*JDK 8u152 Early Access*  build 04 is available at : - jdk.java.net/8/ 
<http://jdk.java.net/8/>

A summary of all the changes in this build are listed here 
<http://download.java.net/java/jdk8u152/changes/jdk8u152-b04.html>.
Information and schedules specific to OpenJDK 8u152 release [3]

Rgds,Rory

[1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2017-June/005867.html
[2] http://openjdk.java.net/projects/jdk9/
[3] http://openjdk.java.net/projects/jdk8u/releases/8u152.html

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170616/8525fee9/attachment.html 

From vrigamon at redhat.com  Fri Jun 16 12:30:55 2017
From: vrigamon at redhat.com (Vittorio Rigamonti)
Date: Fri, 16 Jun 2017 18:30:55 +0200
Subject: [infinispan-dev] please IGNORE 9.0.2 Release
Message-ID: <CADE_1G88LEPpvLhCAw5atn3ZY-FzNHVKDL52NY-1qNDTRHy7ug@mail.gmail.com>

Hi All,

by mistake I've published a 9.0.2.Final which is actually a copy of
9.1.0.Beta1 release.

Please ignore it, next good release for 9.0.x series will be 9.0.3.Final...
coming soon.

Sorry for the added entropy.

-- 

Vittorio Rigamonti

Senior Software Engineer

Red Hat

<https://www.redhat.com>

Milan, Italy

vrigamon at redhat.com

irc: rigazilla
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170616/f914f99c/attachment-0001.html 

From ttarrant at redhat.com  Mon Jun 19 06:22:52 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 19 Jun 2017 12:22:52 +0200
Subject: [infinispan-dev] 9.1.0 endgame
Message-ID: <6068203b-004e-5cb7-4dec-c78e7b784fa1@redhat.com>

Hi all,

I have updated Jira with the next milestones for 9.1.0

9.1.0.CR1 - 30th June
9.1.0.Final - 14th July

This extends the traditional minor-release timebox by two weeks, 
essential because the features we wanted for Beta were a bit late.

 From now until the end we should only consider features whose PR have 
been open for a while (e.g. scattered cache). Bug fixes and minor 
enhancements to existing features are obviously allowed.

Anything else will need to be shunted to 9.2.0.

Tristan
-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From dan.berindei at gmail.com  Mon Jun 19 07:17:21 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 19 Jun 2017 14:17:21 +0300
Subject: [infinispan-dev] Important feedback for transcoding work - Re:
 Quick fix for ISPN-7710
In-Reply-To: <62BF467B-791C-4141-85C1-ACA9AC96AED5@redhat.com>
References: <8538223C-2F4B-4E18-B325-7F77A1298619@redhat.com>
	<c795e59e-60bf-89ca-40db-30784c1ee5e5@redhat.com>
	<A6A6C7CA-E7DB-4B79-8450-41A8C23AF8BF@redhat.com>
	<1cbc3eea-7a52-b13d-1a2a-0c66da7dec55@redhat.com>
	<62BF467B-791C-4141-85C1-ACA9AC96AED5@redhat.com>
Message-ID: <CA+nfvwTWwDT4G=zn5Yqe2u4bL5J5g_PVaUgLknDmLx2fEckUeg@mail.gmail.com>

On Fri, Jun 16, 2017 at 1:07 PM, Galder Zamarre?o <galder at redhat.com> wrote:
>
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>> On 15 Jun 2017, at 15:25, Adrian Nistor <anistor at redhat.com> wrote:
>>
>> Galder, I've seen AddProtobufTask in March or April when you mentioned this issue on the devlist; that approach can work for protostream marshallers or any other code bits that the Cache does not depend on during startup, and which can be deployed anytime later. In this category we currently have : filters, converters. These are currently deployed with the help of a DeploymentUnitProcessor, but we could have done it using a ServerTask as well.
>
> ^ I'm not sure we had ServerTasks in place when we had filters and converters... But if we had server tasks then, we should have used that path. My bad if we didn't do it :\
>
>> Now that we took the route of DUP, I think we should continue in a consistent manner and use it for other 'deployables' we identify from now on, ie. the protobuf entity marshallers.
>
> ^ Having written DUPs, I consider them to be a royal PITA. So, I'm against that.
>
>> As for encoders, lucene analyzers, compatibility marshaller, event marshaller - these are all needed during cache startup. We need to come up with something for these, so I propose to look them up using the "moduleId:slot:className" convention.
>
> As I see it, encoders/compatibility-marshaller/event-marshaller might not be needed on startup. If data is kept in binary and only deserialized lazily when needed, you only need them when you're going to do what you need...
>

What if you start a node and a client immediately tries to register an
even listener?

Not sure about the others, but for the lucene analyzers, I assume some
configurations will have to analyze/index entries that we receive via
state transfer during startup.

Dan


From emmanuel at hibernate.org  Mon Jun 19 07:29:02 2017
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Mon, 19 Jun 2017 13:29:02 +0200
Subject: [infinispan-dev] Reactive Streams + RxJava
In-Reply-To: <CA+YCuUMG1sv8NeUOWCSkzdQf0n7O34Rp3u_w21LiE6TG-xBywA@mail.gmail.com>
References: <CA+YCuUMG1sv8NeUOWCSkzdQf0n7O34Rp3u_w21LiE6TG-xBywA@mail.gmail.com>
Message-ID: <F2FEECE4-8178-44E5-9DAB-82B77FAA4452@hibernate.org>

You?re thinking about a pure implementation play, correct? RxJava or the Reactive Stream constructs would not be exposed to the user as API. Am I correct?
Also for posterity, we had backchannel chats about it and you said you felt vert.x was not necessarily addressing your needs. Could you elaborate a bit here?

Emmanuel

> On 15 Jun 2017, at 23:20, William Burns <mudokonman at gmail.com> wrote:
> 
> I was thinking more about [1] and I find that I was going to implement basically reactive streams. What we have now in github is similar but it uses a very crude method of blocking the thread to prevent back pressure. This can then cause severe issues as many users have found out when they don't close iterator.
> 
> Unfortunately reactive streams is just a spec. I am proposing to add RxJava [2] as a dependency [2] in the core module to provide access to reactive streams and the various conversion methods. This library adds a bunch of support for built in back pressure, transformations and much more which would reduce the amount of code I would need to write substantially.
> 
> In regards to timing, I am thinking this is too close for 9.1, so maybe 9.2 or higher.
> 
> What do you guys think?
> 
> [1] https://issues.jboss.org/browse/ISPN-7865 <https://issues.jboss.org/browse/ISPN-7865>
> [2] https://github.com/ReactiveX/RxJava <https://github.com/ReactiveX/RxJava>
> [3] https://mvnrepository.com/artifact/io.reactivex.rxjava2/rxjava/2.1.0 <https://mvnrepository.com/artifact/io.reactivex.rxjava2/rxjava/2.1.0>_______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170619/19fd0266/attachment.html 

From emmanuel at hibernate.org  Mon Jun 19 07:34:40 2017
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Mon, 19 Jun 2017 13:34:40 +0200
Subject: [infinispan-dev] 9.1.0 endgame
In-Reply-To: <6068203b-004e-5cb7-4dec-c78e7b784fa1@redhat.com>
References: <6068203b-004e-5cb7-4dec-c78e7b784fa1@redhat.com>
Message-ID: <7A87C5DB-1EED-4880-BCAC-D21110483861@hibernate.org>


> On 19 Jun 2017, at 12:22, Tristan Tarrant <ttarrant at redhat.com> wrote:
> 
> 9.1.0.Final - 14th July

Better be named Bastille one way or another.
https://www.ratebeer.com/beer/revolution-bastille/179085/

From vrigamon at redhat.com  Mon Jun 19 08:46:59 2017
From: vrigamon at redhat.com (Vittorio Rigamonti)
Date: Mon, 19 Jun 2017 14:46:59 +0200
Subject: [infinispan-dev] Infinispan 9.1.0.Beta1 Released
Message-ID: <CADE_1G8dOiY=0_Zdivb-myeitudmhAedNDbBmMBjs5Jssxmh9A@mail.gmail.com>

Dear All,

Infinispan 9.1.0.Beta1 is out:

http://blog.infinispan.org/2017/06/infinispan-910beta1.html

Cheers

-- 

Vittorio Rigamonti

Senior Software Engineer

Red Hat

<https://www.redhat.com>

Milan, Italy

vrigamon at redhat.com

irc: rigazilla
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170619/2f124aca/attachment.html 

From ttarrant at redhat.com  Mon Jun 19 09:06:05 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 19 Jun 2017 15:06:05 +0200
Subject: [infinispan-dev] 9.1.0 endgame
In-Reply-To: <7A87C5DB-1EED-4880-BCAC-D21110483861@hibernate.org>
References: <6068203b-004e-5cb7-4dec-c78e7b784fa1@redhat.com>
	<7A87C5DB-1EED-4880-BCAC-D21110483861@hibernate.org>
Message-ID: <97e641b1-b218-f976-6f49-5b719d209a07@redhat.com>

Awesome name  :)

Tristan

On 6/19/17 1:34 PM, Emmanuel Bernard wrote:
> 
>> On 19 Jun 2017, at 12:22, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>
>> 9.1.0.Final - 14th July
> 
> Better be named Bastille one way or another.
> https://www.ratebeer.com/beer/revolution-bastille/179085/
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From mudokonman at gmail.com  Mon Jun 19 09:25:01 2017
From: mudokonman at gmail.com (William Burns)
Date: Mon, 19 Jun 2017 13:25:01 +0000
Subject: [infinispan-dev] Reactive Streams + RxJava
In-Reply-To: <F2FEECE4-8178-44E5-9DAB-82B77FAA4452@hibernate.org>
References: <CA+YCuUMG1sv8NeUOWCSkzdQf0n7O34Rp3u_w21LiE6TG-xBywA@mail.gmail.com>
	<F2FEECE4-8178-44E5-9DAB-82B77FAA4452@hibernate.org>
Message-ID: <CA+YCuUM=8rWRss6AD_sjWmMG6cw9SqPVfxt-m4AgeD8ZOnD_0A@mail.gmail.com>

On Mon, Jun 19, 2017 at 8:34 AM Emmanuel Bernard <emmanuel at hibernate.org>
wrote:

> You?re thinking about a pure implementation play, correct? RxJava or the
> Reactive Stream constructs would not be exposed to the user as API. Am I
> correct?
>

Yes, that is correct. This is only for internal usage. My thought is maybe
when we finally move to Java 9, we may expose the Flow API at some point,
but nothing any time soon.


> Also for posterity, we had backchannel chats about it and you said you
> felt vert.x was not necessarily addressing your needs. Could you elaborate
> a bit here?
>

The main difference is that RxJava [1] has a full suite of utility methods
around reactive streams [2]. Vert.x has some APIs around reactive streams
[3], but it is rather limited.

RxJava is about publishing/consuming your own streams without tying to any
threading model (everything can be done on main thread for example). And it
provides methods of mapping streams to others and even returning blocking
APIs back to the user. Some methods in particular I was looking at were
merge, delay, blockingIterable, rebatch and some others.

Vert.x is really beneficial with reactive programming built upon existing
tools (HTTP, JDBC and others). It also has the event bus and others but
this is built on the vert.x event loop, which we just don't need in
Infinispan. We already have our own threading model and communication over
JGroups.

My usage is to handle multiple streams of data coming from different
locations and merging them together and doing some additional
transformations. I could do this with either of the API, however RxJava
already did a lot of heavy lifting I would have to do otherwise.

Also another side note is that it might be helpful for vert.x to change
over to something more like the upcoming Flow API from Java 9 (maybe [4]
which is pretty much identical currently). RxJava uses this as a base class
for it's Publish/Subscribe.

[1] https://github.com/ReactiveX/RxJava/tree/2.x/src/main/java/io/reactivex
[2]
http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#blockingIterable()
<https://github.com/ReactiveX/RxJava/blob/2.x/src/main/java/io/reactivex/Flowable.java>
[3]
https://github.com/eclipse/vert.x/tree/master/src/main/java/io/vertx/core/streams
[4]
https://mvnrepository.com/artifact/org.reactivestreams/reactive-streams/1.0.0


>
> Emmanuel
>
> On 15 Jun 2017, at 23:20, William Burns <mudokonman at gmail.com> wrote:
>
> I was thinking more about [1] and I find that I was going to implement
> basically reactive streams. What we have now in github is similar but it
> uses a very crude method of blocking the thread to prevent back pressure.
> This can then cause severe issues as many users have found out when they
> don't close iterator.
>
> Unfortunately reactive streams is just a spec. I am proposing to add
> RxJava [2] as a dependency [2] in the core module to provide access to
> reactive streams and the various conversion methods. This library adds a
> bunch of support for built in back pressure, transformations and much more
> which would reduce the amount of code I would need to write substantially.
>
> In regards to timing, I am thinking this is too close for 9.1, so maybe
> 9.2 or higher.
>
> What do you guys think?
>
> [1] https://issues.jboss.org/browse/ISPN-7865
> [2] https://github.com/ReactiveX/RxJava
> [3] https://mvnrepository.com/artifact/io.reactivex.rxjava2/rxjava/2.1.0
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170619/d38a60f0/attachment-0001.html 

From galder at redhat.com  Tue Jun 20 03:57:17 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 20 Jun 2017 09:57:17 +0200
Subject: [infinispan-dev] Reactive Streams + RxJava
In-Reply-To: <CA+YCuUMG1sv8NeUOWCSkzdQf0n7O34Rp3u_w21LiE6TG-xBywA@mail.gmail.com>
References: <CA+YCuUMG1sv8NeUOWCSkzdQf0n7O34Rp3u_w21LiE6TG-xBywA@mail.gmail.com>
Message-ID: <AD0204A8-8EDF-43A5-98C9-6B0414D7AFB4@redhat.com>


--
Galder Zamarre?o
Infinispan, Red Hat

> On 15 Jun 2017, at 23:20, William Burns <mudokonman at gmail.com> wrote:
> 
> I was thinking more about [1] and I find that I was going to implement basically reactive streams. What we have now in github is similar but it uses a very crude method of blocking the thread to prevent back pressure. This can then cause severe issues as many users have found out when they don't close iterator.
> 
> Unfortunately reactive streams is just a spec.

Not just as spec. Java 9 will come with a set of interfaces that allows you to implement reactive streams:
http://download.java.net/java/jdk9/docs/api/java/util/concurrent/Flow.html

> I am proposing to add RxJava [2] as a dependency [2] in the core module to provide access to reactive streams and the various conversion methods. This library adds a bunch of support for built in back pressure, transformations and much more which would reduce the amount of code I would need to write substantially.

I don't think I like the idea of tying our core module to RxJava. Back when I was doing the functional map API we did consider using something like RxJava for the async multi-value streams, but decided against it to avoid tying ourselves to that.

A better alternative, if the APIs fit your use case, would be to depend on Java 9 APIs (oswego might have Java 8 version of those interfaces) and use maybe RxJava as the implementation of that API, if there's a suitable implementation for it.

If there's no choice but to use RxJava for the API part, I'd rather that happens in separate module to core.

Cheers,

> 
> In regards to timing, I am thinking this is too close for 9.1, so maybe 9.2 or higher.
> 
> What do you guys think?
> 
> [1] https://issues.jboss.org/browse/ISPN-7865
> [2] https://github.com/ReactiveX/RxJava
> [3] https://mvnrepository.com/artifact/io.reactivex.rxjava2/rxjava/2.1.0
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From galder at redhat.com  Tue Jun 20 03:59:28 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 20 Jun 2017 09:59:28 +0200
Subject: [infinispan-dev] Reactive Streams + RxJava
In-Reply-To: <CA+YCuUM=8rWRss6AD_sjWmMG6cw9SqPVfxt-m4AgeD8ZOnD_0A@mail.gmail.com>
References: <CA+YCuUMG1sv8NeUOWCSkzdQf0n7O34Rp3u_w21LiE6TG-xBywA@mail.gmail.com>
	<F2FEECE4-8178-44E5-9DAB-82B77FAA4452@hibernate.org>
	<CA+YCuUM=8rWRss6AD_sjWmMG6cw9SqPVfxt-m4AgeD8ZOnD_0A@mail.gmail.com>
Message-ID: <4DBB2484-BA57-4097-8E6E-297A45D94CA9@redhat.com>


--
Galder Zamarre?o
Infinispan, Red Hat

> On 19 Jun 2017, at 15:25, William Burns <mudokonman at gmail.com> wrote:
> 
> 
> 
> On Mon, Jun 19, 2017 at 8:34 AM Emmanuel Bernard <emmanuel at hibernate.org> wrote:
> You?re thinking about a pure implementation play, correct? RxJava or the Reactive Stream constructs would not be exposed to the user as API. Am I correct?
> 
> Yes, that is correct. This is only for internal usage. My thought is maybe when we finally move to Java 9, we may expose the Flow API at some point, but nothing any time soon.

^ Ok, that wasn't clear in the original email.

If it's only for internal usage and no RxJava APIs are exposed, then I'm fine with it.

>  
> Also for posterity, we had backchannel chats about it and you said you felt vert.x was not necessarily addressing your needs. Could you elaborate a bit here?
> 
> The main difference is that RxJava [1] has a full suite of utility methods around reactive streams [2]. Vert.x has some APIs around reactive streams [3], but it is rather limited.
> 
> RxJava is about publishing/consuming your own streams without tying to any threading model (everything can be done on main thread for example). And it provides methods of mapping streams to others and even returning blocking APIs back to the user. Some methods in particular I was looking at were merge, delay, blockingIterable, rebatch and some others.
> 
> Vert.x is really beneficial with reactive programming built upon existing tools (HTTP, JDBC and others). It also has the event bus and others but this is built on the vert.x event loop, which we just don't need in Infinispan. We already have our own threading model and communication over JGroups.
> 
> My usage is to handle multiple streams of data coming from different locations and merging them together and doing some additional transformations. I could do this with either of the API, however RxJava already did a lot of heavy lifting I would have to do otherwise.
> 
> Also another side note is that it might be helpful for vert.x to change over to something more like the upcoming Flow API from Java 9 (maybe [4] which is pretty much identical currently). RxJava uses this as a base class for it's Publish/Subscribe.
> 
> [1] https://github.com/ReactiveX/RxJava/tree/2.x/src/main/java/io/reactivex
> [2] http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#blockingIterable()
> [3] https://github.com/eclipse/vert.x/tree/master/src/main/java/io/vertx/core/streams
> [4] https://mvnrepository.com/artifact/org.reactivestreams/reactive-streams/1.0.0
>  
> 
> Emmanuel
> 
>> On 15 Jun 2017, at 23:20, William Burns <mudokonman at gmail.com> wrote:
>> 
>> I was thinking more about [1] and I find that I was going to implement basically reactive streams. What we have now in github is similar but it uses a very crude method of blocking the thread to prevent back pressure. This can then cause severe issues as many users have found out when they don't close iterator.
>> 
>> Unfortunately reactive streams is just a spec. I am proposing to add RxJava [2] as a dependency [2] in the core module to provide access to reactive streams and the various conversion methods. This library adds a bunch of support for built in back pressure, transformations and much more which would reduce the amount of code I would need to write substantially.
>> 
>> In regards to timing, I am thinking this is too close for 9.1, so maybe 9.2 or higher.
>> 
>> What do you guys think?
>> 
>> [1] https://issues.jboss.org/browse/ISPN-7865
>> [2] https://github.com/ReactiveX/RxJava
>> [3] https://mvnrepository.com/artifact/io.reactivex.rxjava2/rxjava/2.1.0
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From mudokonman at gmail.com  Tue Jun 20 08:30:54 2017
From: mudokonman at gmail.com (William Burns)
Date: Tue, 20 Jun 2017 12:30:54 +0000
Subject: [infinispan-dev] Reactive Streams + RxJava
In-Reply-To: <4DBB2484-BA57-4097-8E6E-297A45D94CA9@redhat.com>
References: <CA+YCuUMG1sv8NeUOWCSkzdQf0n7O34Rp3u_w21LiE6TG-xBywA@mail.gmail.com>
	<F2FEECE4-8178-44E5-9DAB-82B77FAA4452@hibernate.org>
	<CA+YCuUM=8rWRss6AD_sjWmMG6cw9SqPVfxt-m4AgeD8ZOnD_0A@mail.gmail.com>
	<4DBB2484-BA57-4097-8E6E-297A45D94CA9@redhat.com>
Message-ID: <CA+YCuUPsPXPZox0eUVQh-2WgZxRV4ShMdtr0=+FjLam0WPaiCg@mail.gmail.com>

On Tue, Jun 20, 2017 at 3:59 AM Galder Zamarre?o <galder at redhat.com> wrote:

>
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
> > On 19 Jun 2017, at 15:25, William Burns <mudokonman at gmail.com> wrote:
> >
> >
> >
> > On Mon, Jun 19, 2017 at 8:34 AM Emmanuel Bernard <emmanuel at hibernate.org>
> wrote:
> > You?re thinking about a pure implementation play, correct? RxJava or the
> Reactive Stream constructs would not be exposed to the user as API. Am I
> correct?
> >
> > Yes, that is correct. This is only for internal usage. My thought is
> maybe when we finally move to Java 9, we may expose the Flow API at some
> point, but nothing any time soon.
>
> ^ Ok, that wasn't clear in the original email.
>
> If it's only for internal usage and no RxJava APIs are exposed, then I'm
> fine with it.
>

Yeah I am definitely not trying to expose any new API. And if we expose
anything related to reactive streams I would recommend the reactive streams
specification [1], which includes interfaces that the Java 9 Flow API is
based off of.

But what I am proposing is the addition of RxJava for is to help with the
internal handling of distributed iterator and possibly cache loader to
reduce the number of threads.

Another side point is we could even use reactive streams for the remote
iterator in the client (in fact it is a perfect fit there, much less
complex). But I am not proposing that at this point as I don't really want
to add a dependency in the client.

[1]
https://github.com/reactive-streams/reactive-streams-jvm/tree/v1.0.0#specification


>
> >
> > Also for posterity, we had backchannel chats about it and you said you
> felt vert.x was not necessarily addressing your needs. Could you elaborate
> a bit here?
> >
> > The main difference is that RxJava [1] has a full suite of utility
> methods around reactive streams [2]. Vert.x has some APIs around reactive
> streams [3], but it is rather limited.
> >
> > RxJava is about publishing/consuming your own streams without tying to
> any threading model (everything can be done on main thread for example).
> And it provides methods of mapping streams to others and even returning
> blocking APIs back to the user. Some methods in particular I was looking at
> were merge, delay, blockingIterable, rebatch and some others.
> >
> > Vert.x is really beneficial with reactive programming built upon
> existing tools (HTTP, JDBC and others). It also has the event bus and
> others but this is built on the vert.x event loop, which we just don't need
> in Infinispan. We already have our own threading model and communication
> over JGroups.
> >
> > My usage is to handle multiple streams of data coming from different
> locations and merging them together and doing some additional
> transformations. I could do this with either of the API, however RxJava
> already did a lot of heavy lifting I would have to do otherwise.
> >
> > Also another side note is that it might be helpful for vert.x to change
> over to something more like the upcoming Flow API from Java 9 (maybe [4]
> which is pretty much identical currently). RxJava uses this as a base class
> for it's Publish/Subscribe.
> >
> > [1]
> https://github.com/ReactiveX/RxJava/tree/2.x/src/main/java/io/reactivex
> > [2]
> http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#blockingIterable()
> > [3]
> https://github.com/eclipse/vert.x/tree/master/src/main/java/io/vertx/core/streams
> > [4]
> https://mvnrepository.com/artifact/org.reactivestreams/reactive-streams/1.0.0
> >
> >
> > Emmanuel
> >
> >> On 15 Jun 2017, at 23:20, William Burns <mudokonman at gmail.com> wrote:
> >>
> >> I was thinking more about [1] and I find that I was going to implement
> basically reactive streams. What we have now in github is similar but it
> uses a very crude method of blocking the thread to prevent back pressure.
> This can then cause severe issues as many users have found out when they
> don't close iterator.
> >>
> >> Unfortunately reactive streams is just a spec. I am proposing to add
> RxJava [2] as a dependency [2] in the core module to provide access to
> reactive streams and the various conversion methods. This library adds a
> bunch of support for built in back pressure, transformations and much more
> which would reduce the amount of code I would need to write substantially.
> >>
> >> In regards to timing, I am thinking this is too close for 9.1, so maybe
> 9.2 or higher.
> >>
> >> What do you guys think?
> >>
> >> [1] https://issues.jboss.org/browse/ISPN-7865
> >> [2] https://github.com/ReactiveX/RxJava
> >> [3]
> https://mvnrepository.com/artifact/io.reactivex.rxjava2/rxjava/2.1.0
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170620/4e0dd7ad/attachment.html 

From ttarrant at redhat.com  Wed Jun 21 08:19:18 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Wed, 21 Jun 2017 14:19:18 +0200
Subject: [infinispan-dev] 9.1.0 endgame
In-Reply-To: <6068203b-004e-5cb7-4dec-c78e7b784fa1@redhat.com>
References: <6068203b-004e-5cb7-4dec-c78e7b784fa1@redhat.com>
Message-ID: <c2210507-eaa1-c167-a7b8-e4365d0abe0e@redhat.com>

The release wrangler for 9.1.0.CR1 will be Will.
Will willl (haha !) perform the release on the 29th. This is a hard date.

Tristan

On 6/19/17 12:22 PM, Tristan Tarrant wrote:
> Hi all,
> 
> I have updated Jira with the next milestones for 9.1.0
> 
> 9.1.0.CR1 - 30th June
> 9.1.0.Final - 14th July
> 
> This extends the traditional minor-release timebox by two weeks, 
> essential because the features we wanted for Beta were a bit late.
> 
>  From now until the end we should only consider features whose PR have 
> been open for a while (e.g. scattered cache). Bug fixes and minor 
> enhancements to existing features are obviously allowed.
> 
> Anything else will need to be shunted to 9.2.0.
> 
> Tristan

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From sanne at infinispan.org  Wed Jun 21 08:57:06 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 21 Jun 2017 13:57:06 +0100
Subject: [infinispan-dev] 9.1.0 endgame
In-Reply-To: <c2210507-eaa1-c167-a7b8-e4365d0abe0e@redhat.com>
References: <6068203b-004e-5cb7-4dec-c78e7b784fa1@redhat.com>
	<c2210507-eaa1-c167-a7b8-e4365d0abe0e@redhat.com>
Message-ID: <CAFm4XO0WYzz_4ZbUh=Nu-m_RuVUx1Uf+AmV6W6eNj4t-yy63vg@mail.gmail.com>

On 21 June 2017 at 13:19, Tristan Tarrant <ttarrant at redhat.com> wrote:
> The release wrangler for 9.1.0.CR1 will be Will.
> Will willl (haha !) perform the release on the 29th. This is a hard date.

The date is hard but the month and year intentionally omitted :)

>
> Tristan
>
> On 6/19/17 12:22 PM, Tristan Tarrant wrote:
>> Hi all,
>>
>> I have updated Jira with the next milestones for 9.1.0
>>
>> 9.1.0.CR1 - 30th June
>> 9.1.0.Final - 14th July
>>
>> This extends the traditional minor-release timebox by two weeks,
>> essential because the features we wanted for Beta were a bit late.
>>
>>  From now until the end we should only consider features whose PR have
>> been open for a while (e.g. scattered cache). Bug fixes and minor
>> enhancements to existing features are obviously allowed.
>>
>> Anything else will need to be shunted to 9.2.0.
>>
>> Tristan
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From vrigamon at redhat.com  Wed Jun 21 12:57:45 2017
From: vrigamon at redhat.com (Vittorio Rigamonti)
Date: Wed, 21 Jun 2017 18:57:45 +0200
Subject: [infinispan-dev] Infinispan 9.0.3.Final and 8.2.7.Final released
Message-ID: <CADE_1G-aUHVP=J7d0+WiJtVherttr4N2HbBEEemEhz5SfCgs=w@mail.gmail.com>

Dear all,

Infinispan 9.0.3.Final and 8.2.7.Final are out!

Check the blog post here:

http://blog.infinispan.org/2017/06/infinispan-903final-
and-827final-are-out.html

Cheers


-- 

Vittorio Rigamonti

Senior Software Engineer

Red Hat

<https://www.redhat.com>

Milan, Italy

vrigamon at redhat.com

irc: rigazilla
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170621/1cfb6207/attachment.html 

From belaban at mailbox.org  Mon Jun 26 05:03:55 2017
From: belaban at mailbox.org (Bela Ban)
Date: Mon, 26 Jun 2017 11:03:55 +0200
Subject: [infinispan-dev] Non-blocking JGroups
Message-ID: <8b42ca1e-e008-e398-613a-faacd411df31@mailbox.org>

FYI: http://belaban.blogspot.ch/2017/06/non-blocking-jgroups.html
-- 
Bela Ban | http://www.jgroups.org


From dan.berindei at gmail.com  Mon Jun 26 10:56:55 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 26 Jun 2017 17:56:55 +0300
Subject: [infinispan-dev] Weekly IRC meeting logs 2017-06-26
Message-ID: <CA+nfvwTUTUEVdytsXAui6zKJ1oHkhLoCD0yUXTYHuCrqBAOWHw@mail.gmail.com>

Hi everyone

Here are the logs of today's meeting on #infinispan:

http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2017/infinispan.2017-06-26-14.03.html

Cheers
Dan

From rvansa at redhat.com  Tue Jun 27 03:31:08 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 27 Jun 2017 09:31:08 +0200
Subject: [infinispan-dev] Write-only commands
Message-ID: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>

Hi,

I am working on entry version history (again). In Como we've discussed 
that previous values are needed for (continuous) query and reliable 
listeners, so I wonder what should we do with functional write-only 
commands. These are different to commands with flags, because flags 
(other than ignore return value) are expected to break something. I see 
the available options as:

1) run write-only commands 'optimized', ignoring any querying and such 
(warn user that he will break it)

2) run write-only without any optimization, rendering them useless

3) detect when querying is set up (ignoring listeners and maybe other 
stuff that could get broken)

4) remove write-only commands completely (and probably functional 
listeners as well because these will lose their purpose)

Right now I am inclined towards 4). There could be some internal use 
(e.g. multimaps) that could use 1) which is ran without a fancy setup, 
though, but it's asking for trouble.

WDYT?

Radim

-- 

Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From sanne at infinispan.org  Tue Jun 27 06:28:28 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 27 Jun 2017 11:28:28 +0100
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
Message-ID: <CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>

On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:

Hi,

I am working on entry version history (again). In Como we've discussed
that previous values are needed for (continuous) query and reliable
listeners,


Index based queries also require the previous value on a write - unless we
can get "strongly typed caches" giving guarantees about the class to
represent the content of a cache to be unique.

Essentially we only need to know the type of the previous object. It might
be worth having a way to load the type metadata if the previous value only.

so I wonder what should we do with functional write-only
commands. These are different to commands with flags, because flags
(other than ignore return value) are expected to break something.


Sorry I hope to not derail the thread but let's remind that we hope to
evolve beyond "flags are expected to break stuff" ; we never got to it but
search the mailing list.

Since flags are exposed to the user I would rather they're not allowed to
break things.
Could they be treated as hints? Ignore the flag (and warn?) if the used
 configuration/integrations veto them.

Alternatively, let's remove them from API. Remember "The Jokre" POC was
intentionally designed to explore pushing the limits on performance w/o end
users having to solve puzzles, such as learning details about these flags
and their possible side effects.

So assuming they become either "safe" or internal, maybe you can take
advantage of them?

I see
the available options as:

1) run write-only commands 'optimized', ignoring any querying and such
(warn user that he will break it)

2) run write-only without any optimization, rendering them useless

3) detect when querying is set up (ignoring listeners and maybe other
stuff that could get broken)


Might be useful for making a POC work, but I believe query will be very
likely to be often enabled.
Having an either / or switch for different features in Infinispan will make
it harder to use and understand, so I'd rather see work on the right design
as taking temporary shortcuts risks baking into stone features which we
later struggle to fix or maintain.


4) remove write-only commands completely (and probably functional
listeners as well because these will lose their purpose)


+1 to remove "unconditional writes", at least an entry version check should
be applied.
I believe we had already pointed out this would eventually happen, pretty
much for the reasons you're hitting now.


Right now I am inclined towards 4). There could be some internal use
(e.g. multimaps) that could use 1) which is ran without a fancy setup,
though, but it's asking for trouble.


I agree!

Thanks


WDYT?

Radim

--

Radim Vansa <rvansa at redhat.com>
JBoss Performance Team

_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170627/f9a9e9c8/attachment.html 

From anistor at redhat.com  Tue Jun 27 07:43:43 2017
From: anistor at redhat.com (Adrian Nistor)
Date: Tue, 27 Jun 2017 14:43:43 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
Message-ID: <8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>

I've said this in a previous thread on this same issue, I will repeat 
myself as many times as needed.

Continuous queries require the previous value itself, not just knowledge 
of the type of the previous value. Strongly typed caches solve no 
problem here.

So if we half-fix query but leave CQ broken I will be half-happy (ie. 
very depressed) :)

I'd remove these commands completely or possibly remove them just from 
public API and keep them internal.

Adrian


On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>
>
> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com 
> <mailto:rvansa at redhat.com>> wrote:
>
>     Hi,
>
>     I am working on entry version history (again). In Como we've discussed
>     that previous values are needed for (continuous) query and reliable
>     listeners, 
>
>
> Index based queries also require the previous value on a write - 
> unless we can get "strongly typed caches" giving guarantees about the 
> class to represent the content of a cache to be unique.
>
> Essentially we only need to know the type of the previous object. It 
> might be worth having a way to load the type metadata if the previous 
> value only.
>
>     so I wonder what should we do with functional write-only
>     commands. These are different to commands with flags, because flags
>     (other than ignore return value) are expected to break something.
>
>
> Sorry I hope to not derail the thread but let's remind that we hope to 
> evolve beyond "flags are expected to break stuff" ; we never got to it 
> but search the mailing list.
>
> Since flags are exposed to the user I would rather they're not allowed 
> to break things.
> Could they be treated as hints? Ignore the flag (and warn?) if the 
> used  configuration/integrations veto them.
>
> Alternatively, let's remove them from API. Remember "The Jokre" POC 
> was intentionally designed to explore pushing the limits on 
> performance w/o end users having to solve puzzles, such as learning 
> details about these flags and their possible side effects.
>
> So assuming they become either "safe" or internal, maybe you can take 
> advantage of them?
>
>     I see
>     the available options as:
>
>     1) run write-only commands 'optimized', ignoring any querying and such
>     (warn user that he will break it)
>
>     2) run write-only without any optimization, rendering them useless
>
>     3) detect when querying is set up (ignoring listeners and maybe other
>     stuff that could get broken)
>
>
> Might be useful for making a POC work, but I believe query will be 
> very likely to be often enabled.
> Having an either / or switch for different features in Infinispan will 
> make it harder to use and understand, so I'd rather see work on the 
> right design as taking temporary shortcuts risks baking into stone 
> features which we later struggle to fix or maintain.
>
>
>     4) remove write-only commands completely (and probably functional
>     listeners as well because these will lose their purpose)
>
>
> +1 to remove "unconditional writes", at least an entry version check 
> should be applied.
> I believe we had already pointed out this would eventually happen, 
> pretty much for the reasons you're hitting now.
>
>
>     Right now I am inclined towards 4). There could be some internal use
>     (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>     though, but it's asking for trouble.
>
>
> I agree!
>
> Thanks
>
>
>     WDYT?
>
>     Radim
>
>     --
>
>     Radim Vansa <rvansa at redhat.com <mailto:rvansa at redhat.com>>
>     JBoss Performance Team
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170627/2080793f/attachment-0001.html 

From dan.berindei at gmail.com  Tue Jun 27 09:54:56 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Tue, 27 Jun 2017 16:54:56 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
Message-ID: <CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>

On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
> I've said this in a previous thread on this same issue, I will repeat myself
> as many times as needed.
>
> Continuous queries require the previous value itself, not just knowledge of
> the type of the previous value. Strongly typed caches solve no problem here.
>
> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
> depressed) :)
>
> I'd remove these commands completely or possibly remove them just from
> public API and keep them internal.
>

+1 to remove the flags from the public API. Most of them are not safe
for applications to use, and ignoring them when they can lead to
inconsistencies would make them useless.

E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
know when it is safe to skip the delete statement, and it relies on
the application making a (possibly wrong) choice.

IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
that applications use it right now. If query or listeners need the
previous value, then we should load it internally, but hide it from
the user.

But removing it opens another discussion: should we replace it in the
public API with a new method AdvancedCache.ignoreReturnValues(), or
should we make it the default and add a method
AdvancedCache.forceReturnPreviousValues()?

>
>
>
> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>
>
>
> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>
> Hi,
>
> I am working on entry version history (again). In Como we've discussed
> that previous values are needed for (continuous) query and reliable
> listeners,
>
>
> Index based queries also require the previous value on a write - unless we
> can get "strongly typed caches" giving guarantees about the class to
> represent the content of a cache to be unique.
>
> Essentially we only need to know the type of the previous object. It might
> be worth having a way to load the type metadata if the previous value only.
>
> so I wonder what should we do with functional write-only
> commands. These are different to commands with flags, because flags
> (other than ignore return value) are expected to break something.
>
>
> Sorry I hope to not derail the thread but let's remind that we hope to
> evolve beyond "flags are expected to break stuff" ; we never got to it but
> search the mailing list.
>
> Since flags are exposed to the user I would rather they're not allowed to
> break things.
> Could they be treated as hints? Ignore the flag (and warn?) if the used
> configuration/integrations veto them.
>
> Alternatively, let's remove them from API. Remember "The Jokre" POC was
> intentionally designed to explore pushing the limits on performance w/o end
> users having to solve puzzles, such as learning details about these flags
> and their possible side effects.
>
> So assuming they become either "safe" or internal, maybe you can take
> advantage of them?
>
> I see
> the available options as:
>
> 1) run write-only commands 'optimized', ignoring any querying and such
> (warn user that he will break it)
>
> 2) run write-only without any optimization, rendering them useless
>
> 3) detect when querying is set up (ignoring listeners and maybe other
> stuff that could get broken)
>
>
> Might be useful for making a POC work, but I believe query will be very
> likely to be often enabled.
> Having an either / or switch for different features in Infinispan will make
> it harder to use and understand, so I'd rather see work on the right design
> as taking temporary shortcuts risks baking into stone features which we
> later struggle to fix or maintain.
>

I vote for this option.

Query, listeners, and other components that need the previous value
should not just assume that the application knows better, they should
be able to change how operations works based on their needs. Of
course, the reverse is also true: if the application uses write-only
commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
be possible for the user to detect why the previous values are still
loaded.

>
> 4) remove write-only commands completely (and probably functional
> listeners as well because these will lose their purpose)
>
>
> +1 to remove "unconditional writes", at least an entry version check should
> be applied.
> I believe we had already pointed out this would eventually happen, pretty
> much for the reasons you're hitting now.
>

IMO version checks should be done internally, we shouldn't force the
users of the functional API to deal with versions themselves because
we know how hard making write skew checks work is for us :)

And I wouldn't go as far as to remove the functional listeners,
instead I would change them so that read-write listeners are invoked
on write-only operations and they force the loading of the previous
value. I would also add a way for the regular listeners to say whether
they need the previous value or not.

>
> Right now I am inclined towards 4). There could be some internal use
> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
> though, but it's asking for trouble.
>
>
> I agree!
>
> Thanks
>
>
> WDYT?
>
> Radim
>

Cheers
Dan

From sanne at infinispan.org  Tue Jun 27 17:00:56 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 27 Jun 2017 23:00:56 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
Message-ID: <CAFm4XO3Aqi=v2QPEySj9wYN9XgQtms4shvr-2yBZ++F=a+9dbQ@mail.gmail.com>

On 27 June 2017 at 13:43, Adrian Nistor <anistor at redhat.com> wrote:
> I've said this in a previous thread on this same issue, I will repeat myself
> as many times as needed.
>
> Continuous queries require the previous value itself, not just knowledge of
> the type of the previous value. Strongly typed caches solve no problem here.

Why do you need to repeat this, Radim already captured this
requirement for CQ in the premise of this very thread?

>
> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
> depressed) :)

There must be a misunderstanding. I merely highlighted the needs for
indexed queries, and point out that it might be useful to have *yet
another* form of previous-version loading as this specific
circumstance would just need some metadata. I'd never suggest to
replace or remove the capability to load the "full" previous version,
definitely not meaning to suggest to break CQ and many other essential
use cases.

>
> I'd remove these commands completely or possibly remove them just from
> public API and keep them internal.
>
> Adrian
>
>
>
> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>
>
>
> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>
> Hi,
>
> I am working on entry version history (again). In Como we've discussed
> that previous values are needed for (continuous) query and reliable
> listeners,
>
>
> Index based queries also require the previous value on a write - unless we
> can get "strongly typed caches" giving guarantees about the class to
> represent the content of a cache to be unique.
>
> Essentially we only need to know the type of the previous object. It might
> be worth having a way to load the type metadata if the previous value only.
>
> so I wonder what should we do with functional write-only
> commands. These are different to commands with flags, because flags
> (other than ignore return value) are expected to break something.
>
>
> Sorry I hope to not derail the thread but let's remind that we hope to
> evolve beyond "flags are expected to break stuff" ; we never got to it but
> search the mailing list.
>
> Since flags are exposed to the user I would rather they're not allowed to
> break things.
> Could they be treated as hints? Ignore the flag (and warn?) if the used
> configuration/integrations veto them.
>
> Alternatively, let's remove them from API. Remember "The Jokre" POC was
> intentionally designed to explore pushing the limits on performance w/o end
> users having to solve puzzles, such as learning details about these flags
> and their possible side effects.
>
> So assuming they become either "safe" or internal, maybe you can take
> advantage of them?
>
> I see
> the available options as:
>
> 1) run write-only commands 'optimized', ignoring any querying and such
> (warn user that he will break it)
>
> 2) run write-only without any optimization, rendering them useless
>
> 3) detect when querying is set up (ignoring listeners and maybe other
> stuff that could get broken)
>
>
> Might be useful for making a POC work, but I believe query will be very
> likely to be often enabled.
> Having an either / or switch for different features in Infinispan will make
> it harder to use and understand, so I'd rather see work on the right design
> as taking temporary shortcuts risks baking into stone features which we
> later struggle to fix or maintain.
>
>
> 4) remove write-only commands completely (and probably functional
> listeners as well because these will lose their purpose)
>
>
> +1 to remove "unconditional writes", at least an entry version check should
> be applied.
> I believe we had already pointed out this would eventually happen, pretty
> much for the reasons you're hitting now.
>
>
> Right now I am inclined towards 4). There could be some internal use
> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
> though, but it's asking for trouble.
>
>
> I agree!
>
> Thanks
>
>
> WDYT?
>
> Radim
>
> --
>
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>

From anistor at redhat.com  Tue Jun 27 17:53:03 2017
From: anistor at redhat.com (Adrian Nistor)
Date: Wed, 28 Jun 2017 00:53:03 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CAFm4XO3Aqi=v2QPEySj9wYN9XgQtms4shvr-2yBZ++F=a+9dbQ@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CAFm4XO3Aqi=v2QPEySj9wYN9XgQtms4shvr-2yBZ++F=a+9dbQ@mail.gmail.com>
Message-ID: <0c4706ce-f74b-db37-d10d-58ef938d7a1c@redhat.com>

The needs for continuous query are stronger than and include the needs 
of normal query.
So by fixing what is needed for continuous query we can all live a happy 
life without needing to provide "*yet another* form of previous-version 
loading". The reverse is not true.
You'll have to convince me this extra mechanism's added complexity is 
worth the effort :)


On 06/28/2017 12:00 AM, Sanne Grinovero wrote:
> On 27 June 2017 at 13:43, Adrian Nistor <anistor at redhat.com> wrote:
>> I've said this in a previous thread on this same issue, I will repeat myself
>> as many times as needed.
>>
>> Continuous queries require the previous value itself, not just knowledge of
>> the type of the previous value. Strongly typed caches solve no problem here.
> Why do you need to repeat this, Radim already captured this
> requirement for CQ in the premise of this very thread?
>
>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>> depressed) :)
> There must be a misunderstanding. I merely highlighted the needs for
> indexed queries, and point out that it might be useful to have *yet
> another* form of previous-version loading as this specific
> circumstance would just need some metadata. I'd never suggest to
> replace or remove the capability to load the "full" previous version,
> definitely not meaning to suggest to break CQ and many other essential
> use cases.
>
>> I'd remove these commands completely or possibly remove them just from
>> public API and keep them internal.
>>
>> Adrian
>>
>>
>>
>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>
>>
>>
>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>
>> Hi,
>>
>> I am working on entry version history (again). In Como we've discussed
>> that previous values are needed for (continuous) query and reliable
>> listeners,
>>
>>
>> Index based queries also require the previous value on a write - unless we
>> can get "strongly typed caches" giving guarantees about the class to
>> represent the content of a cache to be unique.
>>
>> Essentially we only need to know the type of the previous object. It might
>> be worth having a way to load the type metadata if the previous value only.
>>
>> so I wonder what should we do with functional write-only
>> commands. These are different to commands with flags, because flags
>> (other than ignore return value) are expected to break something.
>>
>>
>> Sorry I hope to not derail the thread but let's remind that we hope to
>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>> search the mailing list.
>>
>> Since flags are exposed to the user I would rather they're not allowed to
>> break things.
>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>> configuration/integrations veto them.
>>
>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>> intentionally designed to explore pushing the limits on performance w/o end
>> users having to solve puzzles, such as learning details about these flags
>> and their possible side effects.
>>
>> So assuming they become either "safe" or internal, maybe you can take
>> advantage of them?
>>
>> I see
>> the available options as:
>>
>> 1) run write-only commands 'optimized', ignoring any querying and such
>> (warn user that he will break it)
>>
>> 2) run write-only without any optimization, rendering them useless
>>
>> 3) detect when querying is set up (ignoring listeners and maybe other
>> stuff that could get broken)
>>
>>
>> Might be useful for making a POC work, but I believe query will be very
>> likely to be often enabled.
>> Having an either / or switch for different features in Infinispan will make
>> it harder to use and understand, so I'd rather see work on the right design
>> as taking temporary shortcuts risks baking into stone features which we
>> later struggle to fix or maintain.
>>
>>
>> 4) remove write-only commands completely (and probably functional
>> listeners as well because these will lose their purpose)
>>
>>
>> +1 to remove "unconditional writes", at least an entry version check should
>> be applied.
>> I believe we had already pointed out this would eventually happen, pretty
>> much for the reasons you're hitting now.
>>
>>
>> Right now I am inclined towards 4). There could be some internal use
>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>> though, but it's asking for trouble.
>>
>>
>> I agree!
>>
>> Thanks
>>
>>
>> WDYT?
>>
>> Radim
>>
>> --
>>
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>


From rvansa at redhat.com  Wed Jun 28 03:03:15 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 28 Jun 2017 09:03:15 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <0c4706ce-f74b-db37-d10d-58ef938d7a1c@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CAFm4XO3Aqi=v2QPEySj9wYN9XgQtms4shvr-2yBZ++F=a+9dbQ@mail.gmail.com>
	<0c4706ce-f74b-db37-d10d-58ef938d7a1c@redhat.com>
Message-ID: <66f50447-7082-3a65-0173-5ee0919ecf30@redhat.com>

I am with Adrian here, we want the core as simple as possible. The more 
ifs, the more bugs - it's hard to track all the edge cases. And besides 
declarative approach (saying that this cache can contain only entities 
X, Y and Z) and only checking that, I think that there's little space 
for 'loading just the metadata', because there are (currently) only two 
sources if the entry is not present: remote nodes and cache store. The 
actual read operation is cheapish, the cost is in going to the other 
node/DB machine. The same for maintenance perspective: you need to check 
for errors & have the correct, the type of value (just metadata/whole 
prev value) does not change it.

R.

On 06/27/2017 11:53 PM, Adrian Nistor wrote:
> The needs for continuous query are stronger than and include the needs
> of normal query.
> So by fixing what is needed for continuous query we can all live a happy
> life without needing to provide "*yet another* form of previous-version
> loading". The reverse is not true.
> You'll have to convince me this extra mechanism's added complexity is
> worth the effort :)
>
>
> On 06/28/2017 12:00 AM, Sanne Grinovero wrote:
>> On 27 June 2017 at 13:43, Adrian Nistor <anistor at redhat.com> wrote:
>>> I've said this in a previous thread on this same issue, I will repeat myself
>>> as many times as needed.
>>>
>>> Continuous queries require the previous value itself, not just knowledge of
>>> the type of the previous value. Strongly typed caches solve no problem here.
>> Why do you need to repeat this, Radim already captured this
>> requirement for CQ in the premise of this very thread?
>>
>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>> depressed) :)
>> There must be a misunderstanding. I merely highlighted the needs for
>> indexed queries, and point out that it might be useful to have *yet
>> another* form of previous-version loading as this specific
>> circumstance would just need some metadata. I'd never suggest to
>> replace or remove the capability to load the "full" previous version,
>> definitely not meaning to suggest to break CQ and many other essential
>> use cases.
>>
>>> I'd remove these commands completely or possibly remove them just from
>>> public API and keep them internal.
>>>
>>> Adrian
>>>
>>>
>>>
>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>
>>>
>>>
>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>
>>> Hi,
>>>
>>> I am working on entry version history (again). In Como we've discussed
>>> that previous values are needed for (continuous) query and reliable
>>> listeners,
>>>
>>>
>>> Index based queries also require the previous value on a write - unless we
>>> can get "strongly typed caches" giving guarantees about the class to
>>> represent the content of a cache to be unique.
>>>
>>> Essentially we only need to know the type of the previous object. It might
>>> be worth having a way to load the type metadata if the previous value only.
>>>
>>> so I wonder what should we do with functional write-only
>>> commands. These are different to commands with flags, because flags
>>> (other than ignore return value) are expected to break something.
>>>
>>>
>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>> search the mailing list.
>>>
>>> Since flags are exposed to the user I would rather they're not allowed to
>>> break things.
>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>> configuration/integrations veto them.
>>>
>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>> intentionally designed to explore pushing the limits on performance w/o end
>>> users having to solve puzzles, such as learning details about these flags
>>> and their possible side effects.
>>>
>>> So assuming they become either "safe" or internal, maybe you can take
>>> advantage of them?
>>>
>>> I see
>>> the available options as:
>>>
>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>> (warn user that he will break it)
>>>
>>> 2) run write-only without any optimization, rendering them useless
>>>
>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>> stuff that could get broken)
>>>
>>>
>>> Might be useful for making a POC work, but I believe query will be very
>>> likely to be often enabled.
>>> Having an either / or switch for different features in Infinispan will make
>>> it harder to use and understand, so I'd rather see work on the right design
>>> as taking temporary shortcuts risks baking into stone features which we
>>> later struggle to fix or maintain.
>>>
>>>
>>> 4) remove write-only commands completely (and probably functional
>>> listeners as well because these will lose their purpose)
>>>
>>>
>>> +1 to remove "unconditional writes", at least an entry version check should
>>> be applied.
>>> I believe we had already pointed out this would eventually happen, pretty
>>> much for the reasons you're hitting now.
>>>
>>>
>>> Right now I am inclined towards 4). There could be some internal use
>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>> though, but it's asking for trouble.
>>>
>>>
>>> I agree!
>>>
>>> Thanks
>>>
>>>
>>> WDYT?
>>>
>>> Radim
>>>
>>> --
>>>
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From rvansa at redhat.com  Wed Jun 28 03:17:18 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 28 Jun 2017 09:17:18 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
Message-ID: <3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>

On 06/27/2017 03:54 PM, Dan Berindei wrote:
> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>> I've said this in a previous thread on this same issue, I will repeat myself
>> as many times as needed.
>>
>> Continuous queries require the previous value itself, not just knowledge of
>> the type of the previous value. Strongly typed caches solve no problem here.
>>
>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>> depressed) :)
>>
>> I'd remove these commands completely or possibly remove them just from
>> public API and keep them internal.
>>
> +1 to remove the flags from the public API. Most of them are not safe
> for applications to use, and ignoring them when they can lead to
> inconsistencies would make them useless.
>
> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
> know when it is safe to skip the delete statement, and it relies on
> the application making a (possibly wrong) choice.
>
> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
> that applications use it right now. If query or listeners need the
> previous value, then we should load it internally, but hide it from
> the user.
>
> But removing it opens another discussion: should we replace it in the
> public API with a new method AdvancedCache.ignoreReturnValues(), or
> should we make it the default and add a method
> AdvancedCache.forceReturnPreviousValues()?

Please don't derail the thread.

>
>>
>>
>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>
>>
>>
>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>
>> Hi,
>>
>> I am working on entry version history (again). In Como we've discussed
>> that previous values are needed for (continuous) query and reliable
>> listeners,
>>
>>
>> Index based queries also require the previous value on a write - unless we
>> can get "strongly typed caches" giving guarantees about the class to
>> represent the content of a cache to be unique.
>>
>> Essentially we only need to know the type of the previous object. It might
>> be worth having a way to load the type metadata if the previous value only.
>>
>> so I wonder what should we do with functional write-only
>> commands. These are different to commands with flags, because flags
>> (other than ignore return value) are expected to break something.
>>
>>
>> Sorry I hope to not derail the thread but let's remind that we hope to
>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>> search the mailing list.
>>
>> Since flags are exposed to the user I would rather they're not allowed to
>> break things.
>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>> configuration/integrations veto them.
>>
>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>> intentionally designed to explore pushing the limits on performance w/o end
>> users having to solve puzzles, such as learning details about these flags
>> and their possible side effects.
>>
>> So assuming they become either "safe" or internal, maybe you can take
>> advantage of them?
>>
>> I see
>> the available options as:
>>
>> 1) run write-only commands 'optimized', ignoring any querying and such
>> (warn user that he will break it)
>>
>> 2) run write-only without any optimization, rendering them useless
>>
>> 3) detect when querying is set up (ignoring listeners and maybe other
>> stuff that could get broken)
>>
>>
>> Might be useful for making a POC work, but I believe query will be very
>> likely to be often enabled.
>> Having an either / or switch for different features in Infinispan will make
>> it harder to use and understand, so I'd rather see work on the right design
>> as taking temporary shortcuts risks baking into stone features which we
>> later struggle to fix or maintain.
>>
> I vote for this option.
>
> Query, listeners, and other components that need the previous value
> should not just assume that the application knows better, they should
> be able to change how operations works based on their needs. Of
> course, the reverse is also true: if the application uses write-only
> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
> be possible for the user to detect why the previous values are still
> loaded.

If it were just query (static configuration), I would be okay with this 
idea. But as per listeners - besides tainting the design (event source 
should not check if there's a listener) you'd need to check *before* 
(DistributionI, CacheLoaderI) you have to call notify (cmd.perform, 
EWI). So this is a space for race conditions or weird handling (if 
there's a listener when I am about to call notify and my flags are not 
cleared, skip the notification and pretend that this code was invoked 
before the listener was registered...). Or do you have another solution 
in mind (config option to disable listeners && all features using those?).

R.

>
>> 4) remove write-only commands completely (and probably functional
>> listeners as well because these will lose their purpose)
>>
>>
>> +1 to remove "unconditional writes", at least an entry version check should
>> be applied.
>> I believe we had already pointed out this would eventually happen, pretty
>> much for the reasons you're hitting now.
>>
> IMO version checks should be done internally, we shouldn't force the
> users of the functional API to deal with versions themselves because
> we know how hard making write skew checks work is for us :)
>
> And I wouldn't go as far as to remove the functional listeners,
> instead I would change them so that read-write listeners are invoked
> on write-only operations and they force the loading of the previous
> value. I would also add a way for the regular listeners to say whether
> they need the previous value or not.
>
>> Right now I am inclined towards 4). There could be some internal use
>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>> though, but it's asking for trouble.
>>
>>
>> I agree!
>>
>> Thanks
>>
>>
>> WDYT?
>>
>> Radim
>>
> Cheers
> Dan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Wed Jun 28 04:40:01 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Wed, 28 Jun 2017 11:40:01 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
Message-ID: <CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>

On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>> I've said this in a previous thread on this same issue, I will repeat myself
>>> as many times as needed.
>>>
>>> Continuous queries require the previous value itself, not just knowledge of
>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>
>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>> depressed) :)
>>>
>>> I'd remove these commands completely or possibly remove them just from
>>> public API and keep them internal.
>>>
>> +1 to remove the flags from the public API. Most of them are not safe
>> for applications to use, and ignoring them when they can lead to
>> inconsistencies would make them useless.
>>
>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>> know when it is safe to skip the delete statement, and it relies on
>> the application making a (possibly wrong) choice.
>>
>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>> that applications use it right now. If query or listeners need the
>> previous value, then we should load it internally, but hide it from
>> the user.
>>
>> But removing it opens another discussion: should we replace it in the
>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>> should we make it the default and add a method
>> AdvancedCache.forceReturnPreviousValues()?
>
> Please don't derail the thread.
>

I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
breaks the previous value for listeners, even if the QueryInterceptor
removes it from write commands. And it is public (+recommended) API,
in fact most if not all of our performance tests use it.

For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
previous value on backup owners for most write commands
(LoadType.PRIMARY), we'd need to change that as well.

>>
>>>
>>>
>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>
>>>
>>>
>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>
>>> Hi,
>>>
>>> I am working on entry version history (again). In Como we've discussed
>>> that previous values are needed for (continuous) query and reliable
>>> listeners,
>>>
>>>
>>> Index based queries also require the previous value on a write - unless we
>>> can get "strongly typed caches" giving guarantees about the class to
>>> represent the content of a cache to be unique.
>>>
>>> Essentially we only need to know the type of the previous object. It might
>>> be worth having a way to load the type metadata if the previous value only.
>>>
>>> so I wonder what should we do with functional write-only
>>> commands. These are different to commands with flags, because flags
>>> (other than ignore return value) are expected to break something.
>>>
>>>
>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>> search the mailing list.
>>>
>>> Since flags are exposed to the user I would rather they're not allowed to
>>> break things.
>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>> configuration/integrations veto them.
>>>
>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>> intentionally designed to explore pushing the limits on performance w/o end
>>> users having to solve puzzles, such as learning details about these flags
>>> and their possible side effects.
>>>
>>> So assuming they become either "safe" or internal, maybe you can take
>>> advantage of them?
>>>
>>> I see
>>> the available options as:
>>>
>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>> (warn user that he will break it)
>>>
>>> 2) run write-only without any optimization, rendering them useless
>>>
>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>> stuff that could get broken)
>>>
>>>
>>> Might be useful for making a POC work, but I believe query will be very
>>> likely to be often enabled.
>>> Having an either / or switch for different features in Infinispan will make
>>> it harder to use and understand, so I'd rather see work on the right design
>>> as taking temporary shortcuts risks baking into stone features which we
>>> later struggle to fix or maintain.
>>>
>> I vote for this option.
>>
>> Query, listeners, and other components that need the previous value
>> should not just assume that the application knows better, they should
>> be able to change how operations works based on their needs. Of
>> course, the reverse is also true: if the application uses write-only
>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>> be possible for the user to detect why the previous values are still
>> loaded.
>
> If it were just query (static configuration), I would be okay with this
> idea. But as per listeners - besides tainting the design (event source
> should not check if there's a listener) you'd need to check *before*

The source wouldn't check for listeners explicitly, the notifier would
have an isPreviousValueNeeded() method and precompute that before a
listener is added or after a listener is removed. I was am assuming
some listeners will not need the previous value, e.g. the listeners
installed by streams.

> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
> EWI). So this is a space for race conditions or weird handling (if
> there's a listener when I am about to call notify and my flags are not
> cleared, skip the notification and pretend that this code was invoked
> before the listener was registered...). Or do you have another solution
> in mind (config option to disable listeners && all features using those?).
>

I was definitely going for the weird handling...

My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
it's loaded, and check that before invoking a listener that needs the
previous value. It is missing one edge case: if one thread starts a
write operation, then another thread installs a listener that requires
the previous value and iterates over the cache, the second thread may
not see the value written by the first thread.

So now I'm thinking we should retry the write commands when
isPreviousValueNeeded() changes... Not very appealing, but I think the
performance difference is worth it.

> R.
>
>>
>>> 4) remove write-only commands completely (and probably functional
>>> listeners as well because these will lose their purpose)
>>>
>>>
>>> +1 to remove "unconditional writes", at least an entry version check should
>>> be applied.
>>> I believe we had already pointed out this would eventually happen, pretty
>>> much for the reasons you're hitting now.
>>>
>> IMO version checks should be done internally, we shouldn't force the
>> users of the functional API to deal with versions themselves because
>> we know how hard making write skew checks work is for us :)
>>
>> And I wouldn't go as far as to remove the functional listeners,
>> instead I would change them so that read-write listeners are invoked
>> on write-only operations and they force the loading of the previous
>> value. I would also add a way for the regular listeners to say whether
>> they need the previous value or not.
>>
>>> Right now I am inclined towards 4). There could be some internal use
>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>> though, but it's asking for trouble.
>>>
>>>
>>> I agree!
>>>
>>> Thanks
>>>
>>>
>>> WDYT?
>>>
>>> Radim
>>>
>> Cheers
>> Dan

From rvansa at redhat.com  Wed Jun 28 07:17:20 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 28 Jun 2017 13:17:20 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
Message-ID: <3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>

On 06/28/2017 10:40 AM, Dan Berindei wrote:
> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>> as many times as needed.
>>>>
>>>> Continuous queries require the previous value itself, not just knowledge of
>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>
>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>> depressed) :)
>>>>
>>>> I'd remove these commands completely or possibly remove them just from
>>>> public API and keep them internal.
>>>>
>>> +1 to remove the flags from the public API. Most of them are not safe
>>> for applications to use, and ignoring them when they can lead to
>>> inconsistencies would make them useless.
>>>
>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>> know when it is safe to skip the delete statement, and it relies on
>>> the application making a (possibly wrong) choice.
>>>
>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>> that applications use it right now. If query or listeners need the
>>> previous value, then we should load it internally, but hide it from
>>> the user.
>>>
>>> But removing it opens another discussion: should we replace it in the
>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>> should we make it the default and add a method
>>> AdvancedCache.forceReturnPreviousValues()?
>> Please don't derail the thread.
>>
> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
> breaks the previous value for listeners, even if the QueryInterceptor
> removes it from write commands. And it is public (+recommended) API,
> in fact most if not all of our performance tests use it.

That's just a flawed implementation. IPV is documented to be a 'safe' 
flag that should affect mostly primary -> origin replication, all the 
other is implementation. And we can fix that. Users should *not* expect 
that it e.g. skips loading from a cache store. We have already removed 
the modes that would be broken-by-design.

On the other hand, write-only commands are not about *returning* the 
value but about (not) *reading* it, therefore (in my eyes) user could 
make that assumption and would like to enforce it this way. Even some 
docs explaining PersistenceMode.SKIP suggest that.

I don't want to talk about flags, because I see all flags but IPV as 
'effectively internal'. Let's discuss it more high-level. Some API 
exposes non-reading operation - we can see that under some circumstances 
this is not possible so we have options to 1) break stuff 2) break API 
assumptions 3) sometimes break API assumptions 4) remove such API (to 
not allow the user to make such assumptions). There's also an option 5) 
to fail the operation if the API assumption would be broken. Though, I 
don't fancy getting exception from a WriteOnlyMap.eval just because 
someone has registered a listener.

>
> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
> previous value on backup owners for most write commands
> (LoadType.PRIMARY), we'd need to change that as well.

Yes, all commands will have to load current value on all owners.

>
>>>
>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>
>>>>
>>>>
>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I am working on entry version history (again). In Como we've discussed
>>>> that previous values are needed for (continuous) query and reliable
>>>> listeners,
>>>>
>>>>
>>>> Index based queries also require the previous value on a write - unless we
>>>> can get "strongly typed caches" giving guarantees about the class to
>>>> represent the content of a cache to be unique.
>>>>
>>>> Essentially we only need to know the type of the previous object. It might
>>>> be worth having a way to load the type metadata if the previous value only.
>>>>
>>>> so I wonder what should we do with functional write-only
>>>> commands. These are different to commands with flags, because flags
>>>> (other than ignore return value) are expected to break something.
>>>>
>>>>
>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>> search the mailing list.
>>>>
>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>> break things.
>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>> configuration/integrations veto them.
>>>>
>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>> users having to solve puzzles, such as learning details about these flags
>>>> and their possible side effects.
>>>>
>>>> So assuming they become either "safe" or internal, maybe you can take
>>>> advantage of them?
>>>>
>>>> I see
>>>> the available options as:
>>>>
>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>> (warn user that he will break it)
>>>>
>>>> 2) run write-only without any optimization, rendering them useless
>>>>
>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>> stuff that could get broken)
>>>>
>>>>
>>>> Might be useful for making a POC work, but I believe query will be very
>>>> likely to be often enabled.
>>>> Having an either / or switch for different features in Infinispan will make
>>>> it harder to use and understand, so I'd rather see work on the right design
>>>> as taking temporary shortcuts risks baking into stone features which we
>>>> later struggle to fix or maintain.
>>>>
>>> I vote for this option.
>>>
>>> Query, listeners, and other components that need the previous value
>>> should not just assume that the application knows better, they should
>>> be able to change how operations works based on their needs. Of
>>> course, the reverse is also true: if the application uses write-only
>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>> be possible for the user to detect why the previous values are still
>>> loaded.
>> If it were just query (static configuration), I would be okay with this
>> idea. But as per listeners - besides tainting the design (event source
>> should not check if there's a listener) you'd need to check *before*
> The source wouldn't check for listeners explicitly, the notifier would
> have an isPreviousValueNeeded() method and precompute that before a
> listener is added or after a listener is removed. I was am assuming
> some listeners will not need the previous value, e.g. the listeners
> installed by streams.

You can cover your warts with a make-up but you'll still have warts :)
>
>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>> EWI). So this is a space for race conditions or weird handling (if
>> there's a listener when I am about to call notify and my flags are not
>> cleared, skip the notification and pretend that this code was invoked
>> before the listener was registered...). Or do you have another solution
>> in mind (config option to disable listeners && all features using those?).
>>
> I was definitely going for the weird handling...
>
> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
> it's loaded, and check that before invoking a listener that needs the
> previous value. It is missing one edge case: if one thread starts a
> write operation, then another thread installs a listener that requires
> the previous value and iterates over the cache, the second thread may
> not see the value written by the first thread.

If the operations overlap, you could pretend that the write has finished 
before the listener was invoked and simply not notify the listener. If I 
am missing it please write it down in code. But handling this in any way 
is still clumsy.
> So now I'm thinking we should retry the write commands when
> isPreviousValueNeeded() changes... Not very appealing, but I think the
> performance difference is worth it.
>
>> R.
>>
>>>> 4) remove write-only commands completely (and probably functional
>>>> listeners as well because these will lose their purpose)
>>>>
>>>>
>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>> be applied.
>>>> I believe we had already pointed out this would eventually happen, pretty
>>>> much for the reasons you're hitting now.
>>>>
>>> IMO version checks should be done internally, we shouldn't force the
>>> users of the functional API to deal with versions themselves because
>>> we know how hard making write skew checks work is for us :)
>>>
>>> And I wouldn't go as far as to remove the functional listeners,
>>> instead I would change them so that read-write listeners are invoked
>>> on write-only operations and they force the loading of the previous
>>> value. I would also add a way for the regular listeners to say whether
>>> they need the previous value or not.
>>>
>>>> Right now I am inclined towards 4). There could be some internal use
>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>> though, but it's asking for trouble.
>>>>
>>>>
>>>> I agree!
>>>>
>>>> Thanks
>>>>
>>>>
>>>> WDYT?
>>>>
>>>> Radim
>>>>
>>> Cheers
>>> Dan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Wed Jun 28 10:20:53 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Wed, 28 Jun 2017 17:20:53 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
Message-ID: <CA+nfvwRR3toRrRQiXKYNZGjkeyx1gRaje1Q7yuyVGQaPD7Jk9Q@mail.gmail.com>

On Wed, Jun 28, 2017 at 2:17 PM, Radim Vansa <rvansa at redhat.com> wrote:
> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>> as many times as needed.
>>>>>
>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>
>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>> depressed) :)
>>>>>
>>>>> I'd remove these commands completely or possibly remove them just from
>>>>> public API and keep them internal.
>>>>>
>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>> for applications to use, and ignoring them when they can lead to
>>>> inconsistencies would make them useless.
>>>>
>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>> know when it is safe to skip the delete statement, and it relies on
>>>> the application making a (possibly wrong) choice.
>>>>
>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>> that applications use it right now. If query or listeners need the
>>>> previous value, then we should load it internally, but hide it from
>>>> the user.
>>>>
>>>> But removing it opens another discussion: should we replace it in the
>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>> should we make it the default and add a method
>>>> AdvancedCache.forceReturnPreviousValues()?
>>> Please don't derail the thread.
>>>
>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>> breaks the previous value for listeners, even if the QueryInterceptor
>> removes it from write commands. And it is public (+recommended) API,
>> in fact most if not all of our performance tests use it.
>
> That's just a flawed implementation. IPV is documented to be a 'safe'
> flag that should affect mostly primary -> origin replication, all the
> other is implementation. And we can fix that. Users should *not* expect
> that it e.g. skips loading from a cache store. We have already removed
> the modes that would be broken-by-design.
>

I think you're confusing IGNORE_RETURN_VALUES with SKIP_REMOTE_LOOKUP
here. The IVR javadoc doesn't say anything about remote lookups, only
SRL does.

And I agree that the current status is far from ideal, but there is
one more valid alternative: we can decide that the previous value is
only reliable in clustered listeners, and local listeners don't always
have it. Document that, make sure continuous query uses clustered
listeners, and we're done :)


> On the other hand, write-only commands are not about *returning* the
> value but about (not) *reading* it, therefore (in my eyes) user could
> make that assumption and would like to enforce it this way. Even some
> docs explaining PersistenceMode.SKIP suggest that.
>

To me the purpose the same, there is no difference between returning
the previous value to the application or providing the previous value
via EntryView. Applying this logic to the JCache API, it would mean
put() should never read the previous value, because some users could
assume that only getAndPut() reads it.

In the old times we didn't have IGNORE_RETURN_VALUES, only
SKIP_REMOTE_LOOKUP+SKIP_CACHE_LOAD, and they would sometimes be
ignored (e.g. if the write was conditional). I think that's what
Galder had in mind when he wrote the PersistenceMode api note, not the
current behaviour of SKIP_CACHE_LOAD. I'll let Galder clarify this
himself, but I'll be very disappointed if he says he designed the
write-only operations so that they'll never work with query.


> I don't want to talk about flags, because I see all flags but IPV as
> 'effectively internal'. Let's discuss it more high-level. Some API
> exposes non-reading operation - we can see that under some circumstances
> this is not possible so we have options to 1) break stuff 2) break API
> assumptions 3) sometimes break API assumptions 4) remove such API (to
> not allow the user to make such assumptions). There's also an option 5)
> to fail the operation if the API assumption would be broken. Though, I
> don't fancy getting exception from a WriteOnlyMap.eval just because
> someone has registered a listener.
>

I disagree with the premise: there's no good reason for the user to
assume that write-only commands are *guaranteed* to never load the
previous value from a store. We just need to add a clarification to
the write-only operations' javadoc, no need to break anything.


>>
>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>> previous value on backup owners for most write commands
>> (LoadType.PRIMARY), we'd need to change that as well.
>
> Yes, all commands will have to load current value on all owners.
>
>>
>>>>
>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I am working on entry version history (again). In Como we've discussed
>>>>> that previous values are needed for (continuous) query and reliable
>>>>> listeners,
>>>>>
>>>>>
>>>>> Index based queries also require the previous value on a write - unless we
>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>> represent the content of a cache to be unique.
>>>>>
>>>>> Essentially we only need to know the type of the previous object. It might
>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>
>>>>> so I wonder what should we do with functional write-only
>>>>> commands. These are different to commands with flags, because flags
>>>>> (other than ignore return value) are expected to break something.
>>>>>
>>>>>
>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>> search the mailing list.
>>>>>
>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>> break things.
>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>> configuration/integrations veto them.
>>>>>
>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>> users having to solve puzzles, such as learning details about these flags
>>>>> and their possible side effects.
>>>>>
>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>> advantage of them?
>>>>>
>>>>> I see
>>>>> the available options as:
>>>>>
>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>> (warn user that he will break it)
>>>>>
>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>
>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>> stuff that could get broken)
>>>>>
>>>>>
>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>> likely to be often enabled.
>>>>> Having an either / or switch for different features in Infinispan will make
>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>> later struggle to fix or maintain.
>>>>>
>>>> I vote for this option.
>>>>
>>>> Query, listeners, and other components that need the previous value
>>>> should not just assume that the application knows better, they should
>>>> be able to change how operations works based on their needs. Of
>>>> course, the reverse is also true: if the application uses write-only
>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>> be possible for the user to detect why the previous values are still
>>>> loaded.
>>> If it were just query (static configuration), I would be okay with this
>>> idea. But as per listeners - besides tainting the design (event source
>>> should not check if there's a listener) you'd need to check *before*
>> The source wouldn't check for listeners explicitly, the notifier would
>> have an isPreviousValueNeeded() method and precompute that before a
>> listener is added or after a listener is removed. I was am assuming
>> some listeners will not need the previous value, e.g. the listeners
>> installed by streams.
>
> You can cover your warts with a make-up but you'll still have warts :)

Cutting them off doesn't necessarily work, either :)

>>
>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>> EWI). So this is a space for race conditions or weird handling (if
>>> there's a listener when I am about to call notify and my flags are not
>>> cleared, skip the notification and pretend that this code was invoked
>>> before the listener was registered...). Or do you have another solution
>>> in mind (config option to disable listeners && all features using those?).
>>>
>> I was definitely going for the weird handling...
>>
>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>> it's loaded, and check that before invoking a listener that needs the
>> previous value. It is missing one edge case: if one thread starts a
>> write operation, then another thread installs a listener that requires
>> the previous value and iterates over the cache, the second thread may
>> not see the value written by the first thread.
>
> If the operations overlap, you could pretend that the write has finished
> before the listener was invoked and simply not notify the listener. If I
> am missing it please write it down in code. But handling this in any way
> is still clumsy.

I hope pseudo-code is fine...

1. cache.put(k, v1) starts, doesn't load the previous value v0 in the context
2. cache.addListener(l) runs, doesn't block
3. cache.entrySet().forEach() runs, finds k->v0
4. cache.put(k, v1) commits k->v1, should notify the listener but
doesn't have the previous value
5. cache.put(k, v0) returns, but the code that installed the listener
thinks the value of k is still v0


>> So now I'm thinking we should retry the write commands when
>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>> performance difference is worth it.
>>
>>> R.
>>>
>>>>> 4) remove write-only commands completely (and probably functional
>>>>> listeners as well because these will lose their purpose)
>>>>>
>>>>>
>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>> be applied.
>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>> much for the reasons you're hitting now.
>>>>>
>>>> IMO version checks should be done internally, we shouldn't force the
>>>> users of the functional API to deal with versions themselves because
>>>> we know how hard making write skew checks work is for us :)
>>>>
>>>> And I wouldn't go as far as to remove the functional listeners,
>>>> instead I would change them so that read-write listeners are invoked
>>>> on write-only operations and they force the loading of the previous
>>>> value. I would also add a way for the regular listeners to say whether
>>>> they need the previous value or not.
>>>>
>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>> though, but it's asking for trouble.
>>>>>
>>>>>
>>>>> I agree!
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> WDYT?
>>>>>
>>>>> Radim
>>>>>
>>>> Cheers
>>>> Dan
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From rvansa at redhat.com  Wed Jun 28 10:25:39 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 28 Jun 2017 16:25:39 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
Message-ID: <bf9007b8-ee65-3645-c210-219c0703c3ab@redhat.com>

On 06/28/2017 01:17 PM, Radim Vansa wrote:
> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>> as many times as needed.
>>>>>
>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>
>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>> depressed) :)
>>>>>
>>>>> I'd remove these commands completely or possibly remove them just from
>>>>> public API and keep them internal.
>>>>>
>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>> for applications to use, and ignoring them when they can lead to
>>>> inconsistencies would make them useless.
>>>>
>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>> know when it is safe to skip the delete statement, and it relies on
>>>> the application making a (possibly wrong) choice.
>>>>
>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>> that applications use it right now. If query or listeners need the
>>>> previous value, then we should load it internally, but hide it from
>>>> the user.
>>>>
>>>> But removing it opens another discussion: should we replace it in the
>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>> should we make it the default and add a method
>>>> AdvancedCache.forceReturnPreviousValues()?
>>> Please don't derail the thread.
>>>
>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>> breaks the previous value for listeners, even if the QueryInterceptor
>> removes it from write commands. And it is public (+recommended) API,
>> in fact most if not all of our performance tests use it.
> That's just a flawed implementation. IPV is documented to be a 'safe'
> flag that should affect mostly primary -> origin replication, all the
> other is implementation. And we can fix that. Users should *not* expect
> that it e.g. skips loading from a cache store. We have already removed
> the modes that would be broken-by-design.
>
> On the other hand, write-only commands are not about *returning* the
> value but about (not) *reading* it, therefore (in my eyes) user could
> make that assumption and would like to enforce it this way. Even some
> docs explaining PersistenceMode.SKIP suggest that.
>
> I don't want to talk about flags, because I see all flags but IPV as
> 'effectively internal'. Let's discuss it more high-level. Some API
> exposes non-reading operation - we can see that under some circumstances
> this is not possible so we have options to 1) break stuff 2) break API
> assumptions 3) sometimes break API assumptions 4) remove such API (to
> not allow the user to make such assumptions). There's also an option 5)
> to fail the operation if the API assumption would be broken. Though, I
> don't fancy getting exception from a WriteOnlyMap.eval just because
> someone has registered a listener.
>
>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>> previous value on backup owners for most write commands
>> (LoadType.PRIMARY), we'd need to change that as well.
> Yes, all commands will have to load current value on all owners.

Btw., this has definitely some impact on shared cachestores. It's not 
just that you have to load the value from cachestore - you also need to 
store the previous value along with the current value (in metadata), 
because if the primary owner crashes and the backups don't have it in 
memory, these need to check the persistence to see if the command was 
already executed.

This means that the metadata could grow up to several times the size of 
the value itself (if there's high concurrency). And even worse with 
async caches, as these don't know when the command was actually finished 
and therefore rely on time-based expiration of the previous values :-/

Non-shared cachestores don't need to store previous values as long if we 
can guarantee (flag) the entry as not eligible for eviction.

>
>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I am working on entry version history (again). In Como we've discussed
>>>>> that previous values are needed for (continuous) query and reliable
>>>>> listeners,
>>>>>
>>>>>
>>>>> Index based queries also require the previous value on a write - unless we
>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>> represent the content of a cache to be unique.
>>>>>
>>>>> Essentially we only need to know the type of the previous object. It might
>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>
>>>>> so I wonder what should we do with functional write-only
>>>>> commands. These are different to commands with flags, because flags
>>>>> (other than ignore return value) are expected to break something.
>>>>>
>>>>>
>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>> search the mailing list.
>>>>>
>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>> break things.
>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>> configuration/integrations veto them.
>>>>>
>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>> users having to solve puzzles, such as learning details about these flags
>>>>> and their possible side effects.
>>>>>
>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>> advantage of them?
>>>>>
>>>>> I see
>>>>> the available options as:
>>>>>
>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>> (warn user that he will break it)
>>>>>
>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>
>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>> stuff that could get broken)
>>>>>
>>>>>
>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>> likely to be often enabled.
>>>>> Having an either / or switch for different features in Infinispan will make
>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>> later struggle to fix or maintain.
>>>>>
>>>> I vote for this option.
>>>>
>>>> Query, listeners, and other components that need the previous value
>>>> should not just assume that the application knows better, they should
>>>> be able to change how operations works based on their needs. Of
>>>> course, the reverse is also true: if the application uses write-only
>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>> be possible for the user to detect why the previous values are still
>>>> loaded.
>>> If it were just query (static configuration), I would be okay with this
>>> idea. But as per listeners - besides tainting the design (event source
>>> should not check if there's a listener) you'd need to check *before*
>> The source wouldn't check for listeners explicitly, the notifier would
>> have an isPreviousValueNeeded() method and precompute that before a
>> listener is added or after a listener is removed. I was am assuming
>> some listeners will not need the previous value, e.g. the listeners
>> installed by streams.
> You can cover your warts with a make-up but you'll still have warts :)
>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>> EWI). So this is a space for race conditions or weird handling (if
>>> there's a listener when I am about to call notify and my flags are not
>>> cleared, skip the notification and pretend that this code was invoked
>>> before the listener was registered...). Or do you have another solution
>>> in mind (config option to disable listeners && all features using those?).
>>>
>> I was definitely going for the weird handling...
>>
>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>> it's loaded, and check that before invoking a listener that needs the
>> previous value. It is missing one edge case: if one thread starts a
>> write operation, then another thread installs a listener that requires
>> the previous value and iterates over the cache, the second thread may
>> not see the value written by the first thread.
> If the operations overlap, you could pretend that the write has finished
> before the listener was invoked and simply not notify the listener. If I
> am missing it please write it down in code. But handling this in any way
> is still clumsy.
>> So now I'm thinking we should retry the write commands when
>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>> performance difference is worth it.
>>
>>> R.
>>>
>>>>> 4) remove write-only commands completely (and probably functional
>>>>> listeners as well because these will lose their purpose)
>>>>>
>>>>>
>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>> be applied.
>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>> much for the reasons you're hitting now.
>>>>>
>>>> IMO version checks should be done internally, we shouldn't force the
>>>> users of the functional API to deal with versions themselves because
>>>> we know how hard making write skew checks work is for us :)
>>>>
>>>> And I wouldn't go as far as to remove the functional listeners,
>>>> instead I would change them so that read-write listeners are invoked
>>>> on write-only operations and they force the loading of the previous
>>>> value. I would also add a way for the regular listeners to say whether
>>>> they need the previous value or not.
>>>>
>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>> though, but it's asking for trouble.
>>>>>
>>>>>
>>>>> I agree!
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> WDYT?
>>>>>
>>>>> Radim
>>>>>
>>>> Cheers
>>>> Dan
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Thu Jun 29 04:23:48 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 29 Jun 2017 11:23:48 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <bf9007b8-ee65-3645-c210-219c0703c3ab@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
	<bf9007b8-ee65-3645-c210-219c0703c3ab@redhat.com>
Message-ID: <CA+nfvwSd2=zGcHCPfvAfMtDARJXpU5dyMjFHERO3nxbHXn+Yyw@mail.gmail.com>

On Wed, Jun 28, 2017 at 5:25 PM, Radim Vansa <rvansa at redhat.com> wrote:
> On 06/28/2017 01:17 PM, Radim Vansa wrote:
>> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>>> as many times as needed.
>>>>>>
>>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>>
>>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>>> depressed) :)
>>>>>>
>>>>>> I'd remove these commands completely or possibly remove them just from
>>>>>> public API and keep them internal.
>>>>>>
>>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>>> for applications to use, and ignoring them when they can lead to
>>>>> inconsistencies would make them useless.
>>>>>
>>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>>> know when it is safe to skip the delete statement, and it relies on
>>>>> the application making a (possibly wrong) choice.
>>>>>
>>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>>> that applications use it right now. If query or listeners need the
>>>>> previous value, then we should load it internally, but hide it from
>>>>> the user.
>>>>>
>>>>> But removing it opens another discussion: should we replace it in the
>>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>>> should we make it the default and add a method
>>>>> AdvancedCache.forceReturnPreviousValues()?
>>>> Please don't derail the thread.
>>>>
>>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>>> breaks the previous value for listeners, even if the QueryInterceptor
>>> removes it from write commands. And it is public (+recommended) API,
>>> in fact most if not all of our performance tests use it.
>> That's just a flawed implementation. IPV is documented to be a 'safe'
>> flag that should affect mostly primary -> origin replication, all the
>> other is implementation. And we can fix that. Users should *not* expect
>> that it e.g. skips loading from a cache store. We have already removed
>> the modes that would be broken-by-design.
>>
>> On the other hand, write-only commands are not about *returning* the
>> value but about (not) *reading* it, therefore (in my eyes) user could
>> make that assumption and would like to enforce it this way. Even some
>> docs explaining PersistenceMode.SKIP suggest that.
>>
>> I don't want to talk about flags, because I see all flags but IPV as
>> 'effectively internal'. Let's discuss it more high-level. Some API
>> exposes non-reading operation - we can see that under some circumstances
>> this is not possible so we have options to 1) break stuff 2) break API
>> assumptions 3) sometimes break API assumptions 4) remove such API (to
>> not allow the user to make such assumptions). There's also an option 5)
>> to fail the operation if the API assumption would be broken. Though, I
>> don't fancy getting exception from a WriteOnlyMap.eval just because
>> someone has registered a listener.
>>
>>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>>> previous value on backup owners for most write commands
>>> (LoadType.PRIMARY), we'd need to change that as well.
>> Yes, all commands will have to load current value on all owners.
>
> Btw., this has definitely some impact on shared cachestores. It's not
> just that you have to load the value from cachestore - you also need to
> store the previous value along with the current value (in metadata),
> because if the primary owner crashes and the backups don't have it in
> memory, these need to check the persistence to see if the command was
> already executed.
>

I don't think this would work with the triangle algorithm. The primary
owner updates the shared store without waiting for acks from the
backups, so a backup trying to load the previous value is very likely
to find the updated value instead.

I'll hazard a guess that none of the operations using LoadType.OWNER
currently works with triangle + shared store.

> This means that the metadata could grow up to several times the size of
> the value itself (if there's high concurrency). And even worse with
> async caches, as these don't know when the command was actually finished
> and therefore rely on time-based expiration of the previous values :-/
>
> Non-shared cachestores don't need to store previous values as long if we
> can guarantee (flag) the entry as not eligible for eviction.
>
>>
>>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am working on entry version history (again). In Como we've discussed
>>>>>> that previous values are needed for (continuous) query and reliable
>>>>>> listeners,
>>>>>>
>>>>>>
>>>>>> Index based queries also require the previous value on a write - unless we
>>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>>> represent the content of a cache to be unique.
>>>>>>
>>>>>> Essentially we only need to know the type of the previous object. It might
>>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>>
>>>>>> so I wonder what should we do with functional write-only
>>>>>> commands. These are different to commands with flags, because flags
>>>>>> (other than ignore return value) are expected to break something.
>>>>>>
>>>>>>
>>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>>> search the mailing list.
>>>>>>
>>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>>> break things.
>>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>>> configuration/integrations veto them.
>>>>>>
>>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>>> users having to solve puzzles, such as learning details about these flags
>>>>>> and their possible side effects.
>>>>>>
>>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>>> advantage of them?
>>>>>>
>>>>>> I see
>>>>>> the available options as:
>>>>>>
>>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>>> (warn user that he will break it)
>>>>>>
>>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>>
>>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>>> stuff that could get broken)
>>>>>>
>>>>>>
>>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>>> likely to be often enabled.
>>>>>> Having an either / or switch for different features in Infinispan will make
>>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>>> later struggle to fix or maintain.
>>>>>>
>>>>> I vote for this option.
>>>>>
>>>>> Query, listeners, and other components that need the previous value
>>>>> should not just assume that the application knows better, they should
>>>>> be able to change how operations works based on their needs. Of
>>>>> course, the reverse is also true: if the application uses write-only
>>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>>> be possible for the user to detect why the previous values are still
>>>>> loaded.
>>>> If it were just query (static configuration), I would be okay with this
>>>> idea. But as per listeners - besides tainting the design (event source
>>>> should not check if there's a listener) you'd need to check *before*
>>> The source wouldn't check for listeners explicitly, the notifier would
>>> have an isPreviousValueNeeded() method and precompute that before a
>>> listener is added or after a listener is removed. I was am assuming
>>> some listeners will not need the previous value, e.g. the listeners
>>> installed by streams.
>> You can cover your warts with a make-up but you'll still have warts :)
>>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>>> EWI). So this is a space for race conditions or weird handling (if
>>>> there's a listener when I am about to call notify and my flags are not
>>>> cleared, skip the notification and pretend that this code was invoked
>>>> before the listener was registered...). Or do you have another solution
>>>> in mind (config option to disable listeners && all features using those?).
>>>>
>>> I was definitely going for the weird handling...
>>>
>>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>>> it's loaded, and check that before invoking a listener that needs the
>>> previous value. It is missing one edge case: if one thread starts a
>>> write operation, then another thread installs a listener that requires
>>> the previous value and iterates over the cache, the second thread may
>>> not see the value written by the first thread.
>> If the operations overlap, you could pretend that the write has finished
>> before the listener was invoked and simply not notify the listener. If I
>> am missing it please write it down in code. But handling this in any way
>> is still clumsy.
>>> So now I'm thinking we should retry the write commands when
>>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>>> performance difference is worth it.
>>>
>>>> R.
>>>>
>>>>>> 4) remove write-only commands completely (and probably functional
>>>>>> listeners as well because these will lose their purpose)
>>>>>>
>>>>>>
>>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>>> be applied.
>>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>>> much for the reasons you're hitting now.
>>>>>>
>>>>> IMO version checks should be done internally, we shouldn't force the
>>>>> users of the functional API to deal with versions themselves because
>>>>> we know how hard making write skew checks work is for us :)
>>>>>
>>>>> And I wouldn't go as far as to remove the functional listeners,
>>>>> instead I would change them so that read-write listeners are invoked
>>>>> on write-only operations and they force the loading of the previous
>>>>> value. I would also add a way for the regular listeners to say whether
>>>>> they need the previous value or not.
>>>>>
>>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>>> though, but it's asking for trouble.
>>>>>>
>>>>>>
>>>>>> I agree!
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>> Radim
>>>>>>
>>>>> Cheers
>>>>> Dan
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From rvansa at redhat.com  Thu Jun 29 04:53:10 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 29 Jun 2017 10:53:10 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CA+nfvwRR3toRrRQiXKYNZGjkeyx1gRaje1Q7yuyVGQaPD7Jk9Q@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
	<CA+nfvwRR3toRrRQiXKYNZGjkeyx1gRaje1Q7yuyVGQaPD7Jk9Q@mail.gmail.com>
Message-ID: <caea5f08-0b9d-25bd-4263-ec4ab2bc513b@redhat.com>

On 06/28/2017 04:20 PM, Dan Berindei wrote:
> On Wed, Jun 28, 2017 at 2:17 PM, Radim Vansa <rvansa at redhat.com> wrote:
>> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>>> as many times as needed.
>>>>>>
>>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>>
>>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>>> depressed) :)
>>>>>>
>>>>>> I'd remove these commands completely or possibly remove them just from
>>>>>> public API and keep them internal.
>>>>>>
>>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>>> for applications to use, and ignoring them when they can lead to
>>>>> inconsistencies would make them useless.
>>>>>
>>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>>> know when it is safe to skip the delete statement, and it relies on
>>>>> the application making a (possibly wrong) choice.
>>>>>
>>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>>> that applications use it right now. If query or listeners need the
>>>>> previous value, then we should load it internally, but hide it from
>>>>> the user.
>>>>>
>>>>> But removing it opens another discussion: should we replace it in the
>>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>>> should we make it the default and add a method
>>>>> AdvancedCache.forceReturnPreviousValues()?
>>>> Please don't derail the thread.
>>>>
>>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>>> breaks the previous value for listeners, even if the QueryInterceptor
>>> removes it from write commands. And it is public (+recommended) API,
>>> in fact most if not all of our performance tests use it.
>> That's just a flawed implementation. IPV is documented to be a 'safe'
>> flag that should affect mostly primary -> origin replication, all the
>> other is implementation. And we can fix that. Users should *not* expect
>> that it e.g. skips loading from a cache store. We have already removed
>> the modes that would be broken-by-design.
>>
> I think you're confusing IGNORE_RETURN_VALUES with SKIP_REMOTE_LOOKUP
> here. The IVR javadoc doesn't say anything about remote lookups, only
> SRL does.

No, I am not; While IRV does not mention the replication, it's said to 
be 'safe'. So omitting the primary -> origin replication is basically 
all it can do when listeners are in place. You're right that I have 
missed the second part in SRL talking about put()s; I took it as a flag 
prohibiting any remote lookup (as the RPC operation in its whole) any 
time the remote value is needed. Yes, the second part seems equal to my 
understanding of IRV.

>
> And I agree that the current status is far from ideal, but there is
> one more valid alternative: we can decide that the previous value is
> only reliable in clustered listeners, and local listeners don't always
> have it. Document that, make sure continuous query uses clustered
> listeners, and we're done :)

Unreliable return values are worse than none; I would rather remove them 
if we can't guarantee that these are right. Though, clustered listeners 
are based on regular listeners, so you'd need some means to make them 
reliable.
>> On the other hand, write-only commands are not about *returning* the
>> value but about (not) *reading* it, therefore (in my eyes) user could
>> make that assumption and would like to enforce it this way. Even some
>> docs explaining PersistenceMode.SKIP suggest that.
>>
> To me the purpose the same, there is no difference between returning
> the previous value to the application or providing the previous value
> via EntryView.

There is a difference between what's provided locally and what's send 
over the network.

> Applying this logic to the JCache API, it would mean
> put() should never read the previous value, because some users could
> assume that only getAndPut() reads it.

OK, this is a valid point.

>
> In the old times we didn't have IGNORE_RETURN_VALUES, only
> SKIP_REMOTE_LOOKUP+SKIP_CACHE_LOAD, and they would sometimes be
> ignored (e.g. if the write was conditional). I think that's what
> Galder had in mind when he wrote the PersistenceMode api note, not the
> current behaviour of SKIP_CACHE_LOAD. I'll let Galder clarify this
> himself, but I'll be very disappointed if he says he designed the
> write-only operations so that they'll never work with query.
>
>
>> I don't want to talk about flags, because I see all flags but IPV as
>> 'effectively internal'. Let's discuss it more high-level. Some API
>> exposes non-reading operation - we can see that under some circumstances
>> this is not possible so we have options to 1) break stuff 2) break API
>> assumptions 3) sometimes break API assumptions 4) remove such API (to
>> not allow the user to make such assumptions). There's also an option 5)
>> to fail the operation if the API assumption would be broken. Though, I
>> don't fancy getting exception from a WriteOnlyMap.eval just because
>> someone has registered a listener.
>>
> I disagree with the premise: there's no good reason for the user to
> assume that write-only commands are *guaranteed* to never load the
> previous value from a store. We just need to add a clarification to
> the write-only operations' javadoc, no need to break anything.

OK then, though it diminishes the value of write-only commands a lot.

>
>
>>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>>> previous value on backup owners for most write commands
>>> (LoadType.PRIMARY), we'd need to change that as well.
>> Yes, all commands will have to load current value on all owners.
>>
>>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am working on entry version history (again). In Como we've discussed
>>>>>> that previous values are needed for (continuous) query and reliable
>>>>>> listeners,
>>>>>>
>>>>>>
>>>>>> Index based queries also require the previous value on a write - unless we
>>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>>> represent the content of a cache to be unique.
>>>>>>
>>>>>> Essentially we only need to know the type of the previous object. It might
>>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>>
>>>>>> so I wonder what should we do with functional write-only
>>>>>> commands. These are different to commands with flags, because flags
>>>>>> (other than ignore return value) are expected to break something.
>>>>>>
>>>>>>
>>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>>> search the mailing list.
>>>>>>
>>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>>> break things.
>>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>>> configuration/integrations veto them.
>>>>>>
>>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>>> users having to solve puzzles, such as learning details about these flags
>>>>>> and their possible side effects.
>>>>>>
>>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>>> advantage of them?
>>>>>>
>>>>>> I see
>>>>>> the available options as:
>>>>>>
>>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>>> (warn user that he will break it)
>>>>>>
>>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>>
>>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>>> stuff that could get broken)
>>>>>>
>>>>>>
>>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>>> likely to be often enabled.
>>>>>> Having an either / or switch for different features in Infinispan will make
>>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>>> later struggle to fix or maintain.
>>>>>>
>>>>> I vote for this option.
>>>>>
>>>>> Query, listeners, and other components that need the previous value
>>>>> should not just assume that the application knows better, they should
>>>>> be able to change how operations works based on their needs. Of
>>>>> course, the reverse is also true: if the application uses write-only
>>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>>> be possible for the user to detect why the previous values are still
>>>>> loaded.
>>>> If it were just query (static configuration), I would be okay with this
>>>> idea. But as per listeners - besides tainting the design (event source
>>>> should not check if there's a listener) you'd need to check *before*
>>> The source wouldn't check for listeners explicitly, the notifier would
>>> have an isPreviousValueNeeded() method and precompute that before a
>>> listener is added or after a listener is removed. I was am assuming
>>> some listeners will not need the previous value, e.g. the listeners
>>> installed by streams.
>> You can cover your warts with a make-up but you'll still have warts :)
> Cutting them off doesn't necessarily work, either :)

Yep, some people tend to fix w/ hacks instead of designing :)

>
>>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>>> EWI). So this is a space for race conditions or weird handling (if
>>>> there's a listener when I am about to call notify and my flags are not
>>>> cleared, skip the notification and pretend that this code was invoked
>>>> before the listener was registered...). Or do you have another solution
>>>> in mind (config option to disable listeners && all features using those?).
>>>>
>>> I was definitely going for the weird handling...
>>>
>>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>>> it's loaded, and check that before invoking a listener that needs the
>>> previous value. It is missing one edge case: if one thread starts a
>>> write operation, then another thread installs a listener that requires
>>> the previous value and iterates over the cache, the second thread may
>>> not see the value written by the first thread.
>> If the operations overlap, you could pretend that the write has finished
>> before the listener was invoked and simply not notify the listener. If I
>> am missing it please write it down in code. But handling this in any way
>> is still clumsy.
> I hope pseudo-code is fine...
>
> 1. cache.put(k, v1) starts, doesn't load the previous value v0 in the context
> 2. cache.addListener(l) runs, doesn't block
> 3. cache.entrySet().forEach() runs, finds k->v0
> 4. cache.put(k, v1) commits k->v1, should notify the listener but
> doesn't have the previous value
> 5. cache.put(k, v0) returns, but the code that installed the listener
> thinks the value of k is still v0

Oh OK, I should have drawn that myself when considering the scenario. 
You're right, here we'll have to retry.

All in all, I think this discussion is done. We'll tell users to stick 
their flags where the sun doesn't shine and remove any inconvenient 
ones. Should we issue a warning any time we're removing the flag?

Radim

>
>
>>> So now I'm thinking we should retry the write commands when
>>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>>> performance difference is worth it.
>>>
>>>> R.
>>>>
>>>>>> 4) remove write-only commands completely (and probably functional
>>>>>> listeners as well because these will lose their purpose)
>>>>>>
>>>>>>
>>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>>> be applied.
>>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>>> much for the reasons you're hitting now.
>>>>>>
>>>>> IMO version checks should be done internally, we shouldn't force the
>>>>> users of the functional API to deal with versions themselves because
>>>>> we know how hard making write skew checks work is for us :)
>>>>>
>>>>> And I wouldn't go as far as to remove the functional listeners,
>>>>> instead I would change them so that read-write listeners are invoked
>>>>> on write-only operations and they force the loading of the previous
>>>>> value. I would also add a way for the regular listeners to say whether
>>>>> they need the previous value or not.
>>>>>
>>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>>> though, but it's asking for trouble.
>>>>>>
>>>>>>
>>>>>> I agree!
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>> Radim
>>>>>>
>>>>> Cheers
>>>>> Dan
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From rvansa at redhat.com  Thu Jun 29 05:00:18 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 29 Jun 2017 11:00:18 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CA+nfvwSd2=zGcHCPfvAfMtDARJXpU5dyMjFHERO3nxbHXn+Yyw@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
	<bf9007b8-ee65-3645-c210-219c0703c3ab@redhat.com>
	<CA+nfvwSd2=zGcHCPfvAfMtDARJXpU5dyMjFHERO3nxbHXn+Yyw@mail.gmail.com>
Message-ID: <54cf992e-dd12-cc70-c37a-a2049cc93e64@redhat.com>

On 06/29/2017 10:23 AM, Dan Berindei wrote:
> On Wed, Jun 28, 2017 at 5:25 PM, Radim Vansa <rvansa at redhat.com> wrote:
>> On 06/28/2017 01:17 PM, Radim Vansa wrote:
>>> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>>>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>>>> as many times as needed.
>>>>>>>
>>>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>>>
>>>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>>>> depressed) :)
>>>>>>>
>>>>>>> I'd remove these commands completely or possibly remove them just from
>>>>>>> public API and keep them internal.
>>>>>>>
>>>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>>>> for applications to use, and ignoring them when they can lead to
>>>>>> inconsistencies would make them useless.
>>>>>>
>>>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>>>> know when it is safe to skip the delete statement, and it relies on
>>>>>> the application making a (possibly wrong) choice.
>>>>>>
>>>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>>>> that applications use it right now. If query or listeners need the
>>>>>> previous value, then we should load it internally, but hide it from
>>>>>> the user.
>>>>>>
>>>>>> But removing it opens another discussion: should we replace it in the
>>>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>>>> should we make it the default and add a method
>>>>>> AdvancedCache.forceReturnPreviousValues()?
>>>>> Please don't derail the thread.
>>>>>
>>>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>>>> breaks the previous value for listeners, even if the QueryInterceptor
>>>> removes it from write commands. And it is public (+recommended) API,
>>>> in fact most if not all of our performance tests use it.
>>> That's just a flawed implementation. IPV is documented to be a 'safe'
>>> flag that should affect mostly primary -> origin replication, all the
>>> other is implementation. And we can fix that. Users should *not* expect
>>> that it e.g. skips loading from a cache store. We have already removed
>>> the modes that would be broken-by-design.
>>>
>>> On the other hand, write-only commands are not about *returning* the
>>> value but about (not) *reading* it, therefore (in my eyes) user could
>>> make that assumption and would like to enforce it this way. Even some
>>> docs explaining PersistenceMode.SKIP suggest that.
>>>
>>> I don't want to talk about flags, because I see all flags but IPV as
>>> 'effectively internal'. Let's discuss it more high-level. Some API
>>> exposes non-reading operation - we can see that under some circumstances
>>> this is not possible so we have options to 1) break stuff 2) break API
>>> assumptions 3) sometimes break API assumptions 4) remove such API (to
>>> not allow the user to make such assumptions). There's also an option 5)
>>> to fail the operation if the API assumption would be broken. Though, I
>>> don't fancy getting exception from a WriteOnlyMap.eval just because
>>> someone has registered a listener.
>>>
>>>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>>>> previous value on backup owners for most write commands
>>>> (LoadType.PRIMARY), we'd need to change that as well.
>>> Yes, all commands will have to load current value on all owners.
>> Btw., this has definitely some impact on shared cachestores. It's not
>> just that you have to load the value from cachestore - you also need to
>> store the previous value along with the current value (in metadata),
>> because if the primary owner crashes and the backups don't have it in
>> memory, these need to check the persistence to see if the command was
>> already executed.
>>
> I don't think this would work with the triangle algorithm. The primary
> owner updates the shared store without waiting for acks from the
> backups, so a backup trying to load the previous value is very likely
> to find the updated value instead.

With entry version history you'll be able to find the previous value 
even if it was updated, as long as the operation did not complete. 
That's why it's called a 'history'.

>
> I'll hazard a guess that none of the operations using LoadType.OWNER
> currently works with triangle + shared store.

IIRC triangle does not implement functional commands yet, and these 
don't work during retries anyway. ApplyDelta and put with delta-aware 
flag are about to go (and are probably broken in other interesting ways) 
and then there's the invalidation command that's set to OWNER to have 
more reliable listeners.

>
>> This means that the metadata could grow up to several times the size of
>> the value itself (if there's high concurrency). And even worse with
>> async caches, as these don't know when the command was actually finished
>> and therefore rely on time-based expiration of the previous values :-/
>>
>> Non-shared cachestores don't need to store previous values as long if we
>> can guarantee (flag) the entry as not eligible for eviction.
>>
>>>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am working on entry version history (again). In Como we've discussed
>>>>>>> that previous values are needed for (continuous) query and reliable
>>>>>>> listeners,
>>>>>>>
>>>>>>>
>>>>>>> Index based queries also require the previous value on a write - unless we
>>>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>>>> represent the content of a cache to be unique.
>>>>>>>
>>>>>>> Essentially we only need to know the type of the previous object. It might
>>>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>>>
>>>>>>> so I wonder what should we do with functional write-only
>>>>>>> commands. These are different to commands with flags, because flags
>>>>>>> (other than ignore return value) are expected to break something.
>>>>>>>
>>>>>>>
>>>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>>>> search the mailing list.
>>>>>>>
>>>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>>>> break things.
>>>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>>>> configuration/integrations veto them.
>>>>>>>
>>>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>>>> users having to solve puzzles, such as learning details about these flags
>>>>>>> and their possible side effects.
>>>>>>>
>>>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>>>> advantage of them?
>>>>>>>
>>>>>>> I see
>>>>>>> the available options as:
>>>>>>>
>>>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>>>> (warn user that he will break it)
>>>>>>>
>>>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>>>
>>>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>>>> stuff that could get broken)
>>>>>>>
>>>>>>>
>>>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>>>> likely to be often enabled.
>>>>>>> Having an either / or switch for different features in Infinispan will make
>>>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>>>> later struggle to fix or maintain.
>>>>>>>
>>>>>> I vote for this option.
>>>>>>
>>>>>> Query, listeners, and other components that need the previous value
>>>>>> should not just assume that the application knows better, they should
>>>>>> be able to change how operations works based on their needs. Of
>>>>>> course, the reverse is also true: if the application uses write-only
>>>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>>>> be possible for the user to detect why the previous values are still
>>>>>> loaded.
>>>>> If it were just query (static configuration), I would be okay with this
>>>>> idea. But as per listeners - besides tainting the design (event source
>>>>> should not check if there's a listener) you'd need to check *before*
>>>> The source wouldn't check for listeners explicitly, the notifier would
>>>> have an isPreviousValueNeeded() method and precompute that before a
>>>> listener is added or after a listener is removed. I was am assuming
>>>> some listeners will not need the previous value, e.g. the listeners
>>>> installed by streams.
>>> You can cover your warts with a make-up but you'll still have warts :)
>>>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>>>> EWI). So this is a space for race conditions or weird handling (if
>>>>> there's a listener when I am about to call notify and my flags are not
>>>>> cleared, skip the notification and pretend that this code was invoked
>>>>> before the listener was registered...). Or do you have another solution
>>>>> in mind (config option to disable listeners && all features using those?).
>>>>>
>>>> I was definitely going for the weird handling...
>>>>
>>>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>>>> it's loaded, and check that before invoking a listener that needs the
>>>> previous value. It is missing one edge case: if one thread starts a
>>>> write operation, then another thread installs a listener that requires
>>>> the previous value and iterates over the cache, the second thread may
>>>> not see the value written by the first thread.
>>> If the operations overlap, you could pretend that the write has finished
>>> before the listener was invoked and simply not notify the listener. If I
>>> am missing it please write it down in code. But handling this in any way
>>> is still clumsy.
>>>> So now I'm thinking we should retry the write commands when
>>>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>>>> performance difference is worth it.
>>>>
>>>>> R.
>>>>>
>>>>>>> 4) remove write-only commands completely (and probably functional
>>>>>>> listeners as well because these will lose their purpose)
>>>>>>>
>>>>>>>
>>>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>>>> be applied.
>>>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>>>> much for the reasons you're hitting now.
>>>>>>>
>>>>>> IMO version checks should be done internally, we shouldn't force the
>>>>>> users of the functional API to deal with versions themselves because
>>>>>> we know how hard making write skew checks work is for us :)
>>>>>>
>>>>>> And I wouldn't go as far as to remove the functional listeners,
>>>>>> instead I would change them so that read-write listeners are invoked
>>>>>> on write-only operations and they force the loading of the previous
>>>>>> value. I would also add a way for the regular listeners to say whether
>>>>>> they need the previous value or not.
>>>>>>
>>>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>>>> though, but it's asking for trouble.
>>>>>>>
>>>>>>>
>>>>>>> I agree!
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> WDYT?
>>>>>>>
>>>>>>> Radim
>>>>>>>
>>>>>> Cheers
>>>>>> Dan
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Thu Jun 29 05:16:18 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 29 Jun 2017 12:16:18 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <caea5f08-0b9d-25bd-4263-ec4ab2bc513b@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
	<CA+nfvwRR3toRrRQiXKYNZGjkeyx1gRaje1Q7yuyVGQaPD7Jk9Q@mail.gmail.com>
	<caea5f08-0b9d-25bd-4263-ec4ab2bc513b@redhat.com>
Message-ID: <CA+nfvwQpvbUpkzBEiLH6ppO9to-WAzQSan6XRkhNHNnVbK6SmA@mail.gmail.com>

On Thu, Jun 29, 2017 at 11:53 AM, Radim Vansa <rvansa at redhat.com> wrote:
> On 06/28/2017 04:20 PM, Dan Berindei wrote:
>> On Wed, Jun 28, 2017 at 2:17 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>>>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>>>> as many times as needed.
>>>>>>>
>>>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>>>
>>>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>>>> depressed) :)
>>>>>>>
>>>>>>> I'd remove these commands completely or possibly remove them just from
>>>>>>> public API and keep them internal.
>>>>>>>
>>>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>>>> for applications to use, and ignoring them when they can lead to
>>>>>> inconsistencies would make them useless.
>>>>>>
>>>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>>>> know when it is safe to skip the delete statement, and it relies on
>>>>>> the application making a (possibly wrong) choice.
>>>>>>
>>>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>>>> that applications use it right now. If query or listeners need the
>>>>>> previous value, then we should load it internally, but hide it from
>>>>>> the user.
>>>>>>
>>>>>> But removing it opens another discussion: should we replace it in the
>>>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>>>> should we make it the default and add a method
>>>>>> AdvancedCache.forceReturnPreviousValues()?
>>>>> Please don't derail the thread.
>>>>>
>>>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>>>> breaks the previous value for listeners, even if the QueryInterceptor
>>>> removes it from write commands. And it is public (+recommended) API,
>>>> in fact most if not all of our performance tests use it.
>>> That's just a flawed implementation. IPV is documented to be a 'safe'
>>> flag that should affect mostly primary -> origin replication, all the
>>> other is implementation. And we can fix that. Users should *not* expect
>>> that it e.g. skips loading from a cache store. We have already removed
>>> the modes that would be broken-by-design.
>>>
>> I think you're confusing IGNORE_RETURN_VALUES with SKIP_REMOTE_LOOKUP
>> here. The IVR javadoc doesn't say anything about remote lookups, only
>> SRL does.
>
> No, I am not; While IRV does not mention the replication, it's said to
> be 'safe'. So omitting the primary -> origin replication is basically
> all it can do when listeners are in place. You're right that I have
> missed the second part in SRL talking about put()s; I took it as a flag
> prohibiting any remote lookup (as the RPC operation in its whole) any
> time the remote value is needed. Yes, the second part seems equal to my
> understanding of IRV.
>
>>
>> And I agree that the current status is far from ideal, but there is
>> one more valid alternative: we can decide that the previous value is
>> only reliable in clustered listeners, and local listeners don't always
>> have it. Document that, make sure continuous query uses clustered
>> listeners, and we're done :)
>
> Unreliable return values are worse than none; I would rather remove them
> if we can't guarantee that these are right. Though, clustered listeners
> are based on regular listeners, so you'd need some means to make them
> reliable.

We could change the clustered listeners so that they're not based on
the regular listeners... I've been pestering Will about this ever
since the clustered listeners landed!

But I should have been clearer: I didn't mean that the listeners on
the backups should receive the previous value whenever we feel like
it, I meant we should document and enforce that the previous value is
only included in the event for listeners on the primary owner.

>>> On the other hand, write-only commands are not about *returning* the
>>> value but about (not) *reading* it, therefore (in my eyes) user could
>>> make that assumption and would like to enforce it this way. Even some
>>> docs explaining PersistenceMode.SKIP suggest that.
>>>
>> To me the purpose the same, there is no difference between returning
>> the previous value to the application or providing the previous value
>> via EntryView.
>
> There is a difference between what's provided locally and what's send
> over the network.
>
>> Applying this logic to the JCache API, it would mean
>> put() should never read the previous value, because some users could
>> assume that only getAndPut() reads it.
>
> OK, this is a valid point.
>
>>
>> In the old times we didn't have IGNORE_RETURN_VALUES, only
>> SKIP_REMOTE_LOOKUP+SKIP_CACHE_LOAD, and they would sometimes be
>> ignored (e.g. if the write was conditional). I think that's what
>> Galder had in mind when he wrote the PersistenceMode api note, not the
>> current behaviour of SKIP_CACHE_LOAD. I'll let Galder clarify this
>> himself, but I'll be very disappointed if he says he designed the
>> write-only operations so that they'll never work with query.
>>
>>
>>> I don't want to talk about flags, because I see all flags but IPV as
>>> 'effectively internal'. Let's discuss it more high-level. Some API
>>> exposes non-reading operation - we can see that under some circumstances
>>> this is not possible so we have options to 1) break stuff 2) break API
>>> assumptions 3) sometimes break API assumptions 4) remove such API (to
>>> not allow the user to make such assumptions). There's also an option 5)
>>> to fail the operation if the API assumption would be broken. Though, I
>>> don't fancy getting exception from a WriteOnlyMap.eval just because
>>> someone has registered a listener.
>>>
>> I disagree with the premise: there's no good reason for the user to
>> assume that write-only commands are *guaranteed* to never load the
>> previous value from a store. We just need to add a clarification to
>> the write-only operations' javadoc, no need to break anything.
>
> OK then, though it diminishes the value of write-only commands a lot.
>
>>
>>
>>>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>>>> previous value on backup owners for most write commands
>>>> (LoadType.PRIMARY), we'd need to change that as well.
>>> Yes, all commands will have to load current value on all owners.
>>>
>>>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am working on entry version history (again). In Como we've discussed
>>>>>>> that previous values are needed for (continuous) query and reliable
>>>>>>> listeners,
>>>>>>>
>>>>>>>
>>>>>>> Index based queries also require the previous value on a write - unless we
>>>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>>>> represent the content of a cache to be unique.
>>>>>>>
>>>>>>> Essentially we only need to know the type of the previous object. It might
>>>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>>>
>>>>>>> so I wonder what should we do with functional write-only
>>>>>>> commands. These are different to commands with flags, because flags
>>>>>>> (other than ignore return value) are expected to break something.
>>>>>>>
>>>>>>>
>>>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>>>> search the mailing list.
>>>>>>>
>>>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>>>> break things.
>>>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>>>> configuration/integrations veto them.
>>>>>>>
>>>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>>>> users having to solve puzzles, such as learning details about these flags
>>>>>>> and their possible side effects.
>>>>>>>
>>>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>>>> advantage of them?
>>>>>>>
>>>>>>> I see
>>>>>>> the available options as:
>>>>>>>
>>>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>>>> (warn user that he will break it)
>>>>>>>
>>>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>>>
>>>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>>>> stuff that could get broken)
>>>>>>>
>>>>>>>
>>>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>>>> likely to be often enabled.
>>>>>>> Having an either / or switch for different features in Infinispan will make
>>>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>>>> later struggle to fix or maintain.
>>>>>>>
>>>>>> I vote for this option.
>>>>>>
>>>>>> Query, listeners, and other components that need the previous value
>>>>>> should not just assume that the application knows better, they should
>>>>>> be able to change how operations works based on their needs. Of
>>>>>> course, the reverse is also true: if the application uses write-only
>>>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>>>> be possible for the user to detect why the previous values are still
>>>>>> loaded.
>>>>> If it were just query (static configuration), I would be okay with this
>>>>> idea. But as per listeners - besides tainting the design (event source
>>>>> should not check if there's a listener) you'd need to check *before*
>>>> The source wouldn't check for listeners explicitly, the notifier would
>>>> have an isPreviousValueNeeded() method and precompute that before a
>>>> listener is added or after a listener is removed. I was am assuming
>>>> some listeners will not need the previous value, e.g. the listeners
>>>> installed by streams.
>>> You can cover your warts with a make-up but you'll still have warts :)
>> Cutting them off doesn't necessarily work, either :)
>
> Yep, some people tend to fix w/ hacks instead of designing :)
>
>>
>>>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>>>> EWI). So this is a space for race conditions or weird handling (if
>>>>> there's a listener when I am about to call notify and my flags are not
>>>>> cleared, skip the notification and pretend that this code was invoked
>>>>> before the listener was registered...). Or do you have another solution
>>>>> in mind (config option to disable listeners && all features using those?).
>>>>>
>>>> I was definitely going for the weird handling...
>>>>
>>>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>>>> it's loaded, and check that before invoking a listener that needs the
>>>> previous value. It is missing one edge case: if one thread starts a
>>>> write operation, then another thread installs a listener that requires
>>>> the previous value and iterates over the cache, the second thread may
>>>> not see the value written by the first thread.
>>> If the operations overlap, you could pretend that the write has finished
>>> before the listener was invoked and simply not notify the listener. If I
>>> am missing it please write it down in code. But handling this in any way
>>> is still clumsy.
>> I hope pseudo-code is fine...
>>
>> 1. cache.put(k, v1) starts, doesn't load the previous value v0 in the context
>> 2. cache.addListener(l) runs, doesn't block
>> 3. cache.entrySet().forEach() runs, finds k->v0
>> 4. cache.put(k, v1) commits k->v1, should notify the listener but
>> doesn't have the previous value
>> 5. cache.put(k, v0) returns, but the code that installed the listener
>> thinks the value of k is still v0
>
> Oh OK, I should have drawn that myself when considering the scenario.
> You're right, here we'll have to retry.
>
> All in all, I think this discussion is done. We'll tell users to stick
> their flags where the sun doesn't shine and remove any inconvenient
> ones. Should we issue a warning any time we're removing the flag?
>

If you mean that we should remove the flags from the public API, I
agree. If you mean we should just ignore them, then no, because most
of the flags were added for internal components that really need their
semantics.

Dan


> Radim
>
>>
>>
>>>> So now I'm thinking we should retry the write commands when
>>>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>>>> performance difference is worth it.
>>>>
>>>>> R.
>>>>>
>>>>>>> 4) remove write-only commands completely (and probably functional
>>>>>>> listeners as well because these will lose their purpose)
>>>>>>>
>>>>>>>
>>>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>>>> be applied.
>>>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>>>> much for the reasons you're hitting now.
>>>>>>>
>>>>>> IMO version checks should be done internally, we shouldn't force the
>>>>>> users of the functional API to deal with versions themselves because
>>>>>> we know how hard making write skew checks work is for us :)
>>>>>>
>>>>>> And I wouldn't go as far as to remove the functional listeners,
>>>>>> instead I would change them so that read-write listeners are invoked
>>>>>> on write-only operations and they force the loading of the previous
>>>>>> value. I would also add a way for the regular listeners to say whether
>>>>>> they need the previous value or not.
>>>>>>
>>>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>>>> though, but it's asking for trouble.
>>>>>>>
>>>>>>>
>>>>>>> I agree!
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> WDYT?
>>>>>>>
>>>>>>> Radim
>>>>>>>
>>>>>> Cheers
>>>>>> Dan
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From rvansa at redhat.com  Thu Jun 29 07:19:25 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 29 Jun 2017 13:19:25 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CA+nfvwQpvbUpkzBEiLH6ppO9to-WAzQSan6XRkhNHNnVbK6SmA@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
	<CA+nfvwRR3toRrRQiXKYNZGjkeyx1gRaje1Q7yuyVGQaPD7Jk9Q@mail.gmail.com>
	<caea5f08-0b9d-25bd-4263-ec4ab2bc513b@redhat.com>
	<CA+nfvwQpvbUpkzBEiLH6ppO9to-WAzQSan6XRkhNHNnVbK6SmA@mail.gmail.com>
Message-ID: <b8f9a1e0-b0f7-e056-db92-e9e7c6b7fe20@redhat.com>

On 06/29/2017 11:16 AM, Dan Berindei wrote:
> On Thu, Jun 29, 2017 at 11:53 AM, Radim Vansa <rvansa at redhat.com> wrote:
>> On 06/28/2017 04:20 PM, Dan Berindei wrote:
>>> On Wed, Jun 28, 2017 at 2:17 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>>> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>>>>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>>>>> as many times as needed.
>>>>>>>>
>>>>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>>>>
>>>>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>>>>> depressed) :)
>>>>>>>>
>>>>>>>> I'd remove these commands completely or possibly remove them just from
>>>>>>>> public API and keep them internal.
>>>>>>>>
>>>>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>>>>> for applications to use, and ignoring them when they can lead to
>>>>>>> inconsistencies would make them useless.
>>>>>>>
>>>>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>>>>> know when it is safe to skip the delete statement, and it relies on
>>>>>>> the application making a (possibly wrong) choice.
>>>>>>>
>>>>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>>>>> that applications use it right now. If query or listeners need the
>>>>>>> previous value, then we should load it internally, but hide it from
>>>>>>> the user.
>>>>>>>
>>>>>>> But removing it opens another discussion: should we replace it in the
>>>>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>>>>> should we make it the default and add a method
>>>>>>> AdvancedCache.forceReturnPreviousValues()?
>>>>>> Please don't derail the thread.
>>>>>>
>>>>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>>>>> breaks the previous value for listeners, even if the QueryInterceptor
>>>>> removes it from write commands. And it is public (+recommended) API,
>>>>> in fact most if not all of our performance tests use it.
>>>> That's just a flawed implementation. IPV is documented to be a 'safe'
>>>> flag that should affect mostly primary -> origin replication, all the
>>>> other is implementation. And we can fix that. Users should *not* expect
>>>> that it e.g. skips loading from a cache store. We have already removed
>>>> the modes that would be broken-by-design.
>>>>
>>> I think you're confusing IGNORE_RETURN_VALUES with SKIP_REMOTE_LOOKUP
>>> here. The IVR javadoc doesn't say anything about remote lookups, only
>>> SRL does.
>> No, I am not; While IRV does not mention the replication, it's said to
>> be 'safe'. So omitting the primary -> origin replication is basically
>> all it can do when listeners are in place. You're right that I have
>> missed the second part in SRL talking about put()s; I took it as a flag
>> prohibiting any remote lookup (as the RPC operation in its whole) any
>> time the remote value is needed. Yes, the second part seems equal to my
>> understanding of IRV.
>>
>>> And I agree that the current status is far from ideal, but there is
>>> one more valid alternative: we can decide that the previous value is
>>> only reliable in clustered listeners, and local listeners don't always
>>> have it. Document that, make sure continuous query uses clustered
>>> listeners, and we're done :)
>> Unreliable return values are worse than none; I would rather remove them
>> if we can't guarantee that these are right. Though, clustered listeners
>> are based on regular listeners, so you'd need some means to make them
>> reliable.
> We could change the clustered listeners so that they're not based on
> the regular listeners... I've been pestering Will about this ever
> since the clustered listeners landed!
>
> But I should have been clearer: I didn't mean that the listeners on
> the backups should receive the previous value whenever we feel like
> it, I meant we should document and enforce that the previous value is
> only included in the event for listeners on the primary owner.
>>>> On the other hand, write-only commands are not about *returning* the
>>>> value but about (not) *reading* it, therefore (in my eyes) user could
>>>> make that assumption and would like to enforce it this way. Even some
>>>> docs explaining PersistenceMode.SKIP suggest that.
>>>>
>>> To me the purpose the same, there is no difference between returning
>>> the previous value to the application or providing the previous value
>>> via EntryView.
>> There is a difference between what's provided locally and what's send
>> over the network.
>>
>>> Applying this logic to the JCache API, it would mean
>>> put() should never read the previous value, because some users could
>>> assume that only getAndPut() reads it.
>> OK, this is a valid point.
>>
>>> In the old times we didn't have IGNORE_RETURN_VALUES, only
>>> SKIP_REMOTE_LOOKUP+SKIP_CACHE_LOAD, and they would sometimes be
>>> ignored (e.g. if the write was conditional). I think that's what
>>> Galder had in mind when he wrote the PersistenceMode api note, not the
>>> current behaviour of SKIP_CACHE_LOAD. I'll let Galder clarify this
>>> himself, but I'll be very disappointed if he says he designed the
>>> write-only operations so that they'll never work with query.
>>>
>>>
>>>> I don't want to talk about flags, because I see all flags but IPV as
>>>> 'effectively internal'. Let's discuss it more high-level. Some API
>>>> exposes non-reading operation - we can see that under some circumstances
>>>> this is not possible so we have options to 1) break stuff 2) break API
>>>> assumptions 3) sometimes break API assumptions 4) remove such API (to
>>>> not allow the user to make such assumptions). There's also an option 5)
>>>> to fail the operation if the API assumption would be broken. Though, I
>>>> don't fancy getting exception from a WriteOnlyMap.eval just because
>>>> someone has registered a listener.
>>>>
>>> I disagree with the premise: there's no good reason for the user to
>>> assume that write-only commands are *guaranteed* to never load the
>>> previous value from a store. We just need to add a clarification to
>>> the write-only operations' javadoc, no need to break anything.
>> OK then, though it diminishes the value of write-only commands a lot.
>>
>>>
>>>>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>>>>> previous value on backup owners for most write commands
>>>>> (LoadType.PRIMARY), we'd need to change that as well.
>>>> Yes, all commands will have to load current value on all owners.
>>>>
>>>>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am working on entry version history (again). In Como we've discussed
>>>>>>>> that previous values are needed for (continuous) query and reliable
>>>>>>>> listeners,
>>>>>>>>
>>>>>>>>
>>>>>>>> Index based queries also require the previous value on a write - unless we
>>>>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>>>>> represent the content of a cache to be unique.
>>>>>>>>
>>>>>>>> Essentially we only need to know the type of the previous object. It might
>>>>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>>>>
>>>>>>>> so I wonder what should we do with functional write-only
>>>>>>>> commands. These are different to commands with flags, because flags
>>>>>>>> (other than ignore return value) are expected to break something.
>>>>>>>>
>>>>>>>>
>>>>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>>>>> search the mailing list.
>>>>>>>>
>>>>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>>>>> break things.
>>>>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>>>>> configuration/integrations veto them.
>>>>>>>>
>>>>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>>>>> users having to solve puzzles, such as learning details about these flags
>>>>>>>> and their possible side effects.
>>>>>>>>
>>>>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>>>>> advantage of them?
>>>>>>>>
>>>>>>>> I see
>>>>>>>> the available options as:
>>>>>>>>
>>>>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>>>>> (warn user that he will break it)
>>>>>>>>
>>>>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>>>>
>>>>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>>>>> stuff that could get broken)
>>>>>>>>
>>>>>>>>
>>>>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>>>>> likely to be often enabled.
>>>>>>>> Having an either / or switch for different features in Infinispan will make
>>>>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>>>>> later struggle to fix or maintain.
>>>>>>>>
>>>>>>> I vote for this option.
>>>>>>>
>>>>>>> Query, listeners, and other components that need the previous value
>>>>>>> should not just assume that the application knows better, they should
>>>>>>> be able to change how operations works based on their needs. Of
>>>>>>> course, the reverse is also true: if the application uses write-only
>>>>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>>>>> be possible for the user to detect why the previous values are still
>>>>>>> loaded.
>>>>>> If it were just query (static configuration), I would be okay with this
>>>>>> idea. But as per listeners - besides tainting the design (event source
>>>>>> should not check if there's a listener) you'd need to check *before*
>>>>> The source wouldn't check for listeners explicitly, the notifier would
>>>>> have an isPreviousValueNeeded() method and precompute that before a
>>>>> listener is added or after a listener is removed. I was am assuming
>>>>> some listeners will not need the previous value, e.g. the listeners
>>>>> installed by streams.
>>>> You can cover your warts with a make-up but you'll still have warts :)
>>> Cutting them off doesn't necessarily work, either :)
>> Yep, some people tend to fix w/ hacks instead of designing :)
>>
>>>>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>>>>> EWI). So this is a space for race conditions or weird handling (if
>>>>>> there's a listener when I am about to call notify and my flags are not
>>>>>> cleared, skip the notification and pretend that this code was invoked
>>>>>> before the listener was registered...). Or do you have another solution
>>>>>> in mind (config option to disable listeners && all features using those?).
>>>>>>
>>>>> I was definitely going for the weird handling...
>>>>>
>>>>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>>>>> it's loaded, and check that before invoking a listener that needs the
>>>>> previous value. It is missing one edge case: if one thread starts a
>>>>> write operation, then another thread installs a listener that requires
>>>>> the previous value and iterates over the cache, the second thread may
>>>>> not see the value written by the first thread.
>>>> If the operations overlap, you could pretend that the write has finished
>>>> before the listener was invoked and simply not notify the listener. If I
>>>> am missing it please write it down in code. But handling this in any way
>>>> is still clumsy.
>>> I hope pseudo-code is fine...
>>>
>>> 1. cache.put(k, v1) starts, doesn't load the previous value v0 in the context
>>> 2. cache.addListener(l) runs, doesn't block
>>> 3. cache.entrySet().forEach() runs, finds k->v0
>>> 4. cache.put(k, v1) commits k->v1, should notify the listener but
>>> doesn't have the previous value
>>> 5. cache.put(k, v0) returns, but the code that installed the listener
>>> thinks the value of k is still v0
>> Oh OK, I should have drawn that myself when considering the scenario.
>> You're right, here we'll have to retry.
>>
>> All in all, I think this discussion is done. We'll tell users to stick
>> their flags where the sun doesn't shine and remove any inconvenient
>> ones. Should we issue a warning any time we're removing the flag?
>>
> If you mean that we should remove the flags from the public API, I
> agree. If you mean we should just ignore them, then no, because most
> of the flags were added for internal components that really need their
> semantics.

We can't remove them from public API before Infinspan 10, and I think 
that it will be a quite an unpopular step even after that. But until 10, 
I think that the common agreement was to not break query, that is ignore 
the flags. And make write-only reading.

R.

>
> Dan
>
>
>> Radim
>>
>>>
>>>>> So now I'm thinking we should retry the write commands when
>>>>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>>>>> performance difference is worth it.
>>>>>
>>>>>> R.
>>>>>>
>>>>>>>> 4) remove write-only commands completely (and probably functional
>>>>>>>> listeners as well because these will lose their purpose)
>>>>>>>>
>>>>>>>>
>>>>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>>>>> be applied.
>>>>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>>>>> much for the reasons you're hitting now.
>>>>>>>>
>>>>>>> IMO version checks should be done internally, we shouldn't force the
>>>>>>> users of the functional API to deal with versions themselves because
>>>>>>> we know how hard making write skew checks work is for us :)
>>>>>>>
>>>>>>> And I wouldn't go as far as to remove the functional listeners,
>>>>>>> instead I would change them so that read-write listeners are invoked
>>>>>>> on write-only operations and they force the loading of the previous
>>>>>>> value. I would also add a way for the regular listeners to say whether
>>>>>>> they need the previous value or not.
>>>>>>>
>>>>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>>>>> though, but it's asking for trouble.
>>>>>>>>
>>>>>>>>
>>>>>>>> I agree!
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>
>>>>>>>> WDYT?
>>>>>>>>
>>>>>>>> Radim
>>>>>>>>
>>>>>>> Cheers
>>>>>>> Dan
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> --
>>>> Radim Vansa <rvansa at redhat.com>
>>>> JBoss Performance Team
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Thu Jun 29 08:36:08 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 29 Jun 2017 15:36:08 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <b8f9a1e0-b0f7-e056-db92-e9e7c6b7fe20@redhat.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
	<CA+nfvwRR3toRrRQiXKYNZGjkeyx1gRaje1Q7yuyVGQaPD7Jk9Q@mail.gmail.com>
	<caea5f08-0b9d-25bd-4263-ec4ab2bc513b@redhat.com>
	<CA+nfvwQpvbUpkzBEiLH6ppO9to-WAzQSan6XRkhNHNnVbK6SmA@mail.gmail.com>
	<b8f9a1e0-b0f7-e056-db92-e9e7c6b7fe20@redhat.com>
Message-ID: <CA+nfvwQmTp-nuOUMACAD7d=BpyCNAd=LkRU7ZJCBs_ax-PGkfQ@mail.gmail.com>

On Thu, Jun 29, 2017 at 2:19 PM, Radim Vansa <rvansa at redhat.com> wrote:
> On 06/29/2017 11:16 AM, Dan Berindei wrote:
>> On Thu, Jun 29, 2017 at 11:53 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>> On 06/28/2017 04:20 PM, Dan Berindei wrote:
>>>> On Wed, Jun 28, 2017 at 2:17 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>>>>>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>>>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>>>>>> as many times as needed.
>>>>>>>>>
>>>>>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>>>>>
>>>>>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>>>>>> depressed) :)
>>>>>>>>>
>>>>>>>>> I'd remove these commands completely or possibly remove them just from
>>>>>>>>> public API and keep them internal.
>>>>>>>>>
>>>>>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>>>>>> for applications to use, and ignoring them when they can lead to
>>>>>>>> inconsistencies would make them useless.
>>>>>>>>
>>>>>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>>>>>> know when it is safe to skip the delete statement, and it relies on
>>>>>>>> the application making a (possibly wrong) choice.
>>>>>>>>
>>>>>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>>>>>> that applications use it right now. If query or listeners need the
>>>>>>>> previous value, then we should load it internally, but hide it from
>>>>>>>> the user.
>>>>>>>>
>>>>>>>> But removing it opens another discussion: should we replace it in the
>>>>>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>>>>>> should we make it the default and add a method
>>>>>>>> AdvancedCache.forceReturnPreviousValues()?
>>>>>>> Please don't derail the thread.
>>>>>>>
>>>>>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>>>>>> breaks the previous value for listeners, even if the QueryInterceptor
>>>>>> removes it from write commands. And it is public (+recommended) API,
>>>>>> in fact most if not all of our performance tests use it.
>>>>> That's just a flawed implementation. IPV is documented to be a 'safe'
>>>>> flag that should affect mostly primary -> origin replication, all the
>>>>> other is implementation. And we can fix that. Users should *not* expect
>>>>> that it e.g. skips loading from a cache store. We have already removed
>>>>> the modes that would be broken-by-design.
>>>>>
>>>> I think you're confusing IGNORE_RETURN_VALUES with SKIP_REMOTE_LOOKUP
>>>> here. The IVR javadoc doesn't say anything about remote lookups, only
>>>> SRL does.
>>> No, I am not; While IRV does not mention the replication, it's said to
>>> be 'safe'. So omitting the primary -> origin replication is basically
>>> all it can do when listeners are in place. You're right that I have
>>> missed the second part in SRL talking about put()s; I took it as a flag
>>> prohibiting any remote lookup (as the RPC operation in its whole) any
>>> time the remote value is needed. Yes, the second part seems equal to my
>>> understanding of IRV.
>>>
>>>> And I agree that the current status is far from ideal, but there is
>>>> one more valid alternative: we can decide that the previous value is
>>>> only reliable in clustered listeners, and local listeners don't always
>>>> have it. Document that, make sure continuous query uses clustered
>>>> listeners, and we're done :)
>>> Unreliable return values are worse than none; I would rather remove them
>>> if we can't guarantee that these are right. Though, clustered listeners
>>> are based on regular listeners, so you'd need some means to make them
>>> reliable.
>> We could change the clustered listeners so that they're not based on
>> the regular listeners... I've been pestering Will about this ever
>> since the clustered listeners landed!
>>
>> But I should have been clearer: I didn't mean that the listeners on
>> the backups should receive the previous value whenever we feel like
>> it, I meant we should document and enforce that the previous value is
>> only included in the event for listeners on the primary owner.
>>>>> On the other hand, write-only commands are not about *returning* the
>>>>> value but about (not) *reading* it, therefore (in my eyes) user could
>>>>> make that assumption and would like to enforce it this way. Even some
>>>>> docs explaining PersistenceMode.SKIP suggest that.
>>>>>
>>>> To me the purpose the same, there is no difference between returning
>>>> the previous value to the application or providing the previous value
>>>> via EntryView.
>>> There is a difference between what's provided locally and what's send
>>> over the network.
>>>
>>>> Applying this logic to the JCache API, it would mean
>>>> put() should never read the previous value, because some users could
>>>> assume that only getAndPut() reads it.
>>> OK, this is a valid point.
>>>
>>>> In the old times we didn't have IGNORE_RETURN_VALUES, only
>>>> SKIP_REMOTE_LOOKUP+SKIP_CACHE_LOAD, and they would sometimes be
>>>> ignored (e.g. if the write was conditional). I think that's what
>>>> Galder had in mind when he wrote the PersistenceMode api note, not the
>>>> current behaviour of SKIP_CACHE_LOAD. I'll let Galder clarify this
>>>> himself, but I'll be very disappointed if he says he designed the
>>>> write-only operations so that they'll never work with query.
>>>>
>>>>
>>>>> I don't want to talk about flags, because I see all flags but IPV as
>>>>> 'effectively internal'. Let's discuss it more high-level. Some API
>>>>> exposes non-reading operation - we can see that under some circumstances
>>>>> this is not possible so we have options to 1) break stuff 2) break API
>>>>> assumptions 3) sometimes break API assumptions 4) remove such API (to
>>>>> not allow the user to make such assumptions). There's also an option 5)
>>>>> to fail the operation if the API assumption would be broken. Though, I
>>>>> don't fancy getting exception from a WriteOnlyMap.eval just because
>>>>> someone has registered a listener.
>>>>>
>>>> I disagree with the premise: there's no good reason for the user to
>>>> assume that write-only commands are *guaranteed* to never load the
>>>> previous value from a store. We just need to add a clarification to
>>>> the write-only operations' javadoc, no need to break anything.
>>> OK then, though it diminishes the value of write-only commands a lot.
>>>
>>>>
>>>>>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>>>>>> previous value on backup owners for most write commands
>>>>>> (LoadType.PRIMARY), we'd need to change that as well.
>>>>> Yes, all commands will have to load current value on all owners.
>>>>>
>>>>>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I am working on entry version history (again). In Como we've discussed
>>>>>>>>> that previous values are needed for (continuous) query and reliable
>>>>>>>>> listeners,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Index based queries also require the previous value on a write - unless we
>>>>>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>>>>>> represent the content of a cache to be unique.
>>>>>>>>>
>>>>>>>>> Essentially we only need to know the type of the previous object. It might
>>>>>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>>>>>
>>>>>>>>> so I wonder what should we do with functional write-only
>>>>>>>>> commands. These are different to commands with flags, because flags
>>>>>>>>> (other than ignore return value) are expected to break something.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>>>>>> search the mailing list.
>>>>>>>>>
>>>>>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>>>>>> break things.
>>>>>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>>>>>> configuration/integrations veto them.
>>>>>>>>>
>>>>>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>>>>>> users having to solve puzzles, such as learning details about these flags
>>>>>>>>> and their possible side effects.
>>>>>>>>>
>>>>>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>>>>>> advantage of them?
>>>>>>>>>
>>>>>>>>> I see
>>>>>>>>> the available options as:
>>>>>>>>>
>>>>>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>>>>>> (warn user that he will break it)
>>>>>>>>>
>>>>>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>>>>>
>>>>>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>>>>>> stuff that could get broken)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>>>>>> likely to be often enabled.
>>>>>>>>> Having an either / or switch for different features in Infinispan will make
>>>>>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>>>>>> later struggle to fix or maintain.
>>>>>>>>>
>>>>>>>> I vote for this option.
>>>>>>>>
>>>>>>>> Query, listeners, and other components that need the previous value
>>>>>>>> should not just assume that the application knows better, they should
>>>>>>>> be able to change how operations works based on their needs. Of
>>>>>>>> course, the reverse is also true: if the application uses write-only
>>>>>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>>>>>> be possible for the user to detect why the previous values are still
>>>>>>>> loaded.
>>>>>>> If it were just query (static configuration), I would be okay with this
>>>>>>> idea. But as per listeners - besides tainting the design (event source
>>>>>>> should not check if there's a listener) you'd need to check *before*
>>>>>> The source wouldn't check for listeners explicitly, the notifier would
>>>>>> have an isPreviousValueNeeded() method and precompute that before a
>>>>>> listener is added or after a listener is removed. I was am assuming
>>>>>> some listeners will not need the previous value, e.g. the listeners
>>>>>> installed by streams.
>>>>> You can cover your warts with a make-up but you'll still have warts :)
>>>> Cutting them off doesn't necessarily work, either :)
>>> Yep, some people tend to fix w/ hacks instead of designing :)
>>>
>>>>>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>>>>>> EWI). So this is a space for race conditions or weird handling (if
>>>>>>> there's a listener when I am about to call notify and my flags are not
>>>>>>> cleared, skip the notification and pretend that this code was invoked
>>>>>>> before the listener was registered...). Or do you have another solution
>>>>>>> in mind (config option to disable listeners && all features using those?).
>>>>>>>
>>>>>> I was definitely going for the weird handling...
>>>>>>
>>>>>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>>>>>> it's loaded, and check that before invoking a listener that needs the
>>>>>> previous value. It is missing one edge case: if one thread starts a
>>>>>> write operation, then another thread installs a listener that requires
>>>>>> the previous value and iterates over the cache, the second thread may
>>>>>> not see the value written by the first thread.
>>>>> If the operations overlap, you could pretend that the write has finished
>>>>> before the listener was invoked and simply not notify the listener. If I
>>>>> am missing it please write it down in code. But handling this in any way
>>>>> is still clumsy.
>>>> I hope pseudo-code is fine...
>>>>
>>>> 1. cache.put(k, v1) starts, doesn't load the previous value v0 in the context
>>>> 2. cache.addListener(l) runs, doesn't block
>>>> 3. cache.entrySet().forEach() runs, finds k->v0
>>>> 4. cache.put(k, v1) commits k->v1, should notify the listener but
>>>> doesn't have the previous value
>>>> 5. cache.put(k, v0) returns, but the code that installed the listener
>>>> thinks the value of k is still v0
>>> Oh OK, I should have drawn that myself when considering the scenario.
>>> You're right, here we'll have to retry.
>>>
>>> All in all, I think this discussion is done. We'll tell users to stick
>>> their flags where the sun doesn't shine and remove any inconvenient
>>> ones. Should we issue a warning any time we're removing the flag?
>>>
>> If you mean that we should remove the flags from the public API, I
>> agree. If you mean we should just ignore them, then no, because most
>> of the flags were added for internal components that really need their
>> semantics.
>
> We can't remove them from public API before Infinspan 10, and I think
> that it will be a quite an unpopular step even after that. But until 10,
> I think that the common agreement was to not break query, that is ignore
> the flags. And make write-only reading.
>

So SKIP_INDEXING should not skip indexing because it can break query??

> R.
>
>>
>> Dan
>>
>>
>>> Radim
>>>
>>>>
>>>>>> So now I'm thinking we should retry the write commands when
>>>>>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>>>>>> performance difference is worth it.
>>>>>>
>>>>>>> R.
>>>>>>>
>>>>>>>>> 4) remove write-only commands completely (and probably functional
>>>>>>>>> listeners as well because these will lose their purpose)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>>>>>> be applied.
>>>>>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>>>>>> much for the reasons you're hitting now.
>>>>>>>>>
>>>>>>>> IMO version checks should be done internally, we shouldn't force the
>>>>>>>> users of the functional API to deal with versions themselves because
>>>>>>>> we know how hard making write skew checks work is for us :)
>>>>>>>>
>>>>>>>> And I wouldn't go as far as to remove the functional listeners,
>>>>>>>> instead I would change them so that read-write listeners are invoked
>>>>>>>> on write-only operations and they force the loading of the previous
>>>>>>>> value. I would also add a way for the regular listeners to say whether
>>>>>>>> they need the previous value or not.
>>>>>>>>
>>>>>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>>>>>> though, but it's asking for trouble.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I agree!
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> WDYT?
>>>>>>>>>
>>>>>>>>> Radim
>>>>>>>>>
>>>>>>>> Cheers
>>>>>>>> Dan
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> --
>>>>> Radim Vansa <rvansa at redhat.com>
>>>>> JBoss Performance Team
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From anistor at redhat.com  Thu Jun 29 08:45:18 2017
From: anistor at redhat.com (Adrian Nistor)
Date: Thu, 29 Jun 2017 15:45:18 +0300
Subject: [infinispan-dev] Feedback for PR 5233 needed
Message-ID: <ea32a8b0-9338-801f-c20f-056bd38242a4@redhat.com>

This pr [1] adds a new approach for defining the compat marshaller class 
and the indexed entity classes (in server), and the same approach could 
be used in future for deployment of encoders,  lucene analyzers and 
possilby other code bits that a user would want to add a server in order 
to implement an extension point that we support.

Your feedback is wellcome!

[1] https://github.com/infinispan/infinispan/pull/5233


From rvansa at redhat.com  Thu Jun 29 09:51:22 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 29 Jun 2017 15:51:22 +0200
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CA+nfvwQmTp-nuOUMACAD7d=BpyCNAd=LkRU7ZJCBs_ax-PGkfQ@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
	<CA+nfvwRR3toRrRQiXKYNZGjkeyx1gRaje1Q7yuyVGQaPD7Jk9Q@mail.gmail.com>
	<caea5f08-0b9d-25bd-4263-ec4ab2bc513b@redhat.com>
	<CA+nfvwQpvbUpkzBEiLH6ppO9to-WAzQSan6XRkhNHNnVbK6SmA@mail.gmail.com>
	<b8f9a1e0-b0f7-e056-db92-e9e7c6b7fe20@redhat.com>
	<CA+nfvwQmTp-nuOUMACAD7d=BpyCNAd=LkRU7ZJCBs_ax-PGkfQ@mail.gmail.com>
Message-ID: <c079a93e-8638-7171-ace5-ce03cdf17ee7@redhat.com>

On 06/29/2017 02:36 PM, Dan Berindei wrote:
> On Thu, Jun 29, 2017 at 2:19 PM, Radim Vansa <rvansa at redhat.com> wrote:
>> On 06/29/2017 11:16 AM, Dan Berindei wrote:
>>> On Thu, Jun 29, 2017 at 11:53 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>> On 06/28/2017 04:20 PM, Dan Berindei wrote:
>>>>> On Wed, Jun 28, 2017 at 2:17 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>>>>>>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>>>>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>>>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>>>>>>> as many times as needed.
>>>>>>>>>>
>>>>>>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>>>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>>>>>>
>>>>>>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>>>>>>> depressed) :)
>>>>>>>>>>
>>>>>>>>>> I'd remove these commands completely or possibly remove them just from
>>>>>>>>>> public API and keep them internal.
>>>>>>>>>>
>>>>>>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>>>>>>> for applications to use, and ignoring them when they can lead to
>>>>>>>>> inconsistencies would make them useless.
>>>>>>>>>
>>>>>>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>>>>>>> know when it is safe to skip the delete statement, and it relies on
>>>>>>>>> the application making a (possibly wrong) choice.
>>>>>>>>>
>>>>>>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>>>>>>> that applications use it right now. If query or listeners need the
>>>>>>>>> previous value, then we should load it internally, but hide it from
>>>>>>>>> the user.
>>>>>>>>>
>>>>>>>>> But removing it opens another discussion: should we replace it in the
>>>>>>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>>>>>>> should we make it the default and add a method
>>>>>>>>> AdvancedCache.forceReturnPreviousValues()?
>>>>>>>> Please don't derail the thread.
>>>>>>>>
>>>>>>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>>>>>>> breaks the previous value for listeners, even if the QueryInterceptor
>>>>>>> removes it from write commands. And it is public (+recommended) API,
>>>>>>> in fact most if not all of our performance tests use it.
>>>>>> That's just a flawed implementation. IPV is documented to be a 'safe'
>>>>>> flag that should affect mostly primary -> origin replication, all the
>>>>>> other is implementation. And we can fix that. Users should *not* expect
>>>>>> that it e.g. skips loading from a cache store. We have already removed
>>>>>> the modes that would be broken-by-design.
>>>>>>
>>>>> I think you're confusing IGNORE_RETURN_VALUES with SKIP_REMOTE_LOOKUP
>>>>> here. The IVR javadoc doesn't say anything about remote lookups, only
>>>>> SRL does.
>>>> No, I am not; While IRV does not mention the replication, it's said to
>>>> be 'safe'. So omitting the primary -> origin replication is basically
>>>> all it can do when listeners are in place. You're right that I have
>>>> missed the second part in SRL talking about put()s; I took it as a flag
>>>> prohibiting any remote lookup (as the RPC operation in its whole) any
>>>> time the remote value is needed. Yes, the second part seems equal to my
>>>> understanding of IRV.
>>>>
>>>>> And I agree that the current status is far from ideal, but there is
>>>>> one more valid alternative: we can decide that the previous value is
>>>>> only reliable in clustered listeners, and local listeners don't always
>>>>> have it. Document that, make sure continuous query uses clustered
>>>>> listeners, and we're done :)
>>>> Unreliable return values are worse than none; I would rather remove them
>>>> if we can't guarantee that these are right. Though, clustered listeners
>>>> are based on regular listeners, so you'd need some means to make them
>>>> reliable.
>>> We could change the clustered listeners so that they're not based on
>>> the regular listeners... I've been pestering Will about this ever
>>> since the clustered listeners landed!
>>>
>>> But I should have been clearer: I didn't mean that the listeners on
>>> the backups should receive the previous value whenever we feel like
>>> it, I meant we should document and enforce that the previous value is
>>> only included in the event for listeners on the primary owner.
>>>>>> On the other hand, write-only commands are not about *returning* the
>>>>>> value but about (not) *reading* it, therefore (in my eyes) user could
>>>>>> make that assumption and would like to enforce it this way. Even some
>>>>>> docs explaining PersistenceMode.SKIP suggest that.
>>>>>>
>>>>> To me the purpose the same, there is no difference between returning
>>>>> the previous value to the application or providing the previous value
>>>>> via EntryView.
>>>> There is a difference between what's provided locally and what's send
>>>> over the network.
>>>>
>>>>> Applying this logic to the JCache API, it would mean
>>>>> put() should never read the previous value, because some users could
>>>>> assume that only getAndPut() reads it.
>>>> OK, this is a valid point.
>>>>
>>>>> In the old times we didn't have IGNORE_RETURN_VALUES, only
>>>>> SKIP_REMOTE_LOOKUP+SKIP_CACHE_LOAD, and they would sometimes be
>>>>> ignored (e.g. if the write was conditional). I think that's what
>>>>> Galder had in mind when he wrote the PersistenceMode api note, not the
>>>>> current behaviour of SKIP_CACHE_LOAD. I'll let Galder clarify this
>>>>> himself, but I'll be very disappointed if he says he designed the
>>>>> write-only operations so that they'll never work with query.
>>>>>
>>>>>
>>>>>> I don't want to talk about flags, because I see all flags but IPV as
>>>>>> 'effectively internal'. Let's discuss it more high-level. Some API
>>>>>> exposes non-reading operation - we can see that under some circumstances
>>>>>> this is not possible so we have options to 1) break stuff 2) break API
>>>>>> assumptions 3) sometimes break API assumptions 4) remove such API (to
>>>>>> not allow the user to make such assumptions). There's also an option 5)
>>>>>> to fail the operation if the API assumption would be broken. Though, I
>>>>>> don't fancy getting exception from a WriteOnlyMap.eval just because
>>>>>> someone has registered a listener.
>>>>>>
>>>>> I disagree with the premise: there's no good reason for the user to
>>>>> assume that write-only commands are *guaranteed* to never load the
>>>>> previous value from a store. We just need to add a clarification to
>>>>> the write-only operations' javadoc, no need to break anything.
>>>> OK then, though it diminishes the value of write-only commands a lot.
>>>>
>>>>>>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>>>>>>> previous value on backup owners for most write commands
>>>>>>> (LoadType.PRIMARY), we'd need to change that as well.
>>>>>> Yes, all commands will have to load current value on all owners.
>>>>>>
>>>>>>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I am working on entry version history (again). In Como we've discussed
>>>>>>>>>> that previous values are needed for (continuous) query and reliable
>>>>>>>>>> listeners,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Index based queries also require the previous value on a write - unless we
>>>>>>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>>>>>>> represent the content of a cache to be unique.
>>>>>>>>>>
>>>>>>>>>> Essentially we only need to know the type of the previous object. It might
>>>>>>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>>>>>>
>>>>>>>>>> so I wonder what should we do with functional write-only
>>>>>>>>>> commands. These are different to commands with flags, because flags
>>>>>>>>>> (other than ignore return value) are expected to break something.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>>>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>>>>>>> search the mailing list.
>>>>>>>>>>
>>>>>>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>>>>>>> break things.
>>>>>>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>>>>>>> configuration/integrations veto them.
>>>>>>>>>>
>>>>>>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>>>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>>>>>>> users having to solve puzzles, such as learning details about these flags
>>>>>>>>>> and their possible side effects.
>>>>>>>>>>
>>>>>>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>>>>>>> advantage of them?
>>>>>>>>>>
>>>>>>>>>> I see
>>>>>>>>>> the available options as:
>>>>>>>>>>
>>>>>>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>>>>>>> (warn user that he will break it)
>>>>>>>>>>
>>>>>>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>>>>>>
>>>>>>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>>>>>>> stuff that could get broken)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>>>>>>> likely to be often enabled.
>>>>>>>>>> Having an either / or switch for different features in Infinispan will make
>>>>>>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>>>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>>>>>>> later struggle to fix or maintain.
>>>>>>>>>>
>>>>>>>>> I vote for this option.
>>>>>>>>>
>>>>>>>>> Query, listeners, and other components that need the previous value
>>>>>>>>> should not just assume that the application knows better, they should
>>>>>>>>> be able to change how operations works based on their needs. Of
>>>>>>>>> course, the reverse is also true: if the application uses write-only
>>>>>>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>>>>>>> be possible for the user to detect why the previous values are still
>>>>>>>>> loaded.
>>>>>>>> If it were just query (static configuration), I would be okay with this
>>>>>>>> idea. But as per listeners - besides tainting the design (event source
>>>>>>>> should not check if there's a listener) you'd need to check *before*
>>>>>>> The source wouldn't check for listeners explicitly, the notifier would
>>>>>>> have an isPreviousValueNeeded() method and precompute that before a
>>>>>>> listener is added or after a listener is removed. I was am assuming
>>>>>>> some listeners will not need the previous value, e.g. the listeners
>>>>>>> installed by streams.
>>>>>> You can cover your warts with a make-up but you'll still have warts :)
>>>>> Cutting them off doesn't necessarily work, either :)
>>>> Yep, some people tend to fix w/ hacks instead of designing :)
>>>>
>>>>>>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>>>>>>> EWI). So this is a space for race conditions or weird handling (if
>>>>>>>> there's a listener when I am about to call notify and my flags are not
>>>>>>>> cleared, skip the notification and pretend that this code was invoked
>>>>>>>> before the listener was registered...). Or do you have another solution
>>>>>>>> in mind (config option to disable listeners && all features using those?).
>>>>>>>>
>>>>>>> I was definitely going for the weird handling...
>>>>>>>
>>>>>>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>>>>>>> it's loaded, and check that before invoking a listener that needs the
>>>>>>> previous value. It is missing one edge case: if one thread starts a
>>>>>>> write operation, then another thread installs a listener that requires
>>>>>>> the previous value and iterates over the cache, the second thread may
>>>>>>> not see the value written by the first thread.
>>>>>> If the operations overlap, you could pretend that the write has finished
>>>>>> before the listener was invoked and simply not notify the listener. If I
>>>>>> am missing it please write it down in code. But handling this in any way
>>>>>> is still clumsy.
>>>>> I hope pseudo-code is fine...
>>>>>
>>>>> 1. cache.put(k, v1) starts, doesn't load the previous value v0 in the context
>>>>> 2. cache.addListener(l) runs, doesn't block
>>>>> 3. cache.entrySet().forEach() runs, finds k->v0
>>>>> 4. cache.put(k, v1) commits k->v1, should notify the listener but
>>>>> doesn't have the previous value
>>>>> 5. cache.put(k, v0) returns, but the code that installed the listener
>>>>> thinks the value of k is still v0
>>>> Oh OK, I should have drawn that myself when considering the scenario.
>>>> You're right, here we'll have to retry.
>>>>
>>>> All in all, I think this discussion is done. We'll tell users to stick
>>>> their flags where the sun doesn't shine and remove any inconvenient
>>>> ones. Should we issue a warning any time we're removing the flag?
>>>>
>>> If you mean that we should remove the flags from the public API, I
>>> agree. If you mean we should just ignore them, then no, because most
>>> of the flags were added for internal components that really need their
>>> semantics.
>> We can't remove them from public API before Infinspan 10, and I think
>> that it will be a quite an unpopular step even after that. But until 10,
>> I think that the common agreement was to not break query, that is ignore
>> the flags. And make write-only reading.
>>
> So SKIP_INDEXING should not skip indexing because it can break query??

Ehm... Talking about all flags was wrong, and I think that I've also 
mixed your input on write-only command and on flags. Let's reiterate, 
until we hide the flags (in 10+):

A) how should we treat SKIP_CACHE_LOAD with respect to (clustered) 
listeners, query, and write skew check? (IIRC we ignore that for 
purposes of WSC)
B) for write-only, will we load the value if necessary 
(listeners/query/wsc)? (I guess that the answer is yes)
C) for write-only, will we treat PersistenceMode.SKIP differently?
D) how should we treat SKIP_REMOTE_LOOKUP when the current write-owner 
is not a read-owner?

R.

>
>> R.
>>
>>> Dan
>>>
>>>
>>>> Radim
>>>>
>>>>>>> So now I'm thinking we should retry the write commands when
>>>>>>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>>>>>>> performance difference is worth it.
>>>>>>>
>>>>>>>> R.
>>>>>>>>
>>>>>>>>>> 4) remove write-only commands completely (and probably functional
>>>>>>>>>> listeners as well because these will lose their purpose)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>>>>>>> be applied.
>>>>>>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>>>>>>> much for the reasons you're hitting now.
>>>>>>>>>>
>>>>>>>>> IMO version checks should be done internally, we shouldn't force the
>>>>>>>>> users of the functional API to deal with versions themselves because
>>>>>>>>> we know how hard making write skew checks work is for us :)
>>>>>>>>>
>>>>>>>>> And I wouldn't go as far as to remove the functional listeners,
>>>>>>>>> instead I would change them so that read-write listeners are invoked
>>>>>>>>> on write-only operations and they force the loading of the previous
>>>>>>>>> value. I would also add a way for the regular listeners to say whether
>>>>>>>>> they need the previous value or not.
>>>>>>>>>
>>>>>>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>>>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>>>>>>> though, but it's asking for trouble.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I agree!
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> WDYT?
>>>>>>>>>>
>>>>>>>>>> Radim
>>>>>>>>>>
>>>>>>>>> Cheers
>>>>>>>>> Dan
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> --
>>>>>> Radim Vansa <rvansa at redhat.com>
>>>>>> JBoss Performance Team
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> --
>>>> Radim Vansa <rvansa at redhat.com>
>>>> JBoss Performance Team
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From anistor at redhat.com  Thu Jun 29 11:20:15 2017
From: anistor at redhat.com (Adrian Nistor)
Date: Thu, 29 Jun 2017 18:20:15 +0300
Subject: [infinispan-dev] Feedback for PR 5233 needed
In-Reply-To: <ea32a8b0-9338-801f-c20f-056bd38242a4@redhat.com>
References: <ea32a8b0-9338-801f-c20f-056bd38242a4@redhat.com>
Message-ID: <b1858d9f-e898-d7ca-9f62-702918cb7505@redhat.com>

People, don't be shy, the PR is in now, but things can still change 
based on you feedback. We still have two weeks until we release the Final.

On 06/29/2017 03:45 PM, Adrian Nistor wrote:
> This pr [1] adds a new approach for defining the compat marshaller class
> and the indexed entity classes (in server), and the same approach could
> be used in future for deployment of encoders,  lucene analyzers and
> possilby other code bits that a user would want to add a server in order
> to implement an extension point that we support.
>
> Your feedback is wellcome!
>
> [1] https://github.com/infinispan/infinispan/pull/5233
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From tsegismont at gmail.com  Fri Jun 30 05:40:10 2017
From: tsegismont at gmail.com (Thomas SEGISMONT)
Date: Fri, 30 Jun 2017 11:40:10 +0200
Subject: [infinispan-dev] KUBE_PING changes
In-Reply-To: <CAA9R9O6Vhrd2No5dQd6NawuuAx5VbFdkKsFOvNs5aT0qp9ftRQ@mail.gmail.com>
References: <CAA9R9O7U3w-if7XyLc3bRPM_GVyzr9ywChm+222EptrNSbA8Tg@mail.gmail.com>
	<6b3dc945-449b-1c9b-1384-4a4cdde83eca@mailbox.org>
	<CAA9R9O6Vhrd2No5dQd6NawuuAx5VbFdkKsFOvNs5aT0qp9ftRQ@mail.gmail.com>
Message-ID: <CACiEr_ScR-ANveF5v1dNBfvvd4P_=Dax77SGYnhEH75cMoY4VA@mail.gmail.com>

Hi everyone,

Thank you for this great work, the dependency diet and the extra port
removal are both very useful. The extra port removal is key to enable
Vert.x clustering in Openshift S2I environments.

I tried the new KUBE_PING (beta1) with vertx-infinispan and it worked fine.
I have a few questions though.

I couldn't configure it with env variables. Before you ask, yes I noticed
the name changes ;-) I only had a quick look at JGroups config code but it
seems it only resolves system properties. Did it work for you because you
tried with an Infinispan server?

Since I couldn't configure it externally I had to create a custom JGroups
file. Usually, we recommend [1] Vert.x users to add the infinispan-cloud
dependency and a system property:
-Dvertx.jgroups.config=default-configs/default-jgroups-kubernetes.xml

My custom JGroups file is a just a copy of
default-configs/default-jgroups-kubernetes.xml in which I added the
masterHost and namespace properties.

Is it still recommended to use the
default-configs/default-jgroups-kubernetes.xml stack ? Or is any change
planned after the KUBE_PING changes?
I wouldn't expect a protocol implementation change to impact a stack
configuration but they say there are no stupid questions :)

Thank you,
Thomas


[1] http://vertx.io/docs/vertx-infinispan/java/#_configuring_for_openshift_3

2017-06-15 8:21 GMT+02:00 Sebastian Laskawiec <slaskawi at redhat.com>:

> Yep, no problems found!!!
>
> I had also impression that the new implementation is "faster". Though I
> haven't measured it... it just my impression.
>
> Awesome work Bela!
>
> On Thu, Jun 15, 2017 at 7:42 AM Bela Ban <belaban at mailbox.org> wrote:
>
>> Thanks, Sebastian!
>>
>> I assume testing on GKE and minikube/openshift was successful?
>>
>>
>> On 14/06/17 13:15, Sebastian Laskawiec wrote:
>> > Hey guys,
>> >
>> > Just a heads up, I've just created a PR that upgrades KUBE_PING to
>> > 1.0.0.Beta1 [1]. As you probably seen in [2], 1.0.0.Beta1 was completely
>> > rewritten and might behave slightly differently.
>> >
>> > Here is a summary of changes:
>> >
>> >   * The latest KUBE_PING doesn't require embedded HTTP server for
>> >     discovery. Thus it is no longer required to expose port 8888 in
>> Pods.
>> >   * The number of dependencies has been decreased. Currently we only
>> >     require JGroups and OAuth library.
>> >   * The new KUBE_PING works only with JGroups 4. There will be no
>> >     JGroups 3 support.
>> >   * Some of the environmental variables were shortened and we removed
>> >     `OPENSHIFT` prefix. So if you use `OPENSHIFT_KUBE_PING_NAMESPACE`,
>> >     you will need to change it to `KUBERNETES_NAMESPACE`. Please refer
>> >     to [3] for more information.
>> >
>> > I also switched default branch in Kubernetes Ping repository to master
>> [4].
>> >
>> > Thanks,
>> > Sebastian
>> >
>> > [1] https://github.com/infinispan/infinispan/pull/5201
>> > [2] http://belaban.blogspot.ch/2017/05/running-infinispan-
>> cluster-with.html
>> > [3] https://github.com/jgroups-extras/jgroups-kubernetes/
>> blob/master/README.adoc
>> > [4] https://github.com/jgroups-extras/jgroups-kubernetes
>> > --
>> >
>> > SEBASTIAN ?ASKAWIEC
>> >
>> > INFINISPAN DEVELOPER
>> >
>> > Red Hat EMEA <https://www.redhat.com/>
>> >
>> > <https://red.ht/sig>
>> >
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >
>>
>> --
>> Bela Ban | http://www.jgroups.org
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
>
> SEBASTIAN ?ASKAWIEC
>
> INFINISPAN DEVELOPER
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170630/1a3c6ec3/attachment.html 

From anistor at redhat.com  Fri Jun 30 07:13:08 2017
From: anistor at redhat.com (Adrian Nistor)
Date: Fri, 30 Jun 2017 14:13:08 +0300
Subject: [infinispan-dev] Write-only commands
In-Reply-To: <CA+nfvwQmTp-nuOUMACAD7d=BpyCNAd=LkRU7ZJCBs_ax-PGkfQ@mail.gmail.com>
References: <a946fffd-9de9-3f7a-fa5a-77600424447d@redhat.com>
	<CAFm4XO13KNZW2au2oOtZLqV_qugH_ZcL03MO5g1YTLUZSPpuNQ@mail.gmail.com>
	<8b0b3142-5dc3-c5d6-675c-dd3d2b2723e5@redhat.com>
	<CA+nfvwR_7YC2y+TO_azsJ11umxrpo=KP-TmrP+DZgei7qTV3Ag@mail.gmail.com>
	<3a1c9888-0b9a-7bae-8986-57568ca4668b@redhat.com>
	<CA+nfvwTpwU+j6fXbsCthAE7eW5c-ePnO7_SE=mMyjLfwr0faYA@mail.gmail.com>
	<3e17fe7b-a860-0143-bafd-a2799ebe6ee1@redhat.com>
	<CA+nfvwRR3toRrRQiXKYNZGjkeyx1gRaje1Q7yuyVGQaPD7Jk9Q@mail.gmail.com>
	<caea5f08-0b9d-25bd-4263-ec4ab2bc513b@redhat.com>
	<CA+nfvwQpvbUpkzBEiLH6ppO9to-WAzQSan6XRkhNHNnVbK6SmA@mail.gmail.com>
	<b8f9a1e0-b0f7-e056-db92-e9e7c6b7fe20@redhat.com>
	<CA+nfvwQmTp-nuOUMACAD7d=BpyCNAd=LkRU7ZJCBs_ax-PGkfQ@mail.gmail.com>
Message-ID: <08e7f3d5-8b04-40d8-0214-ff6206177a3b@redhat.com>


<<So SKIP_INDEXING should not skip indexing because it can break query??>>

Dan, drawing the conversation into absurdity is not useful.



On 06/29/2017 03:36 PM, Dan Berindei wrote:
> On Thu, Jun 29, 2017 at 2:19 PM, Radim Vansa <rvansa at redhat.com> wrote:
>> On 06/29/2017 11:16 AM, Dan Berindei wrote:
>>> On Thu, Jun 29, 2017 at 11:53 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>> On 06/28/2017 04:20 PM, Dan Berindei wrote:
>>>>> On Wed, Jun 28, 2017 at 2:17 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>> On 06/28/2017 10:40 AM, Dan Berindei wrote:
>>>>>>> On Wed, Jun 28, 2017 at 10:17 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>>>> On 06/27/2017 03:54 PM, Dan Berindei wrote:
>>>>>>>>> On Tue, Jun 27, 2017 at 2:43 PM, Adrian Nistor <anistor at redhat.com> wrote:
>>>>>>>>>> I've said this in a previous thread on this same issue, I will repeat myself
>>>>>>>>>> as many times as needed.
>>>>>>>>>>
>>>>>>>>>> Continuous queries require the previous value itself, not just knowledge of
>>>>>>>>>> the type of the previous value. Strongly typed caches solve no problem here.
>>>>>>>>>>
>>>>>>>>>> So if we half-fix query but leave CQ broken I will be half-happy (ie. very
>>>>>>>>>> depressed) :)
>>>>>>>>>>
>>>>>>>>>> I'd remove these commands completely or possibly remove them just from
>>>>>>>>>> public API and keep them internal.
>>>>>>>>>>
>>>>>>>>> +1 to remove the flags from the public API. Most of them are not safe
>>>>>>>>> for applications to use, and ignoring them when they can lead to
>>>>>>>>> inconsistencies would make them useless.
>>>>>>>>>
>>>>>>>>> E.g. the whole point of SKIP_INDEX_CLEANUP is that the cache doesn't
>>>>>>>>> know when it is safe to skip the delete statement, and it relies on
>>>>>>>>> the application making a (possibly wrong) choice.
>>>>>>>>>
>>>>>>>>> IGNORE_RETURN_VALUES should be safe to use, and we actually recommend
>>>>>>>>> that applications use it right now. If query or listeners need the
>>>>>>>>> previous value, then we should load it internally, but hide it from
>>>>>>>>> the user.
>>>>>>>>>
>>>>>>>>> But removing it opens another discussion: should we replace it in the
>>>>>>>>> public API with a new method AdvancedCache.ignoreReturnValues(), or
>>>>>>>>> should we make it the default and add a method
>>>>>>>>> AdvancedCache.forceReturnPreviousValues()?
>>>>>>>> Please don't derail the thread.
>>>>>>>>
>>>>>>> I don't think I'm derailing the thread: IGNORE_PREVIOUS_VALUES also
>>>>>>> breaks the previous value for listeners, even if the QueryInterceptor
>>>>>>> removes it from write commands. And it is public (+recommended) API,
>>>>>>> in fact most if not all of our performance tests use it.
>>>>>> That's just a flawed implementation. IPV is documented to be a 'safe'
>>>>>> flag that should affect mostly primary -> origin replication, all the
>>>>>> other is implementation. And we can fix that. Users should *not* expect
>>>>>> that it e.g. skips loading from a cache store. We have already removed
>>>>>> the modes that would be broken-by-design.
>>>>>>
>>>>> I think you're confusing IGNORE_RETURN_VALUES with SKIP_REMOTE_LOOKUP
>>>>> here. The IVR javadoc doesn't say anything about remote lookups, only
>>>>> SRL does.
>>>> No, I am not; While IRV does not mention the replication, it's said to
>>>> be 'safe'. So omitting the primary -> origin replication is basically
>>>> all it can do when listeners are in place. You're right that I have
>>>> missed the second part in SRL talking about put()s; I took it as a flag
>>>> prohibiting any remote lookup (as the RPC operation in its whole) any
>>>> time the remote value is needed. Yes, the second part seems equal to my
>>>> understanding of IRV.
>>>>
>>>>> And I agree that the current status is far from ideal, but there is
>>>>> one more valid alternative: we can decide that the previous value is
>>>>> only reliable in clustered listeners, and local listeners don't always
>>>>> have it. Document that, make sure continuous query uses clustered
>>>>> listeners, and we're done :)
>>>> Unreliable return values are worse than none; I would rather remove them
>>>> if we can't guarantee that these are right. Though, clustered listeners
>>>> are based on regular listeners, so you'd need some means to make them
>>>> reliable.
>>> We could change the clustered listeners so that they're not based on
>>> the regular listeners... I've been pestering Will about this ever
>>> since the clustered listeners landed!
>>>
>>> But I should have been clearer: I didn't mean that the listeners on
>>> the backups should receive the previous value whenever we feel like
>>> it, I meant we should document and enforce that the previous value is
>>> only included in the event for listeners on the primary owner.
>>>>>> On the other hand, write-only commands are not about *returning* the
>>>>>> value but about (not) *reading* it, therefore (in my eyes) user could
>>>>>> make that assumption and would like to enforce it this way. Even some
>>>>>> docs explaining PersistenceMode.SKIP suggest that.
>>>>>>
>>>>> To me the purpose the same, there is no difference between returning
>>>>> the previous value to the application or providing the previous value
>>>>> via EntryView.
>>>> There is a difference between what's provided locally and what's send
>>>> over the network.
>>>>
>>>>> Applying this logic to the JCache API, it would mean
>>>>> put() should never read the previous value, because some users could
>>>>> assume that only getAndPut() reads it.
>>>> OK, this is a valid point.
>>>>
>>>>> In the old times we didn't have IGNORE_RETURN_VALUES, only
>>>>> SKIP_REMOTE_LOOKUP+SKIP_CACHE_LOAD, and they would sometimes be
>>>>> ignored (e.g. if the write was conditional). I think that's what
>>>>> Galder had in mind when he wrote the PersistenceMode api note, not the
>>>>> current behaviour of SKIP_CACHE_LOAD. I'll let Galder clarify this
>>>>> himself, but I'll be very disappointed if he says he designed the
>>>>> write-only operations so that they'll never work with query.
>>>>>
>>>>>
>>>>>> I don't want to talk about flags, because I see all flags but IPV as
>>>>>> 'effectively internal'. Let's discuss it more high-level. Some API
>>>>>> exposes non-reading operation - we can see that under some circumstances
>>>>>> this is not possible so we have options to 1) break stuff 2) break API
>>>>>> assumptions 3) sometimes break API assumptions 4) remove such API (to
>>>>>> not allow the user to make such assumptions). There's also an option 5)
>>>>>> to fail the operation if the API assumption would be broken. Though, I
>>>>>> don't fancy getting exception from a WriteOnlyMap.eval just because
>>>>>> someone has registered a listener.
>>>>>>
>>>>> I disagree with the premise: there's no good reason for the user to
>>>>> assume that write-only commands are *guaranteed* to never load the
>>>>> previous value from a store. We just need to add a clarification to
>>>>> the write-only operations' javadoc, no need to break anything.
>>>> OK then, though it diminishes the value of write-only commands a lot.
>>>>
>>>>>>> For that matter, ClusteredCacheLoaderInterceptor also doesn't load the
>>>>>>> previous value on backup owners for most write commands
>>>>>>> (LoadType.PRIMARY), we'd need to change that as well.
>>>>>> Yes, all commands will have to load current value on all owners.
>>>>>>
>>>>>>>>>> On 06/27/2017 01:28 PM, Sanne Grinovero wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 27 Jun 2017 10:13, "Radim Vansa" <rvansa at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I am working on entry version history (again). In Como we've discussed
>>>>>>>>>> that previous values are needed for (continuous) query and reliable
>>>>>>>>>> listeners,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Index based queries also require the previous value on a write - unless we
>>>>>>>>>> can get "strongly typed caches" giving guarantees about the class to
>>>>>>>>>> represent the content of a cache to be unique.
>>>>>>>>>>
>>>>>>>>>> Essentially we only need to know the type of the previous object. It might
>>>>>>>>>> be worth having a way to load the type metadata if the previous value only.
>>>>>>>>>>
>>>>>>>>>> so I wonder what should we do with functional write-only
>>>>>>>>>> commands. These are different to commands with flags, because flags
>>>>>>>>>> (other than ignore return value) are expected to break something.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sorry I hope to not derail the thread but let's remind that we hope to
>>>>>>>>>> evolve beyond "flags are expected to break stuff" ; we never got to it but
>>>>>>>>>> search the mailing list.
>>>>>>>>>>
>>>>>>>>>> Since flags are exposed to the user I would rather they're not allowed to
>>>>>>>>>> break things.
>>>>>>>>>> Could they be treated as hints? Ignore the flag (and warn?) if the used
>>>>>>>>>> configuration/integrations veto them.
>>>>>>>>>>
>>>>>>>>>> Alternatively, let's remove them from API. Remember "The Jokre" POC was
>>>>>>>>>> intentionally designed to explore pushing the limits on performance w/o end
>>>>>>>>>> users having to solve puzzles, such as learning details about these flags
>>>>>>>>>> and their possible side effects.
>>>>>>>>>>
>>>>>>>>>> So assuming they become either "safe" or internal, maybe you can take
>>>>>>>>>> advantage of them?
>>>>>>>>>>
>>>>>>>>>> I see
>>>>>>>>>> the available options as:
>>>>>>>>>>
>>>>>>>>>> 1) run write-only commands 'optimized', ignoring any querying and such
>>>>>>>>>> (warn user that he will break it)
>>>>>>>>>>
>>>>>>>>>> 2) run write-only without any optimization, rendering them useless
>>>>>>>>>>
>>>>>>>>>> 3) detect when querying is set up (ignoring listeners and maybe other
>>>>>>>>>> stuff that could get broken)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Might be useful for making a POC work, but I believe query will be very
>>>>>>>>>> likely to be often enabled.
>>>>>>>>>> Having an either / or switch for different features in Infinispan will make
>>>>>>>>>> it harder to use and understand, so I'd rather see work on the right design
>>>>>>>>>> as taking temporary shortcuts risks baking into stone features which we
>>>>>>>>>> later struggle to fix or maintain.
>>>>>>>>>>
>>>>>>>>> I vote for this option.
>>>>>>>>>
>>>>>>>>> Query, listeners, and other components that need the previous value
>>>>>>>>> should not just assume that the application knows better, they should
>>>>>>>>> be able to change how operations works based on their needs. Of
>>>>>>>>> course, the reverse is also true: if the application uses write-only
>>>>>>>>> commands (or IGNORE_RETURN_VALUES) for performance reasons, it should
>>>>>>>>> be possible for the user to detect why the previous values are still
>>>>>>>>> loaded.
>>>>>>>> If it were just query (static configuration), I would be okay with this
>>>>>>>> idea. But as per listeners - besides tainting the design (event source
>>>>>>>> should not check if there's a listener) you'd need to check *before*
>>>>>>> The source wouldn't check for listeners explicitly, the notifier would
>>>>>>> have an isPreviousValueNeeded() method and precompute that before a
>>>>>>> listener is added or after a listener is removed. I was am assuming
>>>>>>> some listeners will not need the previous value, e.g. the listeners
>>>>>>> installed by streams.
>>>>>> You can cover your warts with a make-up but you'll still have warts :)
>>>>> Cutting them off doesn't necessarily work, either :)
>>>> Yep, some people tend to fix w/ hacks instead of designing :)
>>>>
>>>>>>>> (DistributionI, CacheLoaderI) you have to call notify (cmd.perform,
>>>>>>>> EWI). So this is a space for race conditions or weird handling (if
>>>>>>>> there's a listener when I am about to call notify and my flags are not
>>>>>>>> cleared, skip the notification and pretend that this code was invoked
>>>>>>>> before the listener was registered...). Or do you have another solution
>>>>>>>> in mind (config option to disable listeners && all features using those?).
>>>>>>>>
>>>>>>> I was definitely going for the weird handling...
>>>>>>>
>>>>>>> My plan was to set a HAS_PREVIOUS_VALUE flag on the context entry when
>>>>>>> it's loaded, and check that before invoking a listener that needs the
>>>>>>> previous value. It is missing one edge case: if one thread starts a
>>>>>>> write operation, then another thread installs a listener that requires
>>>>>>> the previous value and iterates over the cache, the second thread may
>>>>>>> not see the value written by the first thread.
>>>>>> If the operations overlap, you could pretend that the write has finished
>>>>>> before the listener was invoked and simply not notify the listener. If I
>>>>>> am missing it please write it down in code. But handling this in any way
>>>>>> is still clumsy.
>>>>> I hope pseudo-code is fine...
>>>>>
>>>>> 1. cache.put(k, v1) starts, doesn't load the previous value v0 in the context
>>>>> 2. cache.addListener(l) runs, doesn't block
>>>>> 3. cache.entrySet().forEach() runs, finds k->v0
>>>>> 4. cache.put(k, v1) commits k->v1, should notify the listener but
>>>>> doesn't have the previous value
>>>>> 5. cache.put(k, v0) returns, but the code that installed the listener
>>>>> thinks the value of k is still v0
>>>> Oh OK, I should have drawn that myself when considering the scenario.
>>>> You're right, here we'll have to retry.
>>>>
>>>> All in all, I think this discussion is done. We'll tell users to stick
>>>> their flags where the sun doesn't shine and remove any inconvenient
>>>> ones. Should we issue a warning any time we're removing the flag?
>>>>
>>> If you mean that we should remove the flags from the public API, I
>>> agree. If you mean we should just ignore them, then no, because most
>>> of the flags were added for internal components that really need their
>>> semantics.
>> We can't remove them from public API before Infinspan 10, and I think
>> that it will be a quite an unpopular step even after that. But until 10,
>> I think that the common agreement was to not break query, that is ignore
>> the flags. And make write-only reading.
>>
> So SKIP_INDEXING should not skip indexing because it can break query??
>
>> R.
>>
>>> Dan
>>>
>>>
>>>> Radim
>>>>
>>>>>>> So now I'm thinking we should retry the write commands when
>>>>>>> isPreviousValueNeeded() changes... Not very appealing, but I think the
>>>>>>> performance difference is worth it.
>>>>>>>
>>>>>>>> R.
>>>>>>>>
>>>>>>>>>> 4) remove write-only commands completely (and probably functional
>>>>>>>>>> listeners as well because these will lose their purpose)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> +1 to remove "unconditional writes", at least an entry version check should
>>>>>>>>>> be applied.
>>>>>>>>>> I believe we had already pointed out this would eventually happen, pretty
>>>>>>>>>> much for the reasons you're hitting now.
>>>>>>>>>>
>>>>>>>>> IMO version checks should be done internally, we shouldn't force the
>>>>>>>>> users of the functional API to deal with versions themselves because
>>>>>>>>> we know how hard making write skew checks work is for us :)
>>>>>>>>>
>>>>>>>>> And I wouldn't go as far as to remove the functional listeners,
>>>>>>>>> instead I would change them so that read-write listeners are invoked
>>>>>>>>> on write-only operations and they force the loading of the previous
>>>>>>>>> value. I would also add a way for the regular listeners to say whether
>>>>>>>>> they need the previous value or not.
>>>>>>>>>
>>>>>>>>>> Right now I am inclined towards 4). There could be some internal use
>>>>>>>>>> (e.g. multimaps) that could use 1) which is ran without a fancy setup,
>>>>>>>>>> though, but it's asking for trouble.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I agree!
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> WDYT?
>>>>>>>>>>
>>>>>>>>>> Radim
>>>>>>>>>>
>>>>>>>>> Cheers
>>>>>>>>> Dan
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> --
>>>>>> Radim Vansa <rvansa at redhat.com>
>>>>>> JBoss Performance Team
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> --
>>>> Radim Vansa <rvansa at redhat.com>
>>>> JBoss Performance Team
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From tsegismont at gmail.com  Fri Jun 30 12:22:28 2017
From: tsegismont at gmail.com (Thomas SEGISMONT)
Date: Fri, 30 Jun 2017 18:22:28 +0200
Subject: [infinispan-dev] KUBE_PING changes
In-Reply-To: <CACiEr_ScR-ANveF5v1dNBfvvd4P_=Dax77SGYnhEH75cMoY4VA@mail.gmail.com>
References: <CAA9R9O7U3w-if7XyLc3bRPM_GVyzr9ywChm+222EptrNSbA8Tg@mail.gmail.com>
	<6b3dc945-449b-1c9b-1384-4a4cdde83eca@mailbox.org>
	<CAA9R9O6Vhrd2No5dQd6NawuuAx5VbFdkKsFOvNs5aT0qp9ftRQ@mail.gmail.com>
	<CACiEr_ScR-ANveF5v1dNBfvvd4P_=Dax77SGYnhEH75cMoY4VA@mail.gmail.com>
Message-ID: <CACiEr_T4Bw4KexS1-ax3x-wdKUjDsX+uma4nbz-16iDBfB1WEQ@mail.gmail.com>

Also, it seems infinispan-cloud 9.0.3.Final depends on JGroups 0.9.1.

Do you plan to release another 9.0.x version which depends on 1.0.0.Beta1
or later? If so, is there a target date?

2017-06-30 11:40 GMT+02:00 Thomas SEGISMONT <tsegismont at gmail.com>:

> Hi everyone,
>
> Thank you for this great work, the dependency diet and the extra port
> removal are both very useful. The extra port removal is key to enable
> Vert.x clustering in Openshift S2I environments.
>
> I tried the new KUBE_PING (beta1) with vertx-infinispan and it worked
> fine. I have a few questions though.
>
> I couldn't configure it with env variables. Before you ask, yes I noticed
> the name changes ;-) I only had a quick look at JGroups config code but it
> seems it only resolves system properties. Did it work for you because you
> tried with an Infinispan server?
>
> Since I couldn't configure it externally I had to create a custom JGroups
> file. Usually, we recommend [1] Vert.x users to add the infinispan-cloud
> dependency and a system property:
> -Dvertx.jgroups.config=default-configs/default-jgroups-kubernetes.xml
>
> My custom JGroups file is a just a copy of default-configs/default-jgroups-kubernetes.xml
> in which I added the masterHost and namespace properties.
>
> Is it still recommended to use the default-configs/default-jgroups-kubernetes.xml
> stack ? Or is any change planned after the KUBE_PING changes?
> I wouldn't expect a protocol implementation change to impact a stack
> configuration but they say there are no stupid questions :)
>
> Thank you,
> Thomas
>
>
> [1] http://vertx.io/docs/vertx-infinispan/java/#_
> configuring_for_openshift_3
>
>
> 2017-06-15 8:21 GMT+02:00 Sebastian Laskawiec <slaskawi at redhat.com>:
>
>> Yep, no problems found!!!
>>
>> I had also impression that the new implementation is "faster". Though I
>> haven't measured it... it just my impression.
>>
>> Awesome work Bela!
>>
>> On Thu, Jun 15, 2017 at 7:42 AM Bela Ban <belaban at mailbox.org> wrote:
>>
>>> Thanks, Sebastian!
>>>
>>> I assume testing on GKE and minikube/openshift was successful?
>>>
>>>
>>> On 14/06/17 13:15, Sebastian Laskawiec wrote:
>>> > Hey guys,
>>> >
>>> > Just a heads up, I've just created a PR that upgrades KUBE_PING to
>>> > 1.0.0.Beta1 [1]. As you probably seen in [2], 1.0.0.Beta1 was
>>> completely
>>> > rewritten and might behave slightly differently.
>>> >
>>> > Here is a summary of changes:
>>> >
>>> >   * The latest KUBE_PING doesn't require embedded HTTP server for
>>> >     discovery. Thus it is no longer required to expose port 8888 in
>>> Pods.
>>> >   * The number of dependencies has been decreased. Currently we only
>>> >     require JGroups and OAuth library.
>>> >   * The new KUBE_PING works only with JGroups 4. There will be no
>>> >     JGroups 3 support.
>>> >   * Some of the environmental variables were shortened and we removed
>>> >     `OPENSHIFT` prefix. So if you use `OPENSHIFT_KUBE_PING_NAMESPACE`,
>>> >     you will need to change it to `KUBERNETES_NAMESPACE`. Please refer
>>> >     to [3] for more information.
>>> >
>>> > I also switched default branch in Kubernetes Ping repository to master
>>> [4].
>>> >
>>> > Thanks,
>>> > Sebastian
>>> >
>>> > [1] https://github.com/infinispan/infinispan/pull/5201
>>> > [2] http://belaban.blogspot.ch/2017/05/running-infinispan-cluste
>>> r-with.html
>>> > [3] https://github.com/jgroups-extras/jgroups-kubernetes/blob/
>>> master/README.adoc
>>> > [4] https://github.com/jgroups-extras/jgroups-kubernetes
>>> > --
>>> >
>>> > SEBASTIAN ?ASKAWIEC
>>> >
>>> > INFINISPAN DEVELOPER
>>> >
>>> > Red Hat EMEA <https://www.redhat.com/>
>>> >
>>> > <https://red.ht/sig>
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > infinispan-dev mailing list
>>> > infinispan-dev at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> >
>>>
>>> --
>>> Bela Ban | http://www.jgroups.org
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>>
>> SEBASTIAN ?ASKAWIEC
>>
>> INFINISPAN DEVELOPER
>>
>> Red Hat EMEA <https://www.redhat.com/>
>> <https://red.ht/sig>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170630/1316203b/attachment.html