From ttarrant at redhat.com Mon Apr 3 08:31:28 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Mon, 3 Apr 2017 14:31:28 +0200 Subject: [infinispan-dev] My Weekly report Message-ID: I won't be able to attend today's meeting, so here's my summary / plan: - ISPN-7683 unregister the topology cache on Hot Rod server stop - ISPN-7678 prevent endpoint restart on default cache reconfiguration - still some snags due to the Hot Rod server wanting to prestart caches, which I want to remove - Did the 9.0 release - Updated the embedded tutorial - Made some fixes to the website (roadmap, features, etc) - Some security work - Hot Rod authn by default (Digest) - generate the certificate chains on the fly for the testsuite to avoid cert rot and issues with alternate JDKs - Initial design for fine-grained security, will share soon - Resumed work on the threadlocal-less secure cache for better server performance - I also need to introduce "identity-shipping", so that running streams in a secure context will apply the same identity across all nodes. -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From slaskawi at redhat.com Mon Apr 3 08:52:43 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 03 Apr 2017 12:52:43 +0000 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: <06b88c80-433f-3745-2112-1424bcd9c342@redhat.com> Message-ID: That's actually a very good point but it deserves separate discussion I think. The point is that we get an initialized SSLEngine from WF. Then we simply build Netty's JdkSslContext [1] (as you probably know, Netty uses its own SSLContext; this JdkSslContext class acts as a bridge between Netty (it implements Netty's SslContext) and JDK (it can be created from JDK's SSLContext) world). So if we want to depend on WF/Elytron-based SSL initialization, we need to either stick to JDK's SSLContext or implement our own JDK's SSLContext/Netty's OpenSSLContext [2] bridge. I created a JIRA [3] to implement this. [1] https://github.com/infinispan/infinispan/blob/fd3061a684c13aa7c22d215f43910735bf7ff4a6/server/router/src/main/java/org/infinispan/server/router/router/impl/hotrod/handlers/util/SslUtils.java#L35 [2] https://github.com/netty/netty/blob/a2304287a170dc14031928d6d2a3374705305839/handler/src/main/java/io/netty/handler/ssl/OpenSslContext.java [3] https://issues.jboss.org/browse/ISPN-7694 On Fri, Mar 31, 2017 at 6:02 PM Tristan Tarrant wrote: You want to use OpenSSL with Netty: http://netty.io/wiki/requirements-for-4.x.html#wiki-h4-4 Tristan On 31/03/2017 15:55, Sebastian Laskawiec wrote: > Unfortunately TLS still slows down stuff (a lot). When I was doing tests > for the multi-tenancy router (which is based on TLS/SNI), my average > results were like this: > > Use-caseTypeAvgError > initConnectionAndPerform10KPutsSingleServerNoSsl1034.81714.424 > initConnectionAndPerform10KPutsSingleServerWithSsl1567.55324.872 > initConnectionAndPerform10KPutsTwoServersWithSslSni1563.22934.05 > initConnectionOnlySingleServerNoSsl*3.389*0.198 > initConnectionOnlySingleServerWithSsl*14.086*0.794 > initConnectionOnlyTwoServersWithSslSni*14.722*0.684 > perform10KPutsSingleServerNoSsl*4.602*0.585 > perform10KPutsSingleServerWithSsl*16.583*0.198 > perform10KPutsTwoServersWithSslSni*17.02*0.794 > > This is nothing new, but initializing Hot Rod connection took was ~4 > times slower and putting 10K random strings (UUIDs) was also ~4 times > slower. But what's worth to mention, there is no significant difference > between TLS and TLS+SNI. > > As far as I know, it is possible to install specialized hardware to deal > with encryption in data centers. It is called SSL Acceleration [1]. > However I'm not aware of any special processor instructions that can > help you with that. But the implementations are getting better and > better, so who knows... > > But getting back to the original question, I think the problem we are > trying to solve (correct me if I'm wrong) is to prevent unauthorized > folks to put their hands on a victims data (either pushing something > malicious/corrupted to the cache or obtaining something from the cache). > Another problem is transmission security - encryption. If we want our > new devs to be secured out of the box, I think we should do both - use > TLS (without trusting all certificated) and authentication. This makes > Infinispan harder to use of course. So the other extremum is to turn > both things off. > > I voted for the latter, making Infinispan super easy to use. But you > guys convinced me that we should care about the security in this case > too, so I would use PLAIN authentication + TLS. I would also love to see > one magic switch, for example `./bin/standalone.sh --dev-mode`, which > would turn all security off. > > Thanks, > Sebastian > > [1] https://en.wikipedia.org/wiki/SSL_acceleration > > > On Thu, Mar 30, 2017 at 9:22 PM Dan Berindei > wrote: > > I agree with Radim, PLAIN authentication without encryption makes it > too easy to sniff the password from another machine. > > I have no idea how expensive SSL encryption is in WildFly, but I think > all recent processors have specialized instructions for helping with > encryption, so it may not be that bad. > > Even with encryption, if the client trusts all certs, it may be > possible for an attacker to insert itself in the middle and decode > everything -- depending on network topology and what kind of access > the attacker already has. I think it only makes sense to trust all > certs if we also implement something like HPKP [1], to make it more > like ssh. > > [1]: https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning > > Cheers > Dan > > > > On Thu, Mar 30, 2017 at 7:07 PM, Wolf Fink > wrote: > > +1 to make the default secure. > > > > -1 SSL by default as it makes it slower and I think not most will > use it > > > > -1 easy trust all certs, That sounds to me we close one door and > make it > > possible to open another one > > > > > > What if we add an example configuration unsecured which can be > simple copied > > for examples and to start. > > > > > > On Thu, Mar 30, 2017 at 5:31 PM, Dennis Reed > wrote: > >> > >> +1 to authentication and encryption by default. > >> This is 2017, that's how *everything* should be configured. > >> > >> -1 to making it easy to trust all certs. That negates the point of > >> using encryption in the first place and should really never be done. > >> > >> If it's too hard to configure the correct way that we think it would > >> turn users away, that's a usability problem that needs to be fixed. > >> > >> -Dennis > >> > >> > >> On 03/30/2017 09:29 AM, Tristan Tarrant wrote: > >> > While the "unsecure" over loopback is quite tempting, I would > prefer to > >> > have homogeneous behaviour with the possibility to disable > security > >> > altogether for quick demos. > >> > Otherwise a developer would need to code differently for the > local use > >> > case than for the remote one, causing more confusion. > >> > > >> > Tristan > >> > > >> > On 30/03/2017 14:54, Sebastian Laskawiec wrote: > >> >> I agree the security out of the box is good. But at the same > time we > >> >> don't want to make Infinispan harder to use for new > developers. Out of > >> >> the box configuration should be "good enough" to start hacking. > >> >> > >> >> I would propose to make all the endpoints unprotected (with > >> >> authentication disabled) on localhost/loopback and protected when > >> >> calling from the outside world. > >> >> > >> >> On Thu, Mar 30, 2017 at 2:39 PM Tristan Tarrant > > >> >> >> wrote: > >> >> > >> >> Dear all, > >> >> > >> >> after a mini chat on IRC, I wanted to bring this to > everybody's > >> >> attention. > >> >> > >> >> We should make the Hot Rod endpoint require > authentication in the > >> >> out-of-the-box configuration. > >> >> The proposal is to enable the PLAIN (or, preferably, > DIGEST) SASL > >> >> mechanism against the ApplicationRealm and require users > to run the > >> >> add-user script. > >> >> This would achieve two goals: > >> >> - secure out-of-the-box configuration, which is always a > good idea > >> >> - access to the "protected" schema and script caches which is > >> >> prevented > >> >> when not on loopback on non-authenticated endpoints. > >> >> > >> >> Tristan > >> >> -- > >> >> Tristan Tarrant > >> >> Infinispan Lead > >> >> JBoss, a division of Red Hat > >> >> _______________________________________________ > >> >> infinispan-dev mailing list > >> >> infinispan-dev at lists.jboss.org > > >> >> > > >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> >> > >> >> > >> >> > >> >> _______________________________________________ > >> >> infinispan-dev mailing list > >> >> infinispan-dev at lists.jboss.org > > >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> >> > >> > > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat _______________________________________________ infinispan-dev mailing list infinispan-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170403/b35a8d9d/attachment-0001.html From gustavo at infinispan.org Mon Apr 3 09:26:23 2017 From: gustavo at infinispan.org (Gustavo Fernandes) Date: Mon, 3 Apr 2017 14:26:23 +0100 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: <06b88c80-433f-3745-2112-1424bcd9c342@redhat.com> Message-ID: On Mon, Apr 3, 2017 at 1:52 PM, Sebastian Laskawiec wrote: > That's actually a very good point but it deserves separate discussion I > think. > > The point is that we get an initialized SSLEngine from WF. Then we simply > build Netty's JdkSslContext [1] (as you probably know, Netty uses its own > SSLContext; this JdkSslContext class acts as a bridge between Netty (it > implements Netty's SslContext) and JDK (it can be created from JDK's > SSLContext) world). > > So if we want to depend on WF/Elytron-based SSL initialization, we need to > either stick to JDK's SSLContext or implement our own JDK's > SSLContext/Netty's OpenSSLContext [2] bridge. I created a JIRA [3] to > implement this. > AFAICT, Wildfly 11 should export a "org.wildfly.security.ssl-context" capability to allow plugging in custom engines. Gustavo > > [1] https://github.com/infinispan/infinispan/blob/fd3061a684 > c13aa7c22d215f43910735bf7ff4a6/server/router/src/main/java/ > org/infinispan/server/router/router/impl/hotrod/handlers/ > util/SslUtils.java#L35 > [2] https://github.com/netty/netty/blob/a2304287a170dc140319 > 28d6d2a3374705305839/handler/src/main/java/io/netty/ > handler/ssl/OpenSslContext.java > [3] https://issues.jboss.org/browse/ISPN-7694 > > On Fri, Mar 31, 2017 at 6:02 PM Tristan Tarrant > wrote: > > You want to use OpenSSL with Netty: > > http://netty.io/wiki/requirements-for-4.x.html#wiki-h4-4 > > Tristan > > On 31/03/2017 15:55, Sebastian Laskawiec wrote: > > Unfortunately TLS still slows down stuff (a lot). When I was doing tests > > for the multi-tenancy router (which is based on TLS/SNI), my average > > results were like this: > > > > Use-caseTypeAvgError > > initConnectionAndPerform10KPutsSingleServerNoSsl1034.81714.424 > > initConnectionAndPerform10KPutsSingleServerWithSsl1567.55324.872 > > initConnectionAndPerform10KPutsTwoServersWithSslSni1563.22934.05 > > initConnectionOnlySingleServerNoSsl*3.389*0.198 > > initConnectionOnlySingleServerWithSsl*14.086*0.794 > > initConnectionOnlyTwoServersWithSslSni*14.722*0.684 > > perform10KPutsSingleServerNoSsl*4.602*0.585 > > perform10KPutsSingleServerWithSsl*16.583*0.198 > > perform10KPutsTwoServersWithSslSni*17.02*0.794 > > > > This is nothing new, but initializing Hot Rod connection took was ~4 > > times slower and putting 10K random strings (UUIDs) was also ~4 times > > slower. But what's worth to mention, there is no significant difference > > between TLS and TLS+SNI. > > > > As far as I know, it is possible to install specialized hardware to deal > > with encryption in data centers. It is called SSL Acceleration [1]. > > However I'm not aware of any special processor instructions that can > > help you with that. But the implementations are getting better and > > better, so who knows... > > > > But getting back to the original question, I think the problem we are > > trying to solve (correct me if I'm wrong) is to prevent unauthorized > > folks to put their hands on a victims data (either pushing something > > malicious/corrupted to the cache or obtaining something from the cache). > > Another problem is transmission security - encryption. If we want our > > new devs to be secured out of the box, I think we should do both - use > > TLS (without trusting all certificated) and authentication. This makes > > Infinispan harder to use of course. So the other extremum is to turn > > both things off. > > > > I voted for the latter, making Infinispan super easy to use. But you > > guys convinced me that we should care about the security in this case > > too, so I would use PLAIN authentication + TLS. I would also love to see > > one magic switch, for example `./bin/standalone.sh --dev-mode`, which > > would turn all security off. > > > > Thanks, > > Sebastian > > > > [1] https://en.wikipedia.org/wiki/SSL_acceleration > > > > > > On Thu, Mar 30, 2017 at 9:22 PM Dan Berindei > > wrote: > > > > I agree with Radim, PLAIN authentication without encryption makes it > > too easy to sniff the password from another machine. > > > > I have no idea how expensive SSL encryption is in WildFly, but I > think > > all recent processors have specialized instructions for helping with > > encryption, so it may not be that bad. > > > > Even with encryption, if the client trusts all certs, it may be > > possible for an attacker to insert itself in the middle and decode > > everything -- depending on network topology and what kind of access > > the attacker already has. I think it only makes sense to trust all > > certs if we also implement something like HPKP [1], to make it more > > like ssh. > > > > [1]: https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning > > > > Cheers > > Dan > > > > > > > > On Thu, Mar 30, 2017 at 7:07 PM, Wolf Fink > > wrote: > > > +1 to make the default secure. > > > > > > -1 SSL by default as it makes it slower and I think not most will > > use it > > > > > > -1 easy trust all certs, That sounds to me we close one door and > > make it > > > possible to open another one > > > > > > > > > What if we add an example configuration unsecured which can be > > simple copied > > > for examples and to start. > > > > > > > > > On Thu, Mar 30, 2017 at 5:31 PM, Dennis Reed > > wrote: > > >> > > >> +1 to authentication and encryption by default. > > >> This is 2017, that's how *everything* should be configured. > > >> > > >> -1 to making it easy to trust all certs. That negates the point > of > > >> using encryption in the first place and should really never be > done. > > >> > > >> If it's too hard to configure the correct way that we think it > would > > >> turn users away, that's a usability problem that needs to be > fixed. > > >> > > >> -Dennis > > >> > > >> > > >> On 03/30/2017 09:29 AM, Tristan Tarrant wrote: > > >> > While the "unsecure" over loopback is quite tempting, I would > > prefer to > > >> > have homogeneous behaviour with the possibility to disable > > security > > >> > altogether for quick demos. > > >> > Otherwise a developer would need to code differently for the > > local use > > >> > case than for the remote one, causing more confusion. > > >> > > > >> > Tristan > > >> > > > >> > On 30/03/2017 14:54, Sebastian Laskawiec wrote: > > >> >> I agree the security out of the box is good. But at the same > > time we > > >> >> don't want to make Infinispan harder to use for new > > developers. Out of > > >> >> the box configuration should be "good enough" to start > hacking. > > >> >> > > >> >> I would propose to make all the endpoints unprotected (with > > >> >> authentication disabled) on localhost/loopback and protected > when > > >> >> calling from the outside world. > > >> >> > > >> >> On Thu, Mar 30, 2017 at 2:39 PM Tristan Tarrant > > > > >> >> >> > wrote: > > >> >> > > >> >> Dear all, > > >> >> > > >> >> after a mini chat on IRC, I wanted to bring this to > > everybody's > > >> >> attention. > > >> >> > > >> >> We should make the Hot Rod endpoint require > > authentication in the > > >> >> out-of-the-box configuration. > > >> >> The proposal is to enable the PLAIN (or, preferably, > > DIGEST) SASL > > >> >> mechanism against the ApplicationRealm and require users > > to run the > > >> >> add-user script. > > >> >> This would achieve two goals: > > >> >> - secure out-of-the-box configuration, which is always a > > good idea > > >> >> - access to the "protected" schema and script caches > which is > > >> >> prevented > > >> >> when not on loopback on non-authenticated endpoints. > > >> >> > > >> >> Tristan > > >> >> -- > > >> >> Tristan Tarrant > > >> >> Infinispan Lead > > >> >> JBoss, a division of Red Hat > > >> >> _______________________________________________ > > >> >> infinispan-dev mailing list > > >> >> infinispan-dev at lists.jboss.org > > > > >> >> > > > > >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > >> >> > > >> >> > > >> >> > > >> >> _______________________________________________ > > >> >> infinispan-dev mailing list > > >> >> infinispan-dev at lists.jboss.org > > > > >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > >> >> > > >> > > > >> _______________________________________________ > > >> infinispan-dev mailing list > > >> infinispan-dev at lists.jboss.org > > > > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > > > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org > > > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org boss.org> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170403/af6abf73/attachment-0001.html From galder at redhat.com Mon Apr 3 11:44:55 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Mon, 3 Apr 2017 17:44:55 +0200 Subject: [infinispan-dev] Proto file for indexed and non-indexed use case? Message-ID: Hi Adrian, I had a question regarding proto files. I have a single domain of objects that I want to use for two different use cases. In the first use case, I want the proto files to be indexed so I define the comments and related @Indexed/@Field...etc annotations. In the second use case, I'm merely using proto files as way to achieve compatibility mode, and I don't want any indexing to be done at all (cache is distributed with only compatibility and protostream marshaller enabled). Do I need a separate .proto file for this second use case where I remove the commented sections that enable indexing? Or can I use the one for the first use case? I really want to avoid any indexing happening in the second use case since it'd slow down things for no reason. Cheers, -- Galder Zamarre?o Infinispan, Red Hat From sanne at infinispan.org Mon Apr 3 12:58:37 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 3 Apr 2017 17:58:37 +0100 Subject: [infinispan-dev] Proto file for indexed and non-indexed use case? In-Reply-To: References: Message-ID: Hi Galder, did you consider using a non-indexed cache for the second case? Thanks, Sanne On 3 April 2017 at 16:44, Galder Zamarre?o wrote: > Hi Adrian, > > I had a question regarding proto files. I have a single domain of objects that I want to use for two different use cases. > > In the first use case, I want the proto files to be indexed so I define the comments and related @Indexed/@Field...etc annotations. > > In the second use case, I'm merely using proto files as way to achieve compatibility mode, and I don't want any indexing to be done at all (cache is distributed with only compatibility and protostream marshaller enabled). > > Do I need a separate .proto file for this second use case where I remove the commented sections that enable indexing? Or can I use the one for the first use case? I really want to avoid any indexing happening in the second use case since it'd slow down things for no reason. > > Cheers, > -- > Galder Zamarre?o > Infinispan, Red Hat > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From anistor at redhat.com Mon Apr 3 14:15:16 2017 From: anistor at redhat.com (Adrian Nistor) Date: Mon, 3 Apr 2017 21:15:16 +0300 Subject: [infinispan-dev] Proto file for indexed and non-indexed use case? In-Reply-To: References: Message-ID: <8a37ecf4-10a7-7ba8-08ac-ca4ae4089bfb@redhat.com> Yup, that's how I'd do it. On 04/03/2017 07:58 PM, Sanne Grinovero wrote: > Hi Galder, > > did you consider using a non-indexed cache for the second case? > > Thanks, > Sanne > > > On 3 April 2017 at 16:44, Galder Zamarre?o wrote: >> Hi Adrian, >> >> I had a question regarding proto files. I have a single domain of objects that I want to use for two different use cases. >> >> In the first use case, I want the proto files to be indexed so I define the comments and related @Indexed/@Field...etc annotations. >> >> In the second use case, I'm merely using proto files as way to achieve compatibility mode, and I don't want any indexing to be done at all (cache is distributed with only compatibility and protostream marshaller enabled). >> >> Do I need a separate .proto file for this second use case where I remove the commented sections that enable indexing? Or can I use the one for the first use case? I really want to avoid any indexing happening in the second use case since it'd slow down things for no reason. >> >> Cheers, >> -- >> Galder Zamarre?o >> Infinispan, Red Hat >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev From galder at redhat.com Tue Apr 4 08:48:30 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Tue, 4 Apr 2017 14:48:30 +0200 Subject: [infinispan-dev] Proto file for indexed and non-indexed use case? In-Reply-To: References: Message-ID: The cache for the second use case is already non-indexed. Is that enough to make sure the annotations are ignored? Cheers, -- Galder Zamarre?o Infinispan, Red Hat > On 3 Apr 2017, at 18:58, Sanne Grinovero wrote: > > Hi Galder, > > did you consider using a non-indexed cache for the second case? > > Thanks, > Sanne > > > On 3 April 2017 at 16:44, Galder Zamarre?o wrote: >> Hi Adrian, >> >> I had a question regarding proto files. I have a single domain of objects that I want to use for two different use cases. >> >> In the first use case, I want the proto files to be indexed so I define the comments and related @Indexed/@Field...etc annotations. >> >> In the second use case, I'm merely using proto files as way to achieve compatibility mode, and I don't want any indexing to be done at all (cache is distributed with only compatibility and protostream marshaller enabled). >> >> Do I need a separate .proto file for this second use case where I remove the commented sections that enable indexing? Or can I use the one for the first use case? I really want to avoid any indexing happening in the second use case since it'd slow down things for no reason. >> >> Cheers, >> -- >> Galder Zamarre?o >> Infinispan, Red Hat >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From anistor at redhat.com Tue Apr 4 09:22:14 2017 From: anistor at redhat.com (Adrian Nistor) Date: Tue, 4 Apr 2017 16:22:14 +0300 Subject: [infinispan-dev] Proto file for indexed and non-indexed use case? In-Reply-To: References: Message-ID: <05f08a22-c01b-c359-f505-7d8b6e511f82@redhat.com> yes! On 04/04/2017 03:48 PM, Galder Zamarre?o wrote: > The cache for the second use case is already non-indexed. Is that enough to make sure the annotations are ignored? > > Cheers, > -- > Galder Zamarre?o > Infinispan, Red Hat > >> On 3 Apr 2017, at 18:58, Sanne Grinovero wrote: >> >> Hi Galder, >> >> did you consider using a non-indexed cache for the second case? >> >> Thanks, >> Sanne >> >> >> On 3 April 2017 at 16:44, Galder Zamarre?o wrote: >>> Hi Adrian, >>> >>> I had a question regarding proto files. I have a single domain of objects that I want to use for two different use cases. >>> >>> In the first use case, I want the proto files to be indexed so I define the comments and related @Indexed/@Field...etc annotations. >>> >>> In the second use case, I'm merely using proto files as way to achieve compatibility mode, and I don't want any indexing to be done at all (cache is distributed with only compatibility and protostream marshaller enabled). >>> >>> Do I need a separate .proto file for this second use case where I remove the commented sections that enable indexing? Or can I use the one for the first use case? I really want to avoid any indexing happening in the second use case since it'd slow down things for no reason. >>> >>> Cheers, >>> -- >>> Galder Zamarre?o >>> Infinispan, Red Hat >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From dan.berindei at gmail.com Tue Apr 4 10:37:20 2017 From: dan.berindei at gmail.com (Dan Berindei) Date: Tue, 4 Apr 2017 17:37:20 +0300 Subject: [infinispan-dev] Weekly IRC meeting logs 2017-04-03 Message-ID: Hi all I forgot to send out the meeting logs for yesterday's meeting: http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2017/infinispan.2017-04-03-14.01.html Cheers Dan From karesti at redhat.com Tue Apr 4 10:44:57 2017 From: karesti at redhat.com (Katia Aresti) Date: Tue, 4 Apr 2017 16:44:57 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support Message-ID: Hi all, As you probably know, Will and I are working on the vert-x infinispan integration [1], where the primary goal is to make infinispan the default cluster management of vert-x. (yeah!) Vert-x needs support for an Async Multimap. Today's implementation is a wrapper on a normal Cache where only Cache Key's are used to implement the multi map [2]. This is not very efficient, so after trying some other alternative implementations [3] that don't fully work (injection not working), Will and I have come to the conclusion that it might be a good idea to start having our own native CacheMultimap. This first multimap won't support duplicate values on key's. As a quick start, the smallest multimap we need should implement the following interface : public interface CacheMultimap { V put(K key, V value); Collection get(K key); boolean remove(K key, V value); } CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. We could add a new method in EmbeddedCacheManager.java CacheMultimap getCacheMultimap(String cacheName, boolean createIfAbsent); Implementation will create a cache as always and return a new CacheMultimapImpl(cache). What do you think ? Please fell free to suggest any other alternative or idea. Cheers Katia [1] https://github.com/vert-x3/vertx-infinispan [2] https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170404/59935206/attachment-0001.html From gustavo at infinispan.org Tue Apr 4 12:12:38 2017 From: gustavo at infinispan.org (Gustavo Fernandes) Date: Tue, 4 Apr 2017 17:12:38 +0100 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: On Tue, Apr 4, 2017 at 3:44 PM, Katia Aresti wrote: > Hi all, > > As you probably know, Will and I are working on the vert-x infinispan > integration [1], where the primary goal is to make infinispan the default > cluster management of vert-x. (yeah!) > Vert-x needs support for an Async Multimap. Today's implementation is a > wrapper on a normal Cache where only Cache Key's are used to implement the > multi map [2]. > This is not very efficient, so after trying some other alternative > implementations [3] that don't fully work (injection not working), Will and > I have come to the conclusion that it might be a good idea to start having > our own native CacheMultimap. This first multimap won't support duplicate > values on key's. > Sounds a great idea! This structure could be useful internally as well, for example in the Lucene directory, where we emulate a multimap using a DeltaAware Set in the value + IGNORE_RETURN_VALUE flag for better performance. > > As a quick start, the smallest multimap we need should implement the > following interface : > > public interface CacheMultimap { > V put(K key, V value); > > What would it return here? Gustavo > Collection get(K key); > > boolean remove(K key, V value); > } > > CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. > > We could add a new method in EmbeddedCacheManager.java > > CacheMultimap getCacheMultimap(String cacheName, boolean > createIfAbsent); > > Implementation will create a cache as always and return a new > CacheMultimapImpl(cache). > > What do you think ? Please fell free to suggest any other alternative or > idea. > > Cheers > > Katia > > [1] https://github.com/vert-x3/vertx-infinispan > > [2] https://github.com/vert-x3/vertx-infinispan/blob/master/ > src/main/java/io/vertx/ext/cluster/infinispan/impl/ > InfinispanAsyncMultiMap.java > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170404/43914e2a/attachment.html From mudokonman at gmail.com Tue Apr 4 12:40:13 2017 From: mudokonman at gmail.com (William Burns) Date: Tue, 04 Apr 2017 16:40:13 +0000 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti wrote: > Hi all, > > As you probably know, Will and I are working on the vert-x infinispan > integration [1], where the primary goal is to make infinispan the default > cluster management of vert-x. (yeah!) > Vert-x needs support for an Async Multimap. Today's implementation is a > wrapper on a normal Cache where only Cache Key's are used to implement the > multi map [2]. > This is not very efficient, so after trying some other alternative > implementations [3] that don't fully work (injection not working), Will and > I have come to the conclusion that it might be a good idea to start having > our own native CacheMultimap. This first multimap won't support duplicate > values on key's. > > As a quick start, the smallest multimap we need should implement the > following interface : > I agree that having a very slim API to start should be better since we know how much trouble we get into implementing a very large API like ConcurrentMap :) > public interface CacheMultimap { > V put(K key, V value); > > This should probably return a boolean or Void. I am leaning towards the first, but I am open either way. > > Collection get(K key); > > boolean remove(K key, V value); > > We probably want a `boolean remove(K key)` method as well that removes all values mapped to the given key. > } > > CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. > > We could add a new method in EmbeddedCacheManager.java > > CacheMultimap getCacheMultimap(String cacheName, boolean > createIfAbsent); > I was thinking maybe this would exist in a separate module (outside of core)? or class that wraps (similar to DistributedExecutor) instead. My worry is about transactions, since the entry point to that is through Cache interface. The other option is we could add a `getCache` method on the `CacheMultiMap`. > > > Implementation will create a cache as always and return a new > CacheMultimapImpl(cache). > > What do you think ? Please fell free to suggest any other alternative or > idea. > > Cheers > > Katia > > [1] https://github.com/vert-x3/vertx-infinispan > > [2] > https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170404/6d0f79d1/attachment.html From karesti at redhat.com Wed Apr 5 02:47:03 2017 From: karesti at redhat.com (Katia Aresti) Date: Wed, 5 Apr 2017 08:47:03 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: @Gustavo, Concerning put it should return void, yes ! :D a little copy/paste mistake in the mail from me, sorry ! On Tue, Apr 4, 2017 at 6:40 PM, William Burns wrote: > > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti wrote: > >> Hi all, >> >> As you probably know, Will and I are working on the vert-x infinispan >> integration [1], where the primary goal is to make infinispan the default >> cluster management of vert-x. (yeah!) >> Vert-x needs support for an Async Multimap. Today's implementation is a >> wrapper on a normal Cache where only Cache Key's are used to implement the >> multi map [2]. >> This is not very efficient, so after trying some other alternative >> implementations [3] that don't fully work (injection not working), Will and >> I have come to the conclusion that it might be a good idea to start having >> our own native CacheMultimap. This first multimap won't support duplicate >> values on key's. >> >> As a quick start, the smallest multimap we need should implement the >> following interface : >> > > I agree that having a very slim API to start should be better since we > know how much trouble we get into implementing a very large API like > ConcurrentMap :) > >> public interface CacheMultimap { >> V put(K key, V value); >> >> This should probably return a boolean or Void. I am leaning towards the > first, but I am open either way. > >> >> Collection get(K key); >> >> boolean remove(K key, V value); >> >> We probably want a `boolean remove(K key)` method as well that removes > all values mapped to the given key. > >> } >> >> CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. >> >> We could add a new method in EmbeddedCacheManager.java >> >> CacheMultimap getCacheMultimap(String cacheName, boolean >> createIfAbsent); >> > > I was thinking maybe this would exist in a separate module (outside of > core)? or class that wraps (similar to DistributedExecutor) instead. My > worry is about transactions, since the entry point to that is through Cache > interface. The other option is we could add a `getCache` method on the > `CacheMultiMap`. > > >> >> >> Implementation will create a cache as always and return a new >> CacheMultimapImpl(cache). >> >> What do you think ? Please fell free to suggest any other alternative or >> idea. >> >> Cheers >> >> Katia >> >> [1] https://github.com/vert-x3/vertx-infinispan >> >> [2] https://github.com/vert-x3/vertx-infinispan/blob/master/ >> src/main/java/io/vertx/ext/cluster/infinispan/impl/ >> InfinispanAsyncMultiMap.java >> >> [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170405/fb932b83/attachment-0001.html From rvansa at redhat.com Wed Apr 5 03:56:41 2017 From: rvansa at redhat.com (Radim Vansa) Date: Wed, 5 Apr 2017 09:56:41 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: <09134fd7-bef1-3477-a37b-795bf3979f40@redhat.com> On 04/04/2017 06:40 PM, William Burns wrote: > > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti > wrote: > > Hi all, > > As you probably know, Will and I are working on the vert-x > infinispan integration [1], where the primary goal is to make > infinispan the default cluster management of vert-x. (yeah!) > Vert-x needs support for an Async Multimap. Today's implementation > is a wrapper on a normal Cache where only Cache Key's are used to > implement the multi map [2]. > This is not very efficient, so after trying some other alternative > implementations [3] that don't fully work (injection not working), > Will and I have come to the conclusion that it might be a good > idea to start having our own native CacheMultimap. This first > multimap won't support duplicate values on key's. > > As a quick start, the smallest multimap we need should implement > the following interface : > > I agree that having a very slim API to start should be better since we > know how much trouble we get into implementing a very large API like > ConcurrentMap :) > > public interface CacheMultimap { > I don't see anything async in this interface. If that's async, provide CompletableFuture return values. I am also considering if we want any fire & forget variants for these operations, but since we have to do retries to achieve consistency (and therefore we need some messages from owners to originator), I wouldn't include them. > V put(K key,V value); > > This should probably return a boolean or Void. I am leaning towards > the first, but I am open either way. I would rather call this "add", as vert-x does. CompletableFuture as return type here will allow to easily register the handler. > Collection get(K key); > > boolean remove(K key,V value); > > We probably want a `boolean remove(K key)` method as well that removes > all values mapped to the given key. What about "reset(key)"? > } > > CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. > > We could add a new method in EmbeddedCacheManager.java > > CacheMultimap getCacheMultimap(String cacheName, > boolean createIfAbsent); > > > I was thinking maybe this would exist in a separate module (outside of > core)? or class that wraps (similar to DistributedExecutor) instead. > My worry is about transactions, since the entry point to that is > through Cache interface. The other option is we could add a `getCache` > method on the `CacheMultiMap`. +1 Since the names of multimaps and maps will clash, we shouldn't hide that the underlying implementation is a Cache, so I'd suggest something like static CacheMultimap CacheMultimapFactory.get(Cache c) { ... } > > > Implementation will create a cache as always and return a new > CacheMultimapImpl(cache). > > What do you think ? Please fell free to suggest any other > alternative or idea. > > Cheers > > Katia > > [1] https://github.com/vert-x3/vertx-infinispan > > [2] > https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From slaskawi at redhat.com Wed Apr 5 04:05:25 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Wed, 05 Apr 2017 08:05:25 +0000 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: I love the idea of starting with a simple interface, so +1000 from me. I'm also assuming that our new MultiMap will be accessible in both Embedded and Client/Server mode, am I correct? I also think CacheMultimap should extend Iterable. I suspect some of our users might want to use for-each loop with it. Finally, we also need to think about some integration bits (maybe not for the initial implementation but it might be beneficial to create JIRAs for them). With CDI and Spring support we can make them super easy to use (by injecting newly created instances to the users code: @Inject CacheMultimap myMap). I also put some more comments below. Nice proposal Katia! On Tue, Apr 4, 2017 at 7:09 PM William Burns wrote: > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti wrote: > > Hi all, > > As you probably know, Will and I are working on the vert-x infinispan > integration [1], where the primary goal is to make infinispan the default > cluster management of vert-x. (yeah!) > Vert-x needs support for an Async Multimap. Today's implementation is a > wrapper on a normal Cache where only Cache Key's are used to implement the > multi map [2]. > This is not very efficient, so after trying some other alternative > implementations [3] that don't fully work (injection not working), Will and > I have come to the conclusion that it might be a good idea to start having > our own native CacheMultimap. This first multimap won't support duplicate > values on key's. > > As a quick start, the smallest multimap we need should implement the > following interface : > > > I agree that having a very slim API to start should be better since we > know how much trouble we get into implementing a very large API like > ConcurrentMap :) > > public interface CacheMultimap { > V put(K key, V value); > > This should probably return a boolean or Void. I am leaning towards the > first, but I am open either way. > Could you please tell me more why are you suggesting boolean or void? Returning previous value would make it more similar to a Map. > Collection get(K key); > > boolean remove(K key, V value); > > We probably want a `boolean remove(K key)` method as well that removes all > values mapped to the given key. > +1 > } > > CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. > > We could add a new method in EmbeddedCacheManager.java > > CacheMultimap getCacheMultimap(String cacheName, boolean > createIfAbsent); > > How about the other way around? Something like: static CacheMultimap CacheMultimap.create(BasicCache cache); This way we would avoid dependency from DefaultCacheManager to CacheMultimap. If we wanted to support both Embedded/Client Server mode we would probably need to use BasicCache as a parameter. The last argument for this solution is that creating producers in CDI/Spring would be trivial (we would just need to provide a generic producer method and with some luck that would be it). > > I was thinking maybe this would exist in a separate module (outside of > core)? or class that wraps (similar to DistributedExecutor) instead. My > worry is about transactions, since the entry point to that is through Cache > interface. The other option is we could add a `getCache` method on the > `CacheMultiMap`. > If we want to support both Embedded/Client Server mode, it should go to commons. Otherwise I would vote for core. > > > > > Implementation will create a cache as always and return a new > CacheMultimapImpl(cache). > > What do you think ? Please fell free to suggest any other alternative or > idea. > > Cheers > > Katia > > [1] https://github.com/vert-x3/vertx-infinispan > > [2] > https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170405/eb6d6dde/attachment.html From tsegismont at gmail.com Wed Apr 5 04:22:58 2017 From: tsegismont at gmail.com (Thomas SEGISMONT) Date: Wed, 5 Apr 2017 10:22:58 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: Hi Katia, 2017-04-04 16:44 GMT+02:00 Katia Aresti : > Hi all, > > As you probably know, Will and I are working on the vert-x infinispan > integration [1], where the primary goal is to make infinispan the default > cluster management of vert-x. (yeah!) > This can't be the primary goal. Making it the best cluster manager would be great though! > Vert-x needs support for an Async Multimap. Today's implementation is a > wrapper on a normal Cache where only Cache Key's are used to implement the > multi map [2]. > This is not very efficient, so after trying some other alternative > implementations [3] that don't fully work (injection not working), Will and > I have come to the conclusion that it might be a good idea to start having > our own native CacheMultimap. This first multimap won't support duplicate > values on key's. > > As a quick start, the smallest multimap we need should implement the > following interface : > > public interface CacheMultimap { > V put(K key, V value); > > Collection get(K key); > > boolean remove(K key, V value); > } > > CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. > > We could add a new method in EmbeddedCacheManager.java > > CacheMultimap getCacheMultimap(String cacheName, boolean > createIfAbsent); > > Implementation will create a cache as always and return a new > CacheMultimapImpl(cache). > > What do you think ? Please fell free to suggest any other alternative or > idea. > One thing we need in vertx-infinispan is the ability to clean the multimap, meaning removing some key/value pairs if value matches criteria. It doesn't seem possible with this interface. At least, it misses a method to get all the keys from this multimap. Ideally, I would like to be able to supply a Serializable Predicate. > > Cheers > > Katia > Thank you for looking into this! > > [1] https://github.com/vert-x3/vertx-infinispan > > [2] https://github.com/vert-x3/vertx-infinispan/blob/master/ > src/main/java/io/vertx/ext/cluster/infinispan/impl/ > InfinispanAsyncMultiMap.java > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170405/02d6004b/attachment-0001.html From ttarrant at redhat.com Wed Apr 5 08:42:46 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Wed, 5 Apr 2017 14:42:46 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: On 05/04/2017 10:05, Sebastian Laskawiec wrote: > I love the idea of starting with a simple interface, so +1000 from me. > > I'm also assuming that our new MultiMap will be accessible in both > Embedded and Client/Server mode, am I correct? Agreed. The design must take into account both variants before it can be considered ready. Tristan -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From mudokonman at gmail.com Wed Apr 5 10:34:50 2017 From: mudokonman at gmail.com (William Burns) Date: Wed, 05 Apr 2017 14:34:50 +0000 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: On Wed, Apr 5, 2017 at 4:30 AM Sebastian Laskawiec wrote: > I love the idea of starting with a simple interface, so +1000 from me. > > I'm also assuming that our new MultiMap will be accessible in both > Embedded and Client/Server mode, am I correct? I also think CacheMultimap > should extend Iterable. I suspect some of our users might want to use > for-each loop with it. Finally, we also need to think about some > integration bits (maybe not for the initial implementation but it might be > beneficial to create JIRAs for them). With CDI and Spring support we can > make them super easy to use (by injecting newly created instances to the > users code: @Inject CacheMultimap myMap). > > I also put some more comments below. Nice proposal Katia! > > On Tue, Apr 4, 2017 at 7:09 PM William Burns wrote: > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti wrote: > > Hi all, > > As you probably know, Will and I are working on the vert-x infinispan > integration [1], where the primary goal is to make infinispan the default > cluster management of vert-x. (yeah!) > Vert-x needs support for an Async Multimap. Today's implementation is a > wrapper on a normal Cache where only Cache Key's are used to implement the > multi map [2]. > This is not very efficient, so after trying some other alternative > implementations [3] that don't fully work (injection not working), Will and > I have come to the conclusion that it might be a good idea to start having > our own native CacheMultimap. This first multimap won't support duplicate > values on key's. > > As a quick start, the smallest multimap we need should implement the > following interface : > > > I agree that having a very slim API to start should be better since we > know how much trouble we get into implementing a very large API like > ConcurrentMap :) > > public interface CacheMultimap { > V put(K key, V value); > > This should probably return a boolean or Void. I am leaning towards the > first, but I am open either way. > > > Could you please tell me more why are you suggesting boolean or void? > Returning previous value would make it more similar to a Map. > The value of the MultiMap is a Collection which in this version doesn't allow duplicates. By returning boolean we can say if the value was added (true) or it was already present (false). > > > Collection get(K key); > > boolean remove(K key, V value); > > We probably want a `boolean remove(K key)` method as well that removes all > values mapped to the given key. > > > +1 > > > } > > CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. > > We could add a new method in EmbeddedCacheManager.java > > CacheMultimap getCacheMultimap(String cacheName, boolean > createIfAbsent); > > > How about the other way around? Something like: > static CacheMultimap CacheMultimap.create(BasicCache > cache); > > This way we would avoid dependency from DefaultCacheManager > to CacheMultimap. If we wanted to support both Embedded/Client Server mode > we would probably need to use BasicCache as a parameter. The last argument > for this solution is that creating producers in CDI/Spring would be trivial > (we would just need to provide a generic producer method and with some luck > that would be it). > > > > I was thinking maybe this would exist in a separate module (outside of > core)? or class that wraps (similar to DistributedExecutor) instead. My > worry is about transactions, since the entry point to that is through Cache > interface. The other option is we could add a `getCache` method on the > `CacheMultiMap`. > > > If we want to support both Embedded/Client Server mode, it should go to > commons. Otherwise I would vote for core. > Commons should work. Only problem is the functional commands don't really work efficiently over Hot Rod (does get/replace in a loop). We would need to add some more handling in the protocol to allow for only partial replication of values and only 1 remote call. > > > > > > > Implementation will create a cache as always and return a new > CacheMultimapImpl(cache). > > What do you think ? Please fell free to suggest any other alternative or > idea. > > Cheers > > Katia > > [1] https://github.com/vert-x3/vertx-infinispan > > [2] > https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170405/30f8f1cc/attachment.html From karesti at redhat.com Wed Apr 5 18:15:17 2017 From: karesti at redhat.com (Katia Aresti) Date: Thu, 6 Apr 2017 00:15:17 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: <09134fd7-bef1-3477-a37b-795bf3979f40@redhat.com> References: <09134fd7-bef1-3477-a37b-795bf3979f40@redhat.com> Message-ID: On Wed, Apr 5, 2017 at 9:56 AM, Radim Vansa wrote: > On 04/04/2017 06:40 PM, William Burns wrote: > > > > > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti > > wrote: > > > > Hi all, > > > > As you probably know, Will and I are working on the vert-x > > infinispan integration [1], where the primary goal is to make > > infinispan the default cluster management of vert-x. (yeah!) > > Vert-x needs support for an Async Multimap. Today's implementation > > is a wrapper on a normal Cache where only Cache Key's are used to > > implement the multi map [2]. > > This is not very efficient, so after trying some other alternative > > implementations [3] that don't fully work (injection not working), > > Will and I have come to the conclusion that it might be a good > > idea to start having our own native CacheMultimap. This first > > multimap won't support duplicate values on key's. > > > > As a quick start, the smallest multimap we need should implement > > the following interface : > > > > I agree that having a very slim API to start should be better since we > > know how much trouble we get into implementing a very large API like > > ConcurrentMap :) > > > > public interface CacheMultimap { > > > > I don't see anything async in this interface. If that's async, provide > CompletableFuture return values. > I am also considering if we want any fire & forget variants for these > operations, but since we have to do retries to achieve consistency (and > therefore we need some messages from owners to originator), I wouldn't > include them. > Today's vert-x API calls the vertx.executeBlocking(future => cache...) I considered the option of CompletableFuture, but for simplicity I suggested the basic method. Today's CacheAPI makes a difference between "put" and "putAsync". Would you call the interface CacheMultimapAsync or CacheMultimap with addAsyc method ? > > > V put(K key,V value); > > > > This should probably return a boolean or Void. I am leaning towards > > the first, but I am open either way. > > I would rather call this "add", as vert-x does. CompletableFuture as > return type here will allow to easily register the handler -1 I prefer keeping "put" name because it is still a Map and makes more sense to me considering the actual Cache API too. The return type V was a transcription mistake, it should be void for me, as Will pointed out. > > > Collection get(K key); > > > > boolean remove(K key,V value); > > > > We probably want a `boolean remove(K key)` method as well that removes > > all values mapped to the given key. > > What about "reset(key)"? > > } > > > > CacheMultimapImpl will be a wrapper on a normal Cache, similar to > [3]. > > > > We could add a new method in EmbeddedCacheManager.java > > > > CacheMultimap getCacheMultimap(String cacheName, > > boolean createIfAbsent); > > > > > > I was thinking maybe this would exist in a separate module (outside of > > core)? or class that wraps (similar to DistributedExecutor) instead. > > My worry is about transactions, since the entry point to that is > > through Cache interface. The other option is we could add a `getCache` > > method on the `CacheMultiMap`. > > +1 Since the names of multimaps and maps will clash, we shouldn't hide > that the underlying implementation is a Cache, so I'd suggest something > like > > static CacheMultimap CacheMultimapFactory.get(Cache Object> c) { ... } > > > > > > > Implementation will create a cache as always and return a new > > CacheMultimapImpl(cache). > > > > What do you think ? Please fell free to suggest any other > > alternative or idea. > > > > Cheers > > > > Katia > > > > [1] https://github.com/vert-x3/vertx-infinispan > > > > [2] > > https://github.com/vert-x3/vertx-infinispan/blob/master/ > src/main/java/io/vertx/ext/cluster/infinispan/impl/ > InfinispanAsyncMultiMap.java > > > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org jboss.org> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170406/d0d9a8d2/attachment-0001.html From rvansa at redhat.com Thu Apr 6 04:42:33 2017 From: rvansa at redhat.com (Radim Vansa) Date: Thu, 6 Apr 2017 10:42:33 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: <7a8ef3a2-c705-9a22-7719-02f2f6599b15@redhat.com> On 04/05/2017 04:34 PM, William Burns wrote: > > > On Wed, Apr 5, 2017 at 4:30 AM Sebastian Laskawiec > > wrote: > > I love the idea of starting with a simple interface, so +1000 from > me. > > I'm also assuming that our new MultiMap will be accessible in both > Embedded and Client/Server mode, am I correct? I also think > CacheMultimap should extend Iterable. I suspect some of our users > might want to use for-each loop with it. Finally, we also need to > think about some integration bits (maybe not for the initial > implementation but it might be beneficial to create JIRAs for > them). With CDI and Spring support we can make them super easy to > use (by injecting newly created instances to the users code: > @Inject CacheMultimap myMap). > > I also put some more comments below. Nice proposal Katia! > > On Tue, Apr 4, 2017 at 7:09 PM William Burns > wrote: > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti > wrote: > > Hi all, > > As you probably know, Will and I are working on the vert-x > infinispan integration [1], where the primary goal is to > make infinispan the default cluster management of vert-x. > (yeah!) > Vert-x needs support for an Async Multimap. Today's > implementation is a wrapper on a normal Cache where only > Cache Key's are used to implement the multi map [2]. > This is not very efficient, so after trying some other > alternative implementations [3] that don't fully work > (injection not working), Will and I have come to the > conclusion that it might be a good idea to start having > our own native CacheMultimap. This first multimap won't > support duplicate values on key's. > > As a quick start, the smallest multimap we need should > implement the following interface : > > I agree that having a very slim API to start should be better > since we know how much trouble we get into implementing a very > large API like ConcurrentMap :) > > public interface CacheMultimap { > V put(K key,V value); > > This should probably return a boolean or Void. I am leaning > towards the first, but I am open either way. > > > Could you please tell me more why are you suggesting boolean or > void? Returning previous value would make it more similar to a Map. > > > The value of the MultiMap is a Collection which in this version > doesn't allow duplicates. By returning boolean we can say if the value > was added (true) or it was already present (false). If it does not allow duplicates, why get() does not return a Set? Btw., there are other ~useful return types possible, e.g. `int` returning size of the collection pre/post modification (not advocating that, though - I would personally rather see the boolean, if not CompletableFuture). I wouldn't return the whole previous value (full collection). > Collection get(K key); > > boolean remove(K key,V value); > > We probably want a `boolean remove(K key)` method as well that > removes all values mapped to the given key. > > > +1 > > } > > CacheMultimapImpl will be a wrapper on a normal Cache, > similar to [3]. > > We could add a new method in EmbeddedCacheManager.java > > CacheMultimap getCacheMultimap(String > cacheName, boolean createIfAbsent); > > > How about the other way around? Something like: > static CacheMultimap > CacheMultimap.create(BasicCache cache); > > This way we would avoid dependency from DefaultCacheManager > to CacheMultimap. If we wanted to support both Embedded/Client > Server mode we would probably need to use BasicCache as a > parameter. The last argument for this solution is that creating > producers in CDI/Spring would be trivial (we would just need to > provide a generic producer method and with some luck that would be > it). > > > I was thinking maybe this would exist in a separate module > (outside of core)? or class that wraps (similar to > DistributedExecutor) instead. My worry is about transactions, > since the entry point to that is through Cache interface. The > other option is we could add a `getCache` method on the > `CacheMultiMap`. > > > If we want to support both Embedded/Client Server mode, it should > go to commons. Otherwise I would vote for core. > > > Commons should work. Only problem is the functional commands don't > really work efficiently over Hot Rod (does get/replace in a loop). We > would need to add some more handling in the protocol to allow for only > partial replication of values and only 1 remote call. I guess that only the CacheMultimap interface (+ maybe some helper classes) should land in commons, the actual implementation (in hotrod-client module or its extension module) wouldn't use compute() over Hot Rod. R. > > > Implementation will create a cache as always and return a > new CacheMultimapImpl(cache). > > What do you think ? Please fell free to suggest any other > alternative or idea. > > Cheers > > Katia > > [1] https://github.com/vert-x3/vertx-infinispan > > [2] > https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java > > [3] > https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN?ASKAWIEC > > INFINISPAN DEVELOPER > > Red HatEMEA > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From rvansa at redhat.com Thu Apr 6 05:04:45 2017 From: rvansa at redhat.com (Radim Vansa) Date: Thu, 6 Apr 2017 11:04:45 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: <09134fd7-bef1-3477-a37b-795bf3979f40@redhat.com> Message-ID: On 04/06/2017 12:15 AM, Katia Aresti wrote: > > > On Wed, Apr 5, 2017 at 9:56 AM, Radim Vansa > wrote: > > On 04/04/2017 06:40 PM, William Burns wrote: > > > > > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti > > >> wrote: > > > > Hi all, > > > > As you probably know, Will and I are working on the vert-x > > infinispan integration [1], where the primary goal is to make > > infinispan the default cluster management of vert-x. (yeah!) > > Vert-x needs support for an Async Multimap. Today's > implementation > > is a wrapper on a normal Cache where only Cache Key's are > used to > > implement the multi map [2]. > > This is not very efficient, so after trying some other > alternative > > implementations [3] that don't fully work (injection not > working), > > Will and I have come to the conclusion that it might be a good > > idea to start having our own native CacheMultimap. This first > > multimap won't support duplicate values on key's. > > > > As a quick start, the smallest multimap we need should implement > > the following interface : > > > > I agree that having a very slim API to start should be better > since we > > know how much trouble we get into implementing a very large API like > > ConcurrentMap :) > > > > public interface CacheMultimap { > > > > I don't see anything async in this interface. If that's async, provide > CompletableFuture return values. > I am also considering if we want any fire & forget variants for these > operations, but since we have to do retries to achieve consistency > (and > therefore we need some messages from owners to originator), I wouldn't > include them. > > > Today's vert-x API calls the vertx.executeBlocking(future => cache...) > > I considered the option of CompletableFuture, but for simplicity I > suggested the basic method. > Today's CacheAPI makes a difference between "put" and "putAsync". > Would you call the interface CacheMultimapAsync or CacheMultimap with > addAsyc method ? "In a perfect world, there will be no war or hunger, all APIs will be written asynchronously and bunny rabbits will skip hand-in-hand with baby lambs across sunny green meadows." (quoting Vert.x docs) While minimalistic API is a good way to start, it shouldn't contain anything we'd want to get rid of in close future. And especially since the main drive for multimaps is Vert.x which consumes asynchronous APIs (and has support for legacy synchronous APIs, the executeBlocking method), we should have the design adapted to that from the beginning. CompletableFuture is not a rocket science, and you can use the already asynchronous Infinispan internals. I don't think we should have two interfaces, I believe that single interface with async methods only is absolutely sufficient. Though I wouldn't add the *Async suffix at all there. If someone wants to execute the methods synchronously he can call .get() or .join() - just 6/7 characters more. > > > V put(K key,V value); > > > > This should probably return a boolean or Void. I am leaning towards > > the first, but I am open either way. > > I would rather call this "add", as vert-x does. CompletableFuture as > return type here will allow to easily register the handler > > > -1 I prefer keeping "put" name because it is still a Map and makes > more sense to me considering the actual Cache API too. The return type > V was a transcription mistake, it should be void for me, as Will > pointed out. To me "put" is linked with overwriting the previous value, while you add to the underlying collection or create a new single-element one. But whatever, I care more about the return values :) R. > > > > Collection get(K key); > > > > boolean remove(K key,V value); > > > > We probably want a `boolean remove(K key)` method as well that > removes > > all values mapped to the given key. > > What about "reset(key)"? > > > > } > > > > CacheMultimapImpl will be a wrapper on a normal Cache, > similar to [3]. > > > > We could add a new method in EmbeddedCacheManager.java > > > > CacheMultimap getCacheMultimap(String cacheName, > > boolean createIfAbsent); > > > > > > I was thinking maybe this would exist in a separate module > (outside of > > core)? or class that wraps (similar to DistributedExecutor) instead. > > My worry is about transactions, since the entry point to that is > > through Cache interface. The other option is we could add a > `getCache` > > method on the `CacheMultiMap`. > > +1 Since the names of multimaps and maps will clash, we shouldn't hide > that the underlying implementation is a Cache, so I'd suggest > something like > > static CacheMultimap CacheMultimapFactory.get(Cache Object> c) { ... } > > > > > > > Implementation will create a cache as always and return a new > > CacheMultimapImpl(cache). > > > > What do you think ? Please fell free to suggest any other > > alternative or idea. > > > > Cheers > > > > Katia > > > > [1] https://github.com/vert-x3/vertx-infinispan > > > > > [2] > > > https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java > > > > > [3] > https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > -- > Radim Vansa > > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From slaskawi at redhat.com Thu Apr 6 05:31:46 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Thu, 06 Apr 2017 09:31:46 +0000 Subject: [infinispan-dev] Single Endpoint design In-Reply-To: References: <1bcc751c-a246-bcb4-3c43-8bfe0ed252a7@redhat.com> <3ae53674-5362-139b-518e-717642a39bcd@redhat.com> Message-ID: The design has been moved here: https://github.com/infinispan/infinispan-designs/pull/2 Are there any last comments? I plan to start working on that shortly. Thanks, Sebastian On Fri, Mar 31, 2017 at 4:20 PM Sebastian Laskawiec wrote: > With TLS+ALPN the use case is very simple. The protocol is negotiated > during the handshake. Then the existing TCP connection is reused and the > client starts to send Hot Rod binary bits through the wire. Sadly, TLS adds > some significant overhead to the transmission. > > With HTTP/1.1 Upgrade it's a bit more complicated. The Upgrade request > might issued by the client as well as by the server. Each of the parties > can ignore the request if it doesn't recognize it (or doesn't support > protocols proposed by the other side). The more I think about this, the > more I lean towards to server initiated upgrade requests. This way, only > clients which supports the upgrade will react to it. It still needs some > more thought... > > Of course you are right, some additional code will need to be added to the > clients to support switching protocols. The good news is that you won't be > forced to use the Single Endpoint. You may want to expose our old, good Hot > Rod Endpoint and forget about all this switching complexity. > > Regarding to network hops - no, there is no additional network hops. The > router component simply identifies the incoming request and plugs proper > Netty Handler stack to it. Here's an example how it is done with > multi-tenancy based on SNI [1]. > > [1] > https://github.com/infinispan/infinispan/blob/master/server/router/src/main/java/org/infinispan/server/router/router/impl/hotrod/handlers/SniRouteHandler.java#L48-L61 > > On Fri, Mar 31, 2017 at 2:22 PM Tristan Tarrant > wrote: > > No, once the connection is established, I believe the netty pipeline can > be trimmed to the necessary elements. > > Tristan > > On 31/03/2017 13:57, Gustavo Fernandes wrote: > > On Fri, Mar 31, 2017 at 11:02 AM, Tristan Tarrant > > wrote: > > > > You understood incorrectly. > > The only change to the Hot Rod clients is that, if they get a 400 > error > > from a HR PING request, they will initiate an upgrade to Hot Rod and > > then proceed with the usual Hot Rod protocol after that. > > > > > > Thanks for the clarification. Still, after the HR protocol is > > negotiated, communication will go > > through a router, thus adding an extra hop? > > > > Gustavo > > > > Tristan > > > > On 31/03/2017 11:58, Gustavo Fernandes wrote: > > > Hi Sebastian, > > > > > > If I understood it correctly, all the Hot Rod clients will be > changed > > > from using: > > > > > > - Binary over TCP, circa 40 bytes header, no hops to contact the > server, > > > no protocol negotiation, no encryption (default) > > > > > > to > > > > > > - HTTP/2 with SSL, protocol upgrade negotiation, and a hop > (router) to > > > connect to the server. > > > > > > > > > Any idea of how significant would be this extra overhead > introduced? > > > > > > > > > Thanks, > > > Gustavo > > > > > > > > > On Thu, Mar 30, 2017 at 2:01 PM, Sebastian Laskawiec > > > > > >> wrote: > > > > > > Hey! > > > > > > My plan is to start working on a Single Point support for > > Infinispan > > > Server very soon and I prepared a design: > > > https://github.com/infinispan/infinispan/pull/5041 > > > > > > > > > > > > > As you can see I did not use our Wiki (as we used to) because > it > > > doesn't support inline comments (which is pretty bad in my > > opinion). > > > I would like to propose to keep all the designs along with our > > > source code. This approach has been successfully used by the > > > Kubernetes [1] folks (although they migrated designs into the > new > > > Community repository [2] recently). I think it might be a > > good idea > > > to do something similar. > > > > > > Feedback on both items is more than welcome. > > > > > > Thanks, > > > Sebastian > > > > > > [1] > > > > > https://github.com/kubernetes/kubernetes/tree/master/docs/proposals > > > > > > > > < > https://github.com/kubernetes/kubernetes/tree/master/docs/proposals < > https://github.com/kubernetes/kubernetes/tree/master/docs/proposals>> > > > [2] > > > > > > https://github.com/kubernetes/community/tree/master/contributors/design-proposals > > < > https://github.com/kubernetes/community/tree/master/contributors/design-proposals > > > > > > > < > https://github.com/kubernetes/community/tree/master/contributors/design-proposals > < > https://github.com/kubernetes/community/tree/master/contributors/design-proposals > >> > > > > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org > > > > > > > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org infinispan-dev at lists.jboss.org> > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > > -- > > Tristan Tarrant > > Infinispan Lead > > JBoss, a division of Red Hat > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org infinispan-dev at lists.jboss.org> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170406/f6abb1b4/attachment-0001.html From karesti at redhat.com Thu Apr 6 08:29:22 2017 From: karesti at redhat.com (Katia Aresti) Date: Thu, 6 Apr 2017 14:29:22 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: <09134fd7-bef1-3477-a37b-795bf3979f40@redhat.com> Message-ID: What if we move the discussion to https://github.com/infinispan/infinispan-designs ? Katia On Thu, Apr 6, 2017 at 11:04 AM, Radim Vansa wrote: > On 04/06/2017 12:15 AM, Katia Aresti wrote: > > > > > > On Wed, Apr 5, 2017 at 9:56 AM, Radim Vansa > > wrote: > > > > On 04/04/2017 06:40 PM, William Burns wrote: > > > > > > > > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti > > > > >> wrote: > > > > > > Hi all, > > > > > > As you probably know, Will and I are working on the vert-x > > > infinispan integration [1], where the primary goal is to make > > > infinispan the default cluster management of vert-x. (yeah!) > > > Vert-x needs support for an Async Multimap. Today's > > implementation > > > is a wrapper on a normal Cache where only Cache Key's are > > used to > > > implement the multi map [2]. > > > This is not very efficient, so after trying some other > > alternative > > > implementations [3] that don't fully work (injection not > > working), > > > Will and I have come to the conclusion that it might be a good > > > idea to start having our own native CacheMultimap. This first > > > multimap won't support duplicate values on key's. > > > > > > As a quick start, the smallest multimap we need should > implement > > > the following interface : > > > > > > I agree that having a very slim API to start should be better > > since we > > > know how much trouble we get into implementing a very large API > like > > > ConcurrentMap :) > > > > > > public interface CacheMultimap { > > > > > > > I don't see anything async in this interface. If that's async, > provide > > CompletableFuture return values. > > I am also considering if we want any fire & forget variants for these > > operations, but since we have to do retries to achieve consistency > > (and > > therefore we need some messages from owners to originator), I > wouldn't > > include them. > > > > > > Today's vert-x API calls the vertx.executeBlocking(future => cache...) > > > > I considered the option of CompletableFuture, but for simplicity I > > suggested the basic method. > > Today's CacheAPI makes a difference between "put" and "putAsync". > > Would you call the interface CacheMultimapAsync or CacheMultimap with > > addAsyc method ? > > "In a perfect world, there will be no war or hunger, all APIs will be > written asynchronously and bunny rabbits will skip hand-in-hand with > baby lambs across sunny green meadows." (quoting Vert.x docs) > > While minimalistic API is a good way to start, it shouldn't contain > anything we'd want to get rid of in close future. And especially since > the main drive for multimaps is Vert.x which consumes asynchronous APIs > (and has support for legacy synchronous APIs, the executeBlocking > method), we should have the design adapted to that from the beginning. > > CompletableFuture is not a rocket science, and you can use the already > asynchronous Infinispan internals. > > I don't think we should have two interfaces, I believe that single > interface with async methods only is absolutely sufficient. Though I > wouldn't add the *Async suffix at all there. If someone wants to execute > the methods synchronously he can call .get() or .join() - just 6/7 > characters more. > > > > > > V put(K key,V value); > > > > > > This should probably return a boolean or Void. I am leaning towards > > > the first, but I am open either way. > > > > I would rather call this "add", as vert-x does. CompletableFuture as > > return type here will allow to easily register the handler > > > > > > -1 I prefer keeping "put" name because it is still a Map and makes > > more sense to me considering the actual Cache API too. The return type > > V was a transcription mistake, it should be void for me, as Will > > pointed out. > > To me "put" is linked with overwriting the previous value, while you add > to the underlying collection or create a new single-element one. But > whatever, I care more about the return values :) > > R. > > > > > > > > Collection get(K key); > > > > > > boolean remove(K key,V value); > > > > > > We probably want a `boolean remove(K key)` method as well that > > removes > > > all values mapped to the given key. > > > > What about "reset(key)"? > > > > > > > } > > > > > > CacheMultimapImpl will be a wrapper on a normal Cache, > > similar to [3]. > > > > > > We could add a new method in EmbeddedCacheManager.java > > > > > > CacheMultimap getCacheMultimap(String cacheName, > > > boolean createIfAbsent); > > > > > > > > > I was thinking maybe this would exist in a separate module > > (outside of > > > core)? or class that wraps (similar to DistributedExecutor) > instead. > > > My worry is about transactions, since the entry point to that is > > > through Cache interface. The other option is we could add a > > `getCache` > > > method on the `CacheMultiMap`. > > > > +1 Since the names of multimaps and maps will clash, we shouldn't > hide > > that the underlying implementation is a Cache, so I'd suggest > > something like > > > > static CacheMultimap CacheMultimapFactory.get(Cache > Object> c) { ... } > > > > > > > > > > > Implementation will create a cache as always and return a new > > > CacheMultimapImpl(cache). > > > > > > What do you think ? Please fell free to suggest any other > > > alternative or idea. > > > > > > Cheers > > > > > > Katia > > > > > > [1] https://github.com/vert-x3/vertx-infinispan > > > > > > > > [2] > > > > > https://github.com/vert-x3/vertx-infinispan/blob/master/ > src/main/java/io/vertx/ext/cluster/infinispan/impl/ > InfinispanAsyncMultiMap.java > > src/main/java/io/vertx/ext/cluster/infinispan/impl/ > InfinispanAsyncMultiMap.java> > > > > > > [3] > > https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org > > > > > > > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > > > > > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org > > > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > -- > > Radim Vansa > > > JBoss Performance Team > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org jboss.org> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170406/5a015900/attachment.html From galder at redhat.com Fri Apr 7 04:48:23 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Fri, 7 Apr 2017 10:48:23 +0200 Subject: [infinispan-dev] In Memory Data Grid Patterns Demos from Devoxx France! Message-ID: <2000E4DB-7A7F-45E4-8833-7EB3A1C60DF0@redhat.com> Hi all, I've just got back from Devoxx France where Emmanuel and I presented about in-memory data grid use cases, and during this talk we presented a couple of demos on using Infinispan for offline analytics and real-time data processing. I've just created a new blog post with some very quick instructions for you to run these demos: http://blog.infinispan.org/2017/04/in-memory-data-grid-patterns-demos-from.html Give them a try and let us know what you think! Cheers, -- Galder Zamarre?o Infinispan, Red Hat From rory.odonnell at oracle.com Fri Apr 7 05:27:56 2017 From: rory.odonnell at oracle.com (Rory O'Donnell) Date: Fri, 7 Apr 2017 10:27:56 +0100 Subject: [infinispan-dev] JDK 9 Developer Preview is now available on java.net Message-ID: <51f467f8-f97e-c699-a767-829b98f442f6@oracle.com> Hi Galder, *JDK 9 Developer Preview is now available on java.net [1] * Developer Preview milestone: - A reasonably stable build suitable for broad testing by the developer community is available. JDK 9 Builds 163 and higher include all planned features. *Attention annotation processing users and authors - * Request for feedback on annotation processing API changes made in JDK 9. As has been done previously during Java SE 7 and Java SE 8, the JSR 269 annotation processing API is undergoing a maintenance review (MR) as part of Java SE 9. Details of the changes in JDK 9 Early Access build 163 & build 164 available here [2] Please report experiences running processors under JDK 9 and feedback on the API changes to the compiler-dev mailing list. (If you haven?t already subscribed to that list then please do so first, otherwise your message will be discarded as spam.) Rgds, Rory [1] https://jdk9.java.net/download/ [2] http://mail.openjdk.java.net/pipermail/compiler-dev/2017-April/010896.html -- Rgds,Rory O'Donnell Quality Engineering Manager -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170407/4def026c/attachment-0001.html From galder at redhat.com Fri Apr 7 08:20:19 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Fri, 7 Apr 2017 14:20:19 +0200 Subject: [infinispan-dev] Executing server tasks that contain POJOs In-Reply-To: References: <7383B4E7-B386-41CE-A62E-8DFBD439C48D@redhat.com> Message-ID: <5A1EA0D2-D4C1-4968-9E11-50BC673ABC77@redhat.com> -- Galder Zamarre?o Infinispan, Red Hat > On 30 Mar 2017, at 18:33, Dan Berindei wrote: > > On Thu, Mar 30, 2017 at 3:51 PM, Galder Zamarre?o wrote: >> Hi all, >> >> For a demo I'm giving next week, I'd like to show how to use distributed streams via a remote server task. All server tasks that we have in testsuite rely on primitives but in my case I wanted to use POJOs. >> >> To do that, I needed to get compatibility mode working in such way that those POJOs could be unmarshalled for the server task. Since in another demo I'm showing Protostream based POJOs, I thought I'd try to use that as mechanism to unmarshall POJOs server side. >> >> We have a test for such scenario [1], but the reality (running on a proper server) is anything that simple. Here's a list of things I've found out while creating a WordCount example that relies on a POJO: >> >> 1. Out of the box, it's impossible to set compatibility marshaller to org.infinispan.query.remote.CompatibilityProtoStreamMarshaller [1] because "org.infinispan.main" classloader can't access that class. I worked around that by tweaking the module.xml to have an optional dependency to "org.infinispan.remote-query.server" module. >> > > I know Sanne also wanted to add one of the query modules as an > optional dependency to the core module for a similar reason, but it > seems really hacky. Yeah, might be a bit hacky but it's just a configuration change as opposed to a code change. > Could the server create the GlobalConfigurationBuilder with a > classloader that has access to the query module instead? Hmmm, not sure. I mean, in theory you could plug any marshaller, even com.acme.WhateverMarshaller, so it's not really an issue about having access to the query module, but having access a module that contains that marshaller. > Alternatively, I know ModularClassResolver prefixes class names with > the slot and module name, and can load a class from any module. Maybe > we could also allow a slot:module:class format everywhere the > configuration currently accepts a class name? That could be handy. I'm about to create a JIRA on this, so I'll add it as an idea to it. > >> 2. After doing that, I had to register the protofile and associated classes remotely in the server. Again, there's no out of the box mechanism for that, so I created a remote server task that would do that [3]. >> >> 3. Finally, with all that in place, I was able to complete the WordCount test [4] with a final caveat: the return of the word count, and words protofile registration, tasks return objects that are not marshalled by the compatibility marshaller, so I had to make sure that the remote cache manager used for those tasks uses the default marshaller. >> >> Clearly we need to improve on this, and we have plans to address these issues (with new upcoming transcoding capabilities), but I thought it'd be worth mentioning the problems found in case anyone else encounters them before transcoding is in place. >> >> Cheers, >> >> [1] https://github.com/galderz/datagrid-patterns/blob/master/server-config/domain/domain.xml#L139 >> [2] https://github.com/galderz/datagrid-patterns/blob/master/server-config/org.infinispan.main_module.xml#L18 >> [3] https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-server/src/main/java/test/WordsProtoTask.java >> [4] https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-client/src/test/java/test/WordCountTest.java >> -- >> Galder Zamarre?o >> Infinispan, Red Hat >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From galder at redhat.com Fri Apr 7 12:47:09 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Fri, 7 Apr 2017 18:47:09 +0200 Subject: [infinispan-dev] Executing server tasks that contain POJOs In-Reply-To: <5A1EA0D2-D4C1-4968-9E11-50BC673ABC77@redhat.com> References: <7383B4E7-B386-41CE-A62E-8DFBD439C48D@redhat.com> <5A1EA0D2-D4C1-4968-9E11-50BC673ABC77@redhat.com> Message-ID: <64EBFD04-339E-4137-B146-0B17A45BB1CE@redhat.com> FYI, I've created these JIRAs to improve some of the issues highlighted here: https://issues.jboss.org/browse/ISPN-7710 https://issues.jboss.org/browse/ISPN-7711 Cheers, -- Galder Zamarre?o Infinispan, Red Hat > On 7 Apr 2017, at 14:20, Galder Zamarre?o wrote: > > > -- > Galder Zamarre?o > Infinispan, Red Hat > >> On 30 Mar 2017, at 18:33, Dan Berindei wrote: >> >> On Thu, Mar 30, 2017 at 3:51 PM, Galder Zamarre?o wrote: >>> Hi all, >>> >>> For a demo I'm giving next week, I'd like to show how to use distributed streams via a remote server task. All server tasks that we have in testsuite rely on primitives but in my case I wanted to use POJOs. >>> >>> To do that, I needed to get compatibility mode working in such way that those POJOs could be unmarshalled for the server task. Since in another demo I'm showing Protostream based POJOs, I thought I'd try to use that as mechanism to unmarshall POJOs server side. >>> >>> We have a test for such scenario [1], but the reality (running on a proper server) is anything that simple. Here's a list of things I've found out while creating a WordCount example that relies on a POJO: >>> >>> 1. Out of the box, it's impossible to set compatibility marshaller to org.infinispan.query.remote.CompatibilityProtoStreamMarshaller [1] because "org.infinispan.main" classloader can't access that class. I worked around that by tweaking the module.xml to have an optional dependency to "org.infinispan.remote-query.server" module. >>> >> >> I know Sanne also wanted to add one of the query modules as an >> optional dependency to the core module for a similar reason, but it >> seems really hacky. > > Yeah, might be a bit hacky but it's just a configuration change as opposed to a code change. > >> Could the server create the GlobalConfigurationBuilder with a >> classloader that has access to the query module instead? > > Hmmm, not sure. I mean, in theory you could plug any marshaller, even com.acme.WhateverMarshaller, so it's not really an issue about having access to the query module, but having access a module that contains that marshaller. > >> Alternatively, I know ModularClassResolver prefixes class names with >> the slot and module name, and can load a class from any module. Maybe >> we could also allow a slot:module:class format everywhere the >> configuration currently accepts a class name? > > That could be handy. I'm about to create a JIRA on this, so I'll add it as an idea to it. > >> >>> 2. After doing that, I had to register the protofile and associated classes remotely in the server. Again, there's no out of the box mechanism for that, so I created a remote server task that would do that [3]. >>> >>> 3. Finally, with all that in place, I was able to complete the WordCount test [4] with a final caveat: the return of the word count, and words protofile registration, tasks return objects that are not marshalled by the compatibility marshaller, so I had to make sure that the remote cache manager used for those tasks uses the default marshaller. >>> >>> Clearly we need to improve on this, and we have plans to address these issues (with new upcoming transcoding capabilities), but I thought it'd be worth mentioning the problems found in case anyone else encounters them before transcoding is in place. >>> >>> Cheers, >>> >>> [1] https://github.com/galderz/datagrid-patterns/blob/master/server-config/domain/domain.xml#L139 >>> [2] https://github.com/galderz/datagrid-patterns/blob/master/server-config/org.infinispan.main_module.xml#L18 >>> [3] https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-server/src/main/java/test/WordsProtoTask.java >>> [4] https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-client/src/test/java/test/WordCountTest.java >>> -- >>> Galder Zamarre?o >>> Infinispan, Red Hat >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From karesti at redhat.com Mon Apr 10 06:25:35 2017 From: karesti at redhat.com (Katia Aresti) Date: Mon, 10 Apr 2017 12:25:35 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: <09134fd7-bef1-3477-a37b-795bf3979f40@redhat.com> Message-ID: Hi all I've moved the discussion to infinispan-designs https://github.com/infinispan/infinispan-designs/pull/3 I think I put everything we said on this thread. Let me know if I forgot something important on the PR. Let's continue the design on Github, should be nicer to follow ! :) Cheers Katia On Thu, Apr 6, 2017 at 2:29 PM, Katia Aresti wrote: > What if we move the discussion to https://github.com/infinispan/ > infinispan-designs ? > > > Katia > > On Thu, Apr 6, 2017 at 11:04 AM, Radim Vansa wrote: > >> On 04/06/2017 12:15 AM, Katia Aresti wrote: >> > >> > >> > On Wed, Apr 5, 2017 at 9:56 AM, Radim Vansa > > > wrote: >> > >> > On 04/04/2017 06:40 PM, William Burns wrote: >> > > >> > > >> > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti > > >> > > >> wrote: >> > > >> > > Hi all, >> > > >> > > As you probably know, Will and I are working on the vert-x >> > > infinispan integration [1], where the primary goal is to make >> > > infinispan the default cluster management of vert-x. (yeah!) >> > > Vert-x needs support for an Async Multimap. Today's >> > implementation >> > > is a wrapper on a normal Cache where only Cache Key's are >> > used to >> > > implement the multi map [2]. >> > > This is not very efficient, so after trying some other >> > alternative >> > > implementations [3] that don't fully work (injection not >> > working), >> > > Will and I have come to the conclusion that it might be a good >> > > idea to start having our own native CacheMultimap. This first >> > > multimap won't support duplicate values on key's. >> > > >> > > As a quick start, the smallest multimap we need should >> implement >> > > the following interface : >> > > >> > > I agree that having a very slim API to start should be better >> > since we >> > > know how much trouble we get into implementing a very large API >> like >> > > ConcurrentMap :) >> > > >> > > public interface CacheMultimap { >> > > >> > >> > I don't see anything async in this interface. If that's async, >> provide >> > CompletableFuture return values. >> > I am also considering if we want any fire & forget variants for >> these >> > operations, but since we have to do retries to achieve consistency >> > (and >> > therefore we need some messages from owners to originator), I >> wouldn't >> > include them. >> > >> > >> > Today's vert-x API calls the vertx.executeBlocking(future => cache...) >> > >> > I considered the option of CompletableFuture, but for simplicity I >> > suggested the basic method. >> > Today's CacheAPI makes a difference between "put" and "putAsync". >> > Would you call the interface CacheMultimapAsync or CacheMultimap with >> > addAsyc method ? >> >> "In a perfect world, there will be no war or hunger, all APIs will be >> written asynchronously and bunny rabbits will skip hand-in-hand with >> baby lambs across sunny green meadows." (quoting Vert.x docs) >> >> While minimalistic API is a good way to start, it shouldn't contain >> anything we'd want to get rid of in close future. And especially since >> the main drive for multimaps is Vert.x which consumes asynchronous APIs >> (and has support for legacy synchronous APIs, the executeBlocking >> method), we should have the design adapted to that from the beginning. >> >> CompletableFuture is not a rocket science, and you can use the already >> asynchronous Infinispan internals. >> >> I don't think we should have two interfaces, I believe that single >> interface with async methods only is absolutely sufficient. Though I >> wouldn't add the *Async suffix at all there. If someone wants to execute >> the methods synchronously he can call .get() or .join() - just 6/7 >> characters more. >> >> > >> > > V put(K key,V value); >> > > >> > > This should probably return a boolean or Void. I am leaning >> towards >> > > the first, but I am open either way. >> > >> > I would rather call this "add", as vert-x does. CompletableFuture as >> > return type here will allow to easily register the handler >> > >> > >> > -1 I prefer keeping "put" name because it is still a Map and makes >> > more sense to me considering the actual Cache API too. The return type >> > V was a transcription mistake, it should be void for me, as Will >> > pointed out. >> >> To me "put" is linked with overwriting the previous value, while you add >> to the underlying collection or create a new single-element one. But >> whatever, I care more about the return values :) >> >> R. >> >> > >> > >> > > Collection get(K key); >> > > >> > > boolean remove(K key,V value); >> > > >> > > We probably want a `boolean remove(K key)` method as well that >> > removes >> > > all values mapped to the given key. >> > >> > What about "reset(key)"? >> > >> > >> > > } >> > > >> > > CacheMultimapImpl will be a wrapper on a normal Cache, >> > similar to [3]. >> > > >> > > We could add a new method in EmbeddedCacheManager.java >> > > >> > > CacheMultimap getCacheMultimap(String cacheName, >> > > boolean createIfAbsent); >> > > >> > > >> > > I was thinking maybe this would exist in a separate module >> > (outside of >> > > core)? or class that wraps (similar to DistributedExecutor) >> instead. >> > > My worry is about transactions, since the entry point to that is >> > > through Cache interface. The other option is we could add a >> > `getCache` >> > > method on the `CacheMultiMap`. >> > >> > +1 Since the names of multimaps and maps will clash, we shouldn't >> hide >> > that the underlying implementation is a Cache, so I'd suggest >> > something like >> > >> > static CacheMultimap CacheMultimapFactory.get(Cache> > Object> c) { ... } >> > >> > > >> > > >> > > Implementation will create a cache as always and return a new >> > > CacheMultimapImpl(cache). >> > > >> > > What do you think ? Please fell free to suggest any other >> > > alternative or idea. >> > > >> > > Cheers >> > > >> > > Katia >> > > >> > > [1] https://github.com/vert-x3/vertx-infinispan >> > >> > > >> > > [2] >> > > >> > https://github.com/vert-x3/vertx-infinispan/blob/master/src >> /main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAs >> yncMultiMap.java >> > > c/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanA >> syncMultiMap.java> >> > > >> > > [3] >> > https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c >> > >> > > _______________________________________________ >> > > infinispan-dev mailing list >> > > infinispan-dev at lists.jboss.org >> > >> > > > > >> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > > >> > > >> > > >> > > _______________________________________________ >> > > infinispan-dev mailing list >> > > infinispan-dev at lists.jboss.org >> > >> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > >> > >> > -- >> > Radim Vansa > >> > JBoss Performance Team >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev at lists.jboss.org > boss.org> >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > >> > >> > >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> -- >> Radim Vansa >> JBoss Performance Team >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170410/33cf0c67/attachment-0001.html From rvansa at redhat.com Mon Apr 10 06:39:41 2017 From: rvansa at redhat.com (Radim Vansa) Date: Mon, 10 Apr 2017 12:39:41 +0200 Subject: [infinispan-dev] Streams & missing segments Message-ID: <937458f4-04f2-f61a-c816-f8d1e119b468@redhat.com> Hi Will, rebasing scattered cache PR I've found a test failure when handling streams, and I'd like to ask you guidance, how to address it. My problem is that temporarily primary owners can be unknown (when a node crashes), but streams assume that there's always a primary owner. Therefore, a remote streams operation must be delayed until a topology arrives where the new owner is decided. Could you suggest how should I adapt the code? ClusteredStreamManagerImpl currently does not hold DistributionManager reference, ConsistenHash is passed down from DistributedCacheStream, and it seems that segments are suspected after any unsuccessful response. Which components should react to topology changes? Thanks! Radim -- Radim Vansa JBoss Performance Team From sanne at infinispan.org Mon Apr 10 07:00:14 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 10 Apr 2017 12:00:14 +0100 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: <09134fd7-bef1-3477-a37b-795bf3979f40@redhat.com> Message-ID: On 10 April 2017 at 11:25, Katia Aresti wrote: > Hi all > > I've moved the discussion to infinispan-designs > > https://github.com/infinispan/infinispan-designs/pull/3 > > I think I put everything we said on this thread. Let me know if I forgot > something important on the PR. Let's continue the design on Github, should > be nicer to follow ! :) Great idea! I often get a bit lost in long mailing list threads. Thanks, Sanne > > Cheers > > Katia > > On Thu, Apr 6, 2017 at 2:29 PM, Katia Aresti wrote: >> >> What if we move the discussion to >> https://github.com/infinispan/infinispan-designs ? >> >> >> Katia >> >> On Thu, Apr 6, 2017 at 11:04 AM, Radim Vansa wrote: >>> >>> On 04/06/2017 12:15 AM, Katia Aresti wrote: >>> > >>> > >>> > On Wed, Apr 5, 2017 at 9:56 AM, Radim Vansa >> > > wrote: >>> > >>> > On 04/04/2017 06:40 PM, William Burns wrote: >>> > > >>> > > >>> > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti >> > >>> > > >> wrote: >>> > > >>> > > Hi all, >>> > > >>> > > As you probably know, Will and I are working on the vert-x >>> > > infinispan integration [1], where the primary goal is to make >>> > > infinispan the default cluster management of vert-x. (yeah!) >>> > > Vert-x needs support for an Async Multimap. Today's >>> > implementation >>> > > is a wrapper on a normal Cache where only Cache Key's are >>> > used to >>> > > implement the multi map [2]. >>> > > This is not very efficient, so after trying some other >>> > alternative >>> > > implementations [3] that don't fully work (injection not >>> > working), >>> > > Will and I have come to the conclusion that it might be a >>> > good >>> > > idea to start having our own native CacheMultimap. This first >>> > > multimap won't support duplicate values on key's. >>> > > >>> > > As a quick start, the smallest multimap we need should >>> > implement >>> > > the following interface : >>> > > >>> > > I agree that having a very slim API to start should be better >>> > since we >>> > > know how much trouble we get into implementing a very large API >>> > like >>> > > ConcurrentMap :) >>> > > >>> > > public interface CacheMultimap { >>> > > >>> > >>> > I don't see anything async in this interface. If that's async, >>> > provide >>> > CompletableFuture return values. >>> > I am also considering if we want any fire & forget variants for >>> > these >>> > operations, but since we have to do retries to achieve consistency >>> > (and >>> > therefore we need some messages from owners to originator), I >>> > wouldn't >>> > include them. >>> > >>> > >>> > Today's vert-x API calls the vertx.executeBlocking(future => cache...) >>> > >>> > I considered the option of CompletableFuture, but for simplicity I >>> > suggested the basic method. >>> > Today's CacheAPI makes a difference between "put" and "putAsync". >>> > Would you call the interface CacheMultimapAsync or CacheMultimap with >>> > addAsyc method ? >>> >>> "In a perfect world, there will be no war or hunger, all APIs will be >>> written asynchronously and bunny rabbits will skip hand-in-hand with >>> baby lambs across sunny green meadows." (quoting Vert.x docs) >>> >>> While minimalistic API is a good way to start, it shouldn't contain >>> anything we'd want to get rid of in close future. And especially since >>> the main drive for multimaps is Vert.x which consumes asynchronous APIs >>> (and has support for legacy synchronous APIs, the executeBlocking >>> method), we should have the design adapted to that from the beginning. >>> >>> CompletableFuture is not a rocket science, and you can use the already >>> asynchronous Infinispan internals. >>> >>> I don't think we should have two interfaces, I believe that single >>> interface with async methods only is absolutely sufficient. Though I >>> wouldn't add the *Async suffix at all there. If someone wants to execute >>> the methods synchronously he can call .get() or .join() - just 6/7 >>> characters more. >>> >>> > >>> > > V put(K key,V value); >>> > > >>> > > This should probably return a boolean or Void. I am leaning >>> > towards >>> > > the first, but I am open either way. >>> > >>> > I would rather call this "add", as vert-x does. CompletableFuture >>> > as >>> > return type here will allow to easily register the handler >>> > >>> > >>> > -1 I prefer keeping "put" name because it is still a Map and makes >>> > more sense to me considering the actual Cache API too. The return type >>> > V was a transcription mistake, it should be void for me, as Will >>> > pointed out. >>> >>> To me "put" is linked with overwriting the previous value, while you add >>> to the underlying collection or create a new single-element one. But >>> whatever, I care more about the return values :) >>> >>> R. >>> >>> > >>> > >>> > > Collection get(K key); >>> > > >>> > > boolean remove(K key,V value); >>> > > >>> > > We probably want a `boolean remove(K key)` method as well that >>> > removes >>> > > all values mapped to the given key. >>> > >>> > What about "reset(key)"? >>> > >>> > >>> > > } >>> > > >>> > > CacheMultimapImpl will be a wrapper on a normal Cache, >>> > similar to [3]. >>> > > >>> > > We could add a new method in EmbeddedCacheManager.java >>> > > >>> > > CacheMultimap getCacheMultimap(String cacheName, >>> > > boolean createIfAbsent); >>> > > >>> > > >>> > > I was thinking maybe this would exist in a separate module >>> > (outside of >>> > > core)? or class that wraps (similar to DistributedExecutor) >>> > instead. >>> > > My worry is about transactions, since the entry point to that is >>> > > through Cache interface. The other option is we could add a >>> > `getCache` >>> > > method on the `CacheMultiMap`. >>> > >>> > +1 Since the names of multimaps and maps will clash, we shouldn't >>> > hide >>> > that the underlying implementation is a Cache, so I'd suggest >>> > something like >>> > >>> > static CacheMultimap CacheMultimapFactory.get(Cache>> > Object> c) { ... } >>> > >>> > > >>> > > >>> > > Implementation will create a cache as always and return a new >>> > > CacheMultimapImpl(cache). >>> > > >>> > > What do you think ? Please fell free to suggest any other >>> > > alternative or idea. >>> > > >>> > > Cheers >>> > > >>> > > Katia >>> > > >>> > > [1] https://github.com/vert-x3/vertx-infinispan >>> > >>> > > >>> > > [2] >>> > > >>> > >>> > https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java >>> > >>> > >>> > > >>> > > [3] >>> > https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c >>> > >>> > > _______________________________________________ >>> > > infinispan-dev mailing list >>> > > infinispan-dev at lists.jboss.org >>> > >>> > >> > > >>> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> > >>> > > >>> > > >>> > > >>> > > _______________________________________________ >>> > > infinispan-dev mailing list >>> > > infinispan-dev at lists.jboss.org >>> > >>> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> > >>> > >>> > >>> > -- >>> > Radim Vansa > >>> > JBoss Performance Team >>> > >>> > _______________________________________________ >>> > infinispan-dev mailing list >>> > infinispan-dev at lists.jboss.org >>> > >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> > >>> > >>> > >>> > >>> > >>> > _______________________________________________ >>> > infinispan-dev mailing list >>> > infinispan-dev at lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> -- >>> Radim Vansa >>> JBoss Performance Team >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From dan.berindei at gmail.com Mon Apr 10 11:03:31 2017 From: dan.berindei at gmail.com (Dan Berindei) Date: Mon, 10 Apr 2017 18:03:31 +0300 Subject: [infinispan-dev] Weekly IRC meeting logs 2017-04-10 Message-ID: Hi everyone We had another weekly IRC meeting today, and the logs are here: http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2017/infinispan.2017-04-10-14.00.html Cheers Dan From mudokonman at gmail.com Mon Apr 10 11:56:19 2017 From: mudokonman at gmail.com (William Burns) Date: Mon, 10 Apr 2017 15:56:19 +0000 Subject: [infinispan-dev] Streams & missing segments In-Reply-To: <937458f4-04f2-f61a-c816-f8d1e119b468@redhat.com> References: <937458f4-04f2-f61a-c816-f8d1e119b468@redhat.com> Message-ID: Comments inline To be honest it might be easier to talk on IRC sometime since I am not sure the exact part you are trying to work on now. I am guessing you are reusing the ClusterStreamManager and LocalStreamManager parts and adapting as needed but I don't know for sure. On Mon, Apr 10, 2017 at 6:41 AM Radim Vansa wrote: > Hi Will, > > rebasing scattered cache PR I've found a test failure when handling > streams, and I'd like to ask you guidance, how to address it. > > My problem is that temporarily primary owners can be unknown (when a > node crashes), but streams assume that there's always a primary owner. > Therefore, a remote streams operation must be delayed until a topology > arrives where the new owner is decided. > One thing to reduce this is if a node was primary there is a special exception. If this node becomes a backup owner after being primary this is fine. The segment is only suspected if it loses ownership completely. There is no waiting currently it will submit the request again until it eventually gets what it wants. (this could be improved) There is a wait though when the originator has a newer topology if a remote node doesn't yet have the updated topology (StreamRequestCommand implements TopologyAffectedCommand). > Could you suggest how should I adapt the code? > The ClusterStreamManager only does one invocation and stores the results from that invocation. The caller then has to adapt the next call with the segments it still needs and call back into the ClusterStreamManager if needed (segments could be owned locally now). > ClusteredStreamManagerImpl currently does not hold DistributionManager > reference, ConsistenHash is passed down from DistributedCacheStream, and > it seems that segments are suspected after any unsuccessful response. > Which components should react to topology changes? > The only component currently that reacts to topology change is in the LocalStreamManagerImpl which registers a listener to detect when a segment is lost. This is done on remote nodes only. When this segment is lost it will send that those segments weren't completed back to the requester (ClusterStreamManager) on its next response. As you mentioned any unsuccessful response is treated as suspected, since we can't really trust that node, so there is no code on originator required to listen for topology changes. > > Thanks! > > Radim > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170410/e4e51043/attachment.html From ttarrant at redhat.com Mon Apr 10 14:47:25 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Mon, 10 Apr 2017 20:47:25 +0200 Subject: [infinispan-dev] Weekly IRC meeting logs 2017-04-10 In-Reply-To: References: Message-ID: <37e55c28-77ee-806e-4fe4-58506c6ad220@redhat.com> My activities: - Server restructuring proposal [1] - ISPN-7712 Common Name role mapper case, PR @ [2] - ISPN-7706 If mech EXTERNAL and no callback supplied, use VoidCallbackHandler, PR coming soon - ISPN-7707 Finer configuration options for keystores/truststores, PR coming soon - ISPN-7709 Generate certificates/keystores on the fly so they work on all JDKs, PR coming soon I'd like to finish these and also fix the ISPN-7678 PR [3] This week will be short: PTO Thursday and Friday, holiday on Easter Monday. Tristan [1] https://github.com/infinispan/infinispan-designs/blob/master/Server%20Restructuring.md [2] https://github.com/infinispan/infinispan/pull/5066 [3] https://github.com/infinispan/infinispan/pull/5032 On 10/04/2017 17:03, Dan Berindei wrote: > Hi everyone > > We had another weekly IRC meeting today, and the logs are here: > > http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2017/infinispan.2017-04-10-14.00.html > > Cheers > Dan > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From rvansa at redhat.com Tue Apr 11 03:02:09 2017 From: rvansa at redhat.com (Radim Vansa) Date: Tue, 11 Apr 2017 09:02:09 +0200 Subject: [infinispan-dev] Weekly IRC meeting logs 2017-04-10 In-Reply-To: <37e55c28-77ee-806e-4fe4-58506c6ad220@redhat.com> References: <37e55c28-77ee-806e-4fe4-58506c6ad220@redhat.com> Message-ID: <254617aa-bf44-73c7-78de-d3d689478482@redhat.com> I thought that 9.0 should have been branched off yesterday and master set to 9.1, what's the current plan (time-wise)? R. On 04/10/2017 08:47 PM, Tristan Tarrant wrote: > My activities: > > - Server restructuring proposal [1] > - ISPN-7712 Common Name role mapper case, PR @ [2] > - ISPN-7706 If mech EXTERNAL and no callback supplied, use > VoidCallbackHandler, PR coming soon > - ISPN-7707 Finer configuration options for keystores/truststores, PR > coming soon > - ISPN-7709 Generate certificates/keystores on the fly so they work on > all JDKs, PR coming soon > > I'd like to finish these and also fix the ISPN-7678 PR [3] > > This week will be short: PTO Thursday and Friday, holiday on Easter Monday. > > Tristan > > [1] > https://github.com/infinispan/infinispan-designs/blob/master/Server%20Restructuring.md > [2] https://github.com/infinispan/infinispan/pull/5066 > [3] https://github.com/infinispan/infinispan/pull/5032 > > > On 10/04/2017 17:03, Dan Berindei wrote: >> Hi everyone >> >> We had another weekly IRC meeting today, and the logs are here: >> >> http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2017/infinispan.2017-04-10-14.00.html >> >> Cheers >> Dan >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> -- Radim Vansa JBoss Performance Team From rvansa at redhat.com Tue Apr 11 02:59:37 2017 From: rvansa at redhat.com (Radim Vansa) Date: Tue, 11 Apr 2017 08:59:37 +0200 Subject: [infinispan-dev] Streams & missing segments In-Reply-To: References: <937458f4-04f2-f61a-c816-f8d1e119b468@redhat.com> Message-ID: <542a96e6-4142-037b-7128-a36d0859bece@redhat.com> Yes, I am trying to reuse those components and I've did only minor modifications so far. Without topology change that works. But I understand that you had some concept behind these components and I am looking for a natural extension. So, if I need to add waiting, is it acceptable to add StateTransferLock and DistributionManager into ClusterStreamManager? That sounds like the most convenient way, but then I should remove CH as the parameter from calls from DistributedCacheStream, because routing information is already available in ClusterStreamManager. Radim On 04/10/2017 05:56 PM, William Burns wrote: > Comments inline > > To be honest it might be easier to talk on IRC sometime since I am not > sure the exact part you are trying to work on now. I am guessing you > are reusing the ClusterStreamManager and LocalStreamManager parts and > adapting as needed but I don't know for sure. > > On Mon, Apr 10, 2017 at 6:41 AM Radim Vansa > wrote: > > Hi Will, > > rebasing scattered cache PR I've found a test failure when handling > streams, and I'd like to ask you guidance, how to address it. > > My problem is that temporarily primary owners can be unknown (when a > node crashes), but streams assume that there's always a primary owner. > Therefore, a remote streams operation must be delayed until a topology > arrives where the new owner is decided. > > > One thing to reduce this is if a node was primary there is a special > exception. If this node becomes a backup owner after being primary > this is fine. The segment is only suspected if it loses ownership > completely. > > There is no waiting currently it will submit the request again until > it eventually gets what it wants. (this could be improved) There is a > wait though when the originator has a newer topology if a remote node > doesn't yet have the updated topology (StreamRequestCommand implements > TopologyAffectedCommand). > > > Could you suggest how should I adapt the code? > > > The ClusterStreamManager only does one invocation and stores the > results from that invocation. The caller then has to adapt the next > call with the segments it still needs and call back into the > ClusterStreamManager if needed (segments could be owned locally now). > > ClusteredStreamManagerImpl currently does not hold DistributionManager > reference, ConsistenHash is passed down from > DistributedCacheStream, and > it seems that segments are suspected after any unsuccessful response. > Which components should react to topology changes? > > > The only component currently that reacts to topology change is in the > LocalStreamManagerImpl which registers a listener to detect when a > segment is lost. This is done on remote nodes only. When this segment > is lost it will send that those segments weren't completed back to the > requester (ClusterStreamManager) on its next response. > > As you mentioned any unsuccessful response is treated as suspected, > since we can't really trust that node, so there is no code on > originator required to listen for topology changes. > > > Thanks! > > Radim > > > -- > Radim Vansa > > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From mgencur at redhat.com Tue Apr 11 08:33:32 2017 From: mgencur at redhat.com (Martin Gencur) Date: Tue, 11 Apr 2017 14:33:32 +0200 Subject: [infinispan-dev] HotRod client TCK Message-ID: Hello all, we have been working on https://issues.jboss.org/browse/ISPN-7120. Anna has finished the first step from the JIRA - collecting information about tests in the Java HotRod client test suite (including server integration tests) and it is now prepared for wider review. She created a spreadsheet [1]. The spread sheet includes for each Java test its name, the suggested target package in the TCK, whether to include it in the TCK or not, and some other notes. The suggested package also poses grouping for the tests (e.g. tck.query, tck.near, tck.xsite, ...) Let me add that right now the goal is not to create a true TCK [2]. The goal is to make sure that all implementations of the HotRod protocol have sufficient test coverage and possibly the same server side of the client-server test (including the server version and configuration). What are the next step? * Please review the list (at least a quick look) and see if some of the tests which are NOT suggested for the TCK should be added or vice versa. * I suppose the next step would then be to check other implementations (C#, C++, NodeJS, ..) and identify tests which are missing there (there will surely be some). * Gradually implement the missing tests in the other implementations Note: Here we should ensure that the server is configured in the same way for all implementations. One way to achieve this (thanks Anna for suggestion!) is to have a shell/batch scripts for CLI which would be executed before the tests. This can probably be done for all impls. and both UNIX/WINDOWS. I also realize that my PR for ISPN [3] becomes useless because it uses Creaper (Java) and we need a language-neutral solution for configuring the server. Some other notes: * there are some duplicated tests in hotrod-client and server integration test suites, in this case it probably makes sense to only include in the TCK the server integration test * tests from the hotrod-client module which are supposed to be part of the TCK should be copied to the server integration test suite one day (possibly later) Please let us know what you think. Thanks, Martin [1] https://docs.google.com/spreadsheets/d/1bZBBi5m4oLL4lBTZhdRbIC_EA0giQNDZWzFNPWrF5G4/edit#gid=0 [2] https://en.wikipedia.org/wiki/Technology_Compatibility_Kit [3] https://github.com/infinispan/infinispan/pull/5012 From rvansa at redhat.com Tue Apr 11 09:57:26 2017 From: rvansa at redhat.com (Radim Vansa) Date: Tue, 11 Apr 2017 15:57:26 +0200 Subject: [infinispan-dev] HotRod client TCK In-Reply-To: References: Message-ID: <7bba7850-52fa-1b98-45da-603f1443cc34@redhat.com> Since these tests use real server(s), many of them test not only the client behaviour (generating correct commands according to the protocol), but server, too. While this is practical (we need to test server somehow, too), there's nothing all the tests across languages will have physically in common and all comparison is prone to human error. If we want to test various implementations of the client, maybe it would make sense to give the clients a fake server that will have just a scenario of expected commands to receive and pre-defined responses. We could use audit log to generate such scenario based on the actual Java tests. But then we'd have to test the actual behaviour on server, and we'd need a way to issue the commands. Just my 2c Radim On 04/11/2017 02:33 PM, Martin Gencur wrote: > Hello all, > we have been working on https://issues.jboss.org/browse/ISPN-7120. > > Anna has finished the first step from the JIRA - collecting information > about tests in the Java HotRod client test suite (including server > integration tests) and it is now prepared for wider review. > > She created a spreadsheet [1]. The spread sheet includes for each Java > test its name, the suggested target package in the TCK, whether to > include it in the TCK or not, and some other notes. The suggested > package also poses grouping for the tests (e.g. tck.query, tck.near, > tck.xsite, ...) > > Let me add that right now the goal is not to create a true TCK [2]. The > goal is to make sure that all implementations of the HotRod protocol > have sufficient test coverage and possibly the same server side of the > client-server test (including the server version and configuration). > > What are the next step? > > * Please review the list (at least a quick look) and see if some of the > tests which are NOT suggested for the TCK should be added or vice versa. > * I suppose the next step would then be to check other implementations > (C#, C++, NodeJS, ..) and identify tests which are missing there (there > will surely be some). > * Gradually implement the missing tests in the other implementations > Note: Here we should ensure that the server is configured in the same > way for all implementations. One way to achieve this (thanks Anna for > suggestion!) is to have a shell/batch scripts for CLI which would be > executed before the tests. This can probably be done for all impls. and > both UNIX/WINDOWS. I also realize that my PR for ISPN [3] becomes > useless because it uses Creaper (Java) and we need a language-neutral > solution for configuring the server. > > Some other notes: > * there are some duplicated tests in hotrod-client and server > integration test suites, in this case it probably makes sense to only > include in the TCK the server integration test > * tests from the hotrod-client module which are supposed to be part of > the TCK should be copied to the server integration test suite one day > (possibly later) > > Please let us know what you think. > > Thanks, > Martin > > > [1] > https://docs.google.com/spreadsheets/d/1bZBBi5m4oLL4lBTZhdRbIC_EA0giQNDZWzFNPWrF5G4/edit#gid=0 > [2] https://en.wikipedia.org/wiki/Technology_Compatibility_Kit > [3] https://github.com/infinispan/infinispan/pull/5012 > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From ttarrant at redhat.com Tue Apr 11 15:05:19 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Tue, 11 Apr 2017 21:05:19 +0200 Subject: [infinispan-dev] Branching 9.0.x Message-ID: <21b38883-dca8-59bc-adbc-09d4179f0aa5@redhat.com> Hi all, I am going to branch 9.0.x tomorrow at 12:00 CEST. Let me know if I should delay this. Tristan -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From vjuranek at redhat.com Wed Apr 12 03:13:09 2017 From: vjuranek at redhat.com (Vojtech Juranek) Date: Wed, 12 Apr 2017 09:13:09 +0200 Subject: [infinispan-dev] In Memory Data Grid Patterns Demos from Devoxx France! In-Reply-To: <2000E4DB-7A7F-45E4-8833-7EB3A1C60DF0@redhat.com> References: <2000E4DB-7A7F-45E4-8833-7EB3A1C60DF0@redhat.com> Message-ID: <2872808.gplZBrWE3V@dhcp-10-40-5-95.brq.redhat.com> Thanks for sharing, nice demos! On a similar data processing note, here [1] is my demo from DevConf how to use ISPN in machine learning pipeline (here the data is not processed direcly in ISPN but in TensorFlow) [1] https://github.com/vjuranek/tf-ispn-demo On p?tek 7. dubna 2017 10:48:23 CEST Galder Zamarre?o wrote: > Hi all, > > I've just got back from Devoxx France where Emmanuel and I presented about > in-memory data grid use cases, and during this talk we presented a couple > of demos on using Infinispan for offline analytics and real-time data > processing. > > I've just created a new blog post with some very quick instructions for you > to run these demos: > http://blog.infinispan.org/2017/04/in-memory-data-grid-patterns-demos-from. > html > > Give them a try and let us know what you think! > > Cheers, > -- > Galder Zamarre?o > Infinispan, Red Hat > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: This is a digitally signed message part. Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170412/3b63456d/attachment.bin From karesti at redhat.com Wed Apr 12 06:11:25 2017 From: karesti at redhat.com (Katia Aresti) Date: Wed, 12 Apr 2017 12:11:25 +0200 Subject: [infinispan-dev] to be a command, or not to be a command, that is the question Message-ID: Hi all, As you might know I'm working since my arrival, among other things, on ISPN-5728 Jira [1], where the idea is to override the default ConcurrentMap methods that are missing in CacheImpl (merge, replaceAll, compute ... ) I've created a pull-request [2] for compute, computeIfAbsent and computeIfPresent methods, creating two new commands. By the way, I did the same thing for the merge method in a branch that I haven't pull requested yet. There is an opposite view between Radim and Will concerning the implementation of these methods. To make it short : In one side Will considers compute/merge best implementation should be as a new Command (so what is already done) In the other side, Radim considers adding another command is not necessary as we could simple implement these methods using ReadWriteKeyCommand The detailed discussion and arguments of both sides is on GitHub [2] Before moving forward and making any choice by myself, I would like to hear your opinions. For the record, it doesn't bother me redoing everything if most people think like Radim because working on commands has helped me to learn and understand more about infinispan internals, so this hasn't been a waste of time for me. Katia [1] https://issues.jboss.org/browse/ISPN-5728 [2] https://github.com/infinispan/infinispan/pull/5046 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170412/ef5d414b/attachment-0001.html From ttarrant at redhat.com Wed Apr 12 07:40:45 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Wed, 12 Apr 2017 13:40:45 +0200 Subject: [infinispan-dev] Branching 9.0.x In-Reply-To: <21b38883-dca8-59bc-adbc-09d4179f0aa5@redhat.com> References: <21b38883-dca8-59bc-adbc-09d4179f0aa5@redhat.com> Message-ID: <00e08f0f-b460-de1f-08b1-e8b663c04054@redhat.com> master is now 9.1.0-SNAPSHOT. 9.0.x is 9.0.1-SNAPSHOT Tristan On 11/04/2017 21:05, Tristan Tarrant wrote: > Hi all, > > I am going to branch 9.0.x tomorrow at 12:00 CEST. Let me know if I > should delay this. > > Tristan -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From galder at redhat.com Thu Apr 13 01:38:19 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Thu, 13 Apr 2017 07:38:19 +0200 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: Message-ID: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> Hi all, As per some discussions we had yesterday on IRC w/ Tristan, Gustavo and Sebastian, I've created a docker image snapshot that reverts the change stop protected caches from requiring security enabled [1]. In other words, I've removed [2]. The reason for temporarily doing that is because with the change as is, the changes required for a default server distro require that the entire cache manager's security is enabled. This is in turn creates a lot of problems with health and running checks used by Kubernetes/OpenShift amongst other things. Judging from our discussions on IRC, the idea is for such change to be present in 9.0.1, but I'd like to get final confirmation from Tristan et al. Cheers, [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ (9.0.1-SNAPSHOT tag for anyone interested) [2] https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/java/org/infinispan/server/hotrod/CacheDecodeContext.java#L114-L118 -- Galder Zamarre?o Infinispan, Red Hat > On 30 Mar 2017, at 14:25, Tristan Tarrant wrote: > > Dear all, > > after a mini chat on IRC, I wanted to bring this to everybody's attention. > > We should make the Hot Rod endpoint require authentication in the > out-of-the-box configuration. > The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL > mechanism against the ApplicationRealm and require users to run the > add-user script. > This would achieve two goals: > - secure out-of-the-box configuration, which is always a good idea > - access to the "protected" schema and script caches which is prevented > when not on loopback on non-authenticated endpoints. > > Tristan > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From gustavo at infinispan.org Thu Apr 13 03:50:37 2017 From: gustavo at infinispan.org (Gustavo Fernandes) Date: Thu, 13 Apr 2017 08:50:37 +0100 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> Message-ID: On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o wrote: > Hi all, > > As per some discussions we had yesterday on IRC w/ Tristan, Gustavo and > Sebastian, I've created a docker image snapshot that reverts the change > stop protected caches from requiring security enabled [1]. > > In other words, I've removed [2]. The reason for temporarily doing that is > because with the change as is, the changes required for a default server > distro require that the entire cache manager's security is enabled. This is > in turn creates a lot of problems with health and running checks used by > Kubernetes/OpenShift amongst other things. > > Judging from our discussions on IRC, the idea is for such change to be > present in 9.0.1, but I'd like to get final confirmation from Tristan et al. > > +1 Regarding the "security by default" discussion, I think we should ship configurations cloud.xml, clustered.xml and standalone.xml with security enabled and disabled variants, and let users decide which one to pick based on the use case. Gustavo. > Cheers, > > [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ > (9.0.1-SNAPSHOT tag for anyone interested) > [2] https://github.com/infinispan/infinispan/blob/master/server/ > hotrod/src/main/java/org/infinispan/server/hotrod/ > CacheDecodeContext.java#L114-L118 > -- > Galder Zamarre?o > Infinispan, Red Hat > > > On 30 Mar 2017, at 14:25, Tristan Tarrant wrote: > > > > Dear all, > > > > after a mini chat on IRC, I wanted to bring this to everybody's > attention. > > > > We should make the Hot Rod endpoint require authentication in the > > out-of-the-box configuration. > > The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL > > mechanism against the ApplicationRealm and require users to run the > > add-user script. > > This would achieve two goals: > > - secure out-of-the-box configuration, which is always a good idea > > - access to the "protected" schema and script caches which is prevented > > when not on loopback on non-authenticated endpoints. > > > > Tristan > > -- > > Tristan Tarrant > > Infinispan Lead > > JBoss, a division of Red Hat > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170413/d05c8e61/attachment.html From galder at redhat.com Thu Apr 13 05:17:12 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Thu, 13 Apr 2017 11:17:12 +0200 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> Message-ID: -- Galder Zamarre?o Infinispan, Red Hat > On 13 Apr 2017, at 09:50, Gustavo Fernandes wrote: > > On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o wrote: > Hi all, > > As per some discussions we had yesterday on IRC w/ Tristan, Gustavo and Sebastian, I've created a docker image snapshot that reverts the change stop protected caches from requiring security enabled [1]. > > In other words, I've removed [2]. The reason for temporarily doing that is because with the change as is, the changes required for a default server distro require that the entire cache manager's security is enabled. This is in turn creates a lot of problems with health and running checks used by Kubernetes/OpenShift amongst other things. > > Judging from our discussions on IRC, the idea is for such change to be present in 9.0.1, but I'd like to get final confirmation from Tristan et al. > > > +1 > > Regarding the "security by default" discussion, I think we should ship configurations cloud.xml, clustered.xml and standalone.xml with security enabled and disabled variants, and let users > decide which one to pick based on the use case. I think that's a better idea. We could by default have a secured one, but switching to an insecure configuration should be doable with minimal effort, e.g. just switching config file. As highlighted above, any secured configuration should work out-of-the-box with our docker images, e.g. WRT healthy/running checks. Cheers, > > Gustavo. > > > Cheers, > > [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ (9.0.1-SNAPSHOT tag for anyone interested) > [2] https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/java/org/infinispan/server/hotrod/CacheDecodeContext.java#L114-L118 > -- > Galder Zamarre?o > Infinispan, Red Hat > > > On 30 Mar 2017, at 14:25, Tristan Tarrant wrote: > > > > Dear all, > > > > after a mini chat on IRC, I wanted to bring this to everybody's attention. > > > > We should make the Hot Rod endpoint require authentication in the > > out-of-the-box configuration. > > The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL > > mechanism against the ApplicationRealm and require users to run the > > add-user script. > > This would achieve two goals: > > - secure out-of-the-box configuration, which is always a good idea > > - access to the "protected" schema and script caches which is prevented > > when not on loopback on non-authenticated endpoints. > > > > Tristan > > -- > > Tristan Tarrant > > Infinispan Lead > > JBoss, a division of Red Hat > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From slaskawi at redhat.com Thu Apr 13 10:13:07 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Thu, 13 Apr 2017 14:13:07 +0000 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> Message-ID: I still think it would be better to create an extra switch to run infinispan in "development mode". This means no authentication, no encryption, possibly with JGroups stack tuned for fast discovery (especially in Kubernetes) and a big warning saying "You are in development mode, do not use this in production". Just something very easy to get you going. On Thu, Apr 13, 2017 at 12:16 PM Galder Zamarre?o wrote: > > -- > Galder Zamarre?o > Infinispan, Red Hat > > > On 13 Apr 2017, at 09:50, Gustavo Fernandes > wrote: > > > > On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o > wrote: > > Hi all, > > > > As per some discussions we had yesterday on IRC w/ Tristan, Gustavo and > Sebastian, I've created a docker image snapshot that reverts the change > stop protected caches from requiring security enabled [1]. > > > > In other words, I've removed [2]. The reason for temporarily doing that > is because with the change as is, the changes required for a default server > distro require that the entire cache manager's security is enabled. This is > in turn creates a lot of problems with health and running checks used by > Kubernetes/OpenShift amongst other things. > > > > Judging from our discussions on IRC, the idea is for such change to be > present in 9.0.1, but I'd like to get final confirmation from Tristan et al. > > > > > > +1 > > > > Regarding the "security by default" discussion, I think we should ship > configurations cloud.xml, clustered.xml and standalone.xml with security > enabled and disabled variants, and let users > > decide which one to pick based on the use case. > > I think that's a better idea. > > We could by default have a secured one, but switching to an insecure > configuration should be doable with minimal effort, e.g. just switching > config file. > > As highlighted above, any secured configuration should work out-of-the-box > with our docker images, e.g. WRT healthy/running checks. > > Cheers, > > > > > Gustavo. > > > > > > Cheers, > > > > [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ > (9.0.1-SNAPSHOT tag for anyone interested) > > [2] > https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/java/org/infinispan/server/hotrod/CacheDecodeContext.java#L114-L118 > > -- > > Galder Zamarre?o > > Infinispan, Red Hat > > > > > On 30 Mar 2017, at 14:25, Tristan Tarrant wrote: > > > > > > Dear all, > > > > > > after a mini chat on IRC, I wanted to bring this to everybody's > attention. > > > > > > We should make the Hot Rod endpoint require authentication in the > > > out-of-the-box configuration. > > > The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL > > > mechanism against the ApplicationRealm and require users to run the > > > add-user script. > > > This would achieve two goals: > > > - secure out-of-the-box configuration, which is always a good idea > > > - access to the "protected" schema and script caches which is prevented > > > when not on loopback on non-authenticated endpoints. > > > > > > Tristan > > > -- > > > Tristan Tarrant > > > Infinispan Lead > > > JBoss, a division of Red Hat > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170413/54b2f484/attachment-0001.html From galder at redhat.com Thu Apr 13 17:31:03 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Thu, 13 Apr 2017 23:31:03 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: <09134fd7-bef1-3477-a37b-795bf3979f40@redhat.com> Message-ID: -- Galder Zamarre?o Infinispan, Red Hat > On 6 Apr 2017, at 11:04, Radim Vansa wrote: > > On 04/06/2017 12:15 AM, Katia Aresti wrote: >> >> >> On Wed, Apr 5, 2017 at 9:56 AM, Radim Vansa > > wrote: >> >> On 04/04/2017 06:40 PM, William Burns wrote: >>> >>> >>> On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti > >>> >> wrote: >>> >>> Hi all, >>> >>> As you probably know, Will and I are working on the vert-x >>> infinispan integration [1], where the primary goal is to make >>> infinispan the default cluster management of vert-x. (yeah!) >>> Vert-x needs support for an Async Multimap. Today's >> implementation >>> is a wrapper on a normal Cache where only Cache Key's are >> used to >>> implement the multi map [2]. >>> This is not very efficient, so after trying some other >> alternative >>> implementations [3] that don't fully work (injection not >> working), >>> Will and I have come to the conclusion that it might be a good >>> idea to start having our own native CacheMultimap. This first >>> multimap won't support duplicate values on key's. >>> >>> As a quick start, the smallest multimap we need should implement >>> the following interface : >>> >>> I agree that having a very slim API to start should be better >> since we >>> know how much trouble we get into implementing a very large API like >>> ConcurrentMap :) >>> >>> public interface CacheMultimap { >>> >> >> I don't see anything async in this interface. If that's async, provide >> CompletableFuture return values. >> I am also considering if we want any fire & forget variants for these >> operations, but since we have to do retries to achieve consistency >> (and >> therefore we need some messages from owners to originator), I wouldn't >> include them. >> >> >> Today's vert-x API calls the vertx.executeBlocking(future => cache...) >> >> I considered the option of CompletableFuture, but for simplicity I >> suggested the basic method. >> Today's CacheAPI makes a difference between "put" and "putAsync". >> Would you call the interface CacheMultimapAsync or CacheMultimap with >> addAsyc method ? > > "In a perfect world, there will be no war or hunger, all APIs will be > written asynchronously and bunny rabbits will skip hand-in-hand with > baby lambs across sunny green meadows." (quoting Vert.x docs) > > While minimalistic API is a good way to start, it shouldn't contain > anything we'd want to get rid of in close future. And especially since > the main drive for multimaps is Vert.x which consumes asynchronous APIs > (and has support for legacy synchronous APIs, the executeBlocking > method), we should have the design adapted to that from the beginning. Amen! > CompletableFuture is not a rocket science, and you can use the already > asynchronous Infinispan internals. Indeed! CompletableFuture is good. In hindsight, I would have maybe chosen java.util.concurrent.CompletionStage since it's more flexible (interface vs class), and doesn't bring in java.util.concurrent.Future which contains blocking methods. CompletableFuture/CompletionStage works for single returns. The bigger problem is when you want multiple returns asynchronously. Here a can of worms opens up, e.g. do you push the results? do you pull the results? For the functional map API, we experimented with a pull model using Traversable. A push model is harder to implement and non-trivial, and there you're getting into Rx territory. > I don't think we should have two interfaces, I believe that single > interface with async methods only is absolutely sufficient. ^ I'm not so sure actually... Both sets of methods are used for different use cases. Conceptually and from a user's perspective, I'd rather have separate interfaces since I'd not expect calls to both use cases to be interleaved. > Though I > wouldn't add the *Async suffix at all there. If someone wants to execute > the methods synchronously he can call .get() or .join() - just 6/7 > characters more. ^ We shouldn't promote calling Future.get() for asynchronous APIs since it goes against everything that async APIs stand for ;) > >> >>> V put(K key,V value); >>> >>> This should probably return a boolean or Void. I am leaning towards >>> the first, but I am open either way. >> >> I would rather call this "add", as vert-x does. CompletableFuture as >> return type here will allow to easily register the handler >> >> >> -1 I prefer keeping "put" name because it is still a Map and makes >> more sense to me considering the actual Cache API too. The return type >> V was a transcription mistake, it should be void for me, as Will >> pointed out. > > To me "put" is linked with overwriting the previous value, while you add > to the underlying collection or create a new single-element one. But > whatever, I care more about the return values :) > > R. > >> >> >>> Collection get(K key); >>> >>> boolean remove(K key,V value); >>> >>> We probably want a `boolean remove(K key)` method as well that >> removes >>> all values mapped to the given key. >> >> What about "reset(key)"? >> >> >>> } >>> >>> CacheMultimapImpl will be a wrapper on a normal Cache, >> similar to [3]. >>> >>> We could add a new method in EmbeddedCacheManager.java >>> >>> CacheMultimap getCacheMultimap(String cacheName, >>> boolean createIfAbsent); >>> >>> >>> I was thinking maybe this would exist in a separate module >> (outside of >>> core)? or class that wraps (similar to DistributedExecutor) instead. >>> My worry is about transactions, since the entry point to that is >>> through Cache interface. The other option is we could add a >> `getCache` >>> method on the `CacheMultiMap`. >> >> +1 Since the names of multimaps and maps will clash, we shouldn't hide >> that the underlying implementation is a Cache, so I'd suggest >> something like >> >> static CacheMultimap CacheMultimapFactory.get(Cache> Object> c) { ... } >> >>> >>> >>> Implementation will create a cache as always and return a new >>> CacheMultimapImpl(cache). >>> >>> What do you think ? Please fell free to suggest any other >>> alternative or idea. >>> >>> Cheers >>> >>> Katia >>> >>> [1] https://github.com/vert-x3/vertx-infinispan >> >>> >>> [2] >>> >> https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java >> >>> >>> [3] >> https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c >> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >> >> > > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> -- >> Radim Vansa > >> JBoss Performance Team >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From galder at redhat.com Thu Apr 13 17:31:08 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Thu, 13 Apr 2017 23:31:08 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: References: Message-ID: <84B5B79A-C65B-4142-8A1D-A7F0A0B18304@redhat.com> -- Galder Zamarre?o Infinispan, Red Hat > On 5 Apr 2017, at 10:05, Sebastian Laskawiec wrote: > > I love the idea of starting with a simple interface, so +1000 from me. > > I'm also assuming that our new MultiMap will be accessible in both Embedded and Client/Server mode, am I correct? I also think CacheMultimap should extend Iterable. I suspect some of our users might want to use for-each loop with it. Hmmmm, that would only really work for a synchronous API version. For async you'd something like Traversable that we did for the functional map experiment. > Finally, we also need to think about some integration bits (maybe not for the initial implementation but it might be beneficial to create JIRAs for them). With CDI and Spring support we can make them super easy to use (by injecting newly created instances to the users code: @Inject CacheMultimap myMap). > > I also put some more comments below. Nice proposal Katia! > > On Tue, Apr 4, 2017 at 7:09 PM William Burns wrote: > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti wrote: > Hi all, > > As you probably know, Will and I are working on the vert-x infinispan integration [1], where the primary goal is to make infinispan the default cluster management of vert-x. (yeah!) > Vert-x needs support for an Async Multimap. Today's implementation is a wrapper on a normal Cache where only Cache Key's are used to implement the multi map [2]. > This is not very efficient, so after trying some other alternative implementations [3] that don't fully work (injection not working), Will and I have come to the conclusion that it might be a good idea to start having our own native CacheMultimap. This first multimap won't support duplicate values on key's. > > As a quick start, the smallest multimap we need should implement the following interface : > > I agree that having a very slim API to start should be better since we know how much trouble we get into implementing a very large API like ConcurrentMap :) > public interface CacheMultimap { > V put(K key, V value); > This should probably return a boolean or Void. I am leaning towards the first, but I am open either way. > > Could you please tell me more why are you suggesting boolean or void? Returning previous value would make it more similar to a Map. > > Collection get(K key); > > boolean remove(K key, V value); > We probably want a `boolean remove(K key)` method as well that removes all values mapped to the given key. > > +1 > > } > CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. > > We could add a new method in EmbeddedCacheManager.java > > CacheMultimap getCacheMultimap(String cacheName, boolean createIfAbsent); > > How about the other way around? Something like: > static CacheMultimap CacheMultimap.create(BasicCache cache); > > This way we would avoid dependency from DefaultCacheManager to CacheMultimap. If we wanted to support both Embedded/Client Server mode we would probably need to use BasicCache as a parameter. The last argument for this solution is that creating producers in CDI/Spring would be trivial (we would just need to provide a generic producer method and with some luck that would be it). > > > I was thinking maybe this would exist in a separate module (outside of core)? or class that wraps (similar to DistributedExecutor) instead. My worry is about transactions, since the entry point to that is through Cache interface. The other option is we could add a `getCache` method on the `CacheMultiMap`. > > If we want to support both Embedded/Client Server mode, it should go to commons. Otherwise I would vote for core. > > > > > Implementation will create a cache as always and return a new CacheMultimapImpl(cache). > > What do you think ? Please fell free to suggest any other alternative or idea. > > Cheers > > Katia > > [1] https://github.com/vert-x3/vertx-infinispan > > [2] https://github.com/vert-x3/vertx-infinispan/blob/master/src/main/java/io/vertx/ext/cluster/infinispan/impl/InfinispanAsyncMultiMap.java > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- > SEBASTIAN ?ASKAWIEC > INFINISPAN DEVELOPER > Red Hat EMEA > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From karesti at redhat.com Fri Apr 14 03:47:45 2017 From: karesti at redhat.com (Katia Aresti) Date: Fri, 14 Apr 2017 09:47:45 +0200 Subject: [infinispan-dev] Native Infinispan Multimap support In-Reply-To: <84B5B79A-C65B-4142-8A1D-A7F0A0B18304@redhat.com> References: <84B5B79A-C65B-4142-8A1D-A7F0A0B18304@redhat.com> Message-ID: Hi Galder ! Thanks for your comments and ideas ! I'm going to move the content you posted in the list to the design PR here : https://github.com/infinispan/ infinispan-designs/pull/3 ++ Katia On Thu, Apr 13, 2017 at 11:31 PM, Galder Zamarre?o wrote: > > -- > Galder Zamarre?o > Infinispan, Red Hat > > > On 5 Apr 2017, at 10:05, Sebastian Laskawiec > wrote: > > > > I love the idea of starting with a simple interface, so +1000 from me. > > > > I'm also assuming that our new MultiMap will be accessible in both > Embedded and Client/Server mode, am I correct? I also think CacheMultimap > should extend Iterable. I suspect some of our users might want to use > for-each loop with it. > > Hmmmm, that would only really work for a synchronous API version. For > async you'd something like Traversable that we did for the functional map > experiment. > > > Finally, we also need to think about some integration bits (maybe not > for the initial implementation but it might be beneficial to create JIRAs > for them). With CDI and Spring support we can make them super easy to use > (by injecting newly created instances to the users code: @Inject > CacheMultimap myMap). > > > > I also put some more comments below. Nice proposal Katia! > > > > On Tue, Apr 4, 2017 at 7:09 PM William Burns > wrote: > > On Tue, Apr 4, 2017 at 11:45 AM Katia Aresti wrote: > > Hi all, > > > > As you probably know, Will and I are working on the vert-x infinispan > integration [1], where the primary goal is to make infinispan the default > cluster management of vert-x. (yeah!) > > Vert-x needs support for an Async Multimap. Today's implementation is a > wrapper on a normal Cache where only Cache Key's are used to implement the > multi map [2]. > > This is not very efficient, so after trying some other alternative > implementations [3] that don't fully work (injection not working), Will and > I have come to the conclusion that it might be a good idea to start having > our own native CacheMultimap. This first multimap won't support duplicate > values on key's. > > > > As a quick start, the smallest multimap we need should implement the > following interface : > > > > I agree that having a very slim API to start should be better since we > know how much trouble we get into implementing a very large API like > ConcurrentMap :) > > public interface CacheMultimap { > > V put(K key, V value); > > This should probably return a boolean or Void. I am leaning towards the > first, but I am open either way. > > > > Could you please tell me more why are you suggesting boolean or void? > Returning previous value would make it more similar to a Map. > > > > Collection get(K key); > > > > boolean remove(K key, V value); > > We probably want a `boolean remove(K key)` method as well that removes > all values mapped to the given key. > > > > +1 > > > > } > > CacheMultimapImpl will be a wrapper on a normal Cache, similar to [3]. > > > > We could add a new method in EmbeddedCacheManager.java > > > > CacheMultimap getCacheMultimap(String cacheName, boolean > createIfAbsent); > > > > How about the other way around? Something like: > > static CacheMultimap CacheMultimap.create(BasicCache > cache); > > > > This way we would avoid dependency from DefaultCacheManager to > CacheMultimap. If we wanted to support both Embedded/Client Server mode we > would probably need to use BasicCache as a parameter. The last argument for > this solution is that creating producers in CDI/Spring would be trivial (we > would just need to provide a generic producer method and with some luck > that would be it). > > > > > > I was thinking maybe this would exist in a separate module (outside of > core)? or class that wraps (similar to DistributedExecutor) instead. My > worry is about transactions, since the entry point to that is through Cache > interface. The other option is we could add a `getCache` method on the > `CacheMultiMap`. > > > > If we want to support both Embedded/Client Server mode, it should go to > commons. Otherwise I would vote for core. > > > > > > > > > > Implementation will create a cache as always and return a new > CacheMultimapImpl(cache). > > > > What do you think ? Please fell free to suggest any other alternative or > idea. > > > > Cheers > > > > Katia > > > > [1] https://github.com/vert-x3/vertx-infinispan > > > > [2] https://github.com/vert-x3/vertx-infinispan/blob/master/ > src/main/java/io/vertx/ext/cluster/infinispan/impl/ > InfinispanAsyncMultiMap.java > > > > [3] https://gist.github.com/karesti/194bb998856d4a2828d83754130ed79c > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170414/d121b3bb/attachment-0001.html From slaskawi at redhat.com Fri Apr 14 09:50:08 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Fri, 14 Apr 2017 13:50:08 +0000 Subject: [infinispan-dev] HotRod client TCK In-Reply-To: <7bba7850-52fa-1b98-45da-603f1443cc34@redhat.com> References: <7bba7850-52fa-1b98-45da-603f1443cc34@redhat.com> Message-ID: I think that's a very good point Radim. But on the other hand I think we may treat it as an implementation detail. The TCK must run without Infinispan server. I also believe it would be beneficial to define as least 2-3 levels of support, e.g. basic (only basic operations, no consistent hash support), advanced (all features). This way we could validate some more exotic clients such as Go (https://github.com/ugol/infinispan-go). And in my opinion, a final version of this document go to infinispan design repository: https://github.com/infinispan/infinispan-designs Apart from that, that's huge effort and really great job! On Tue, Apr 11, 2017 at 5:05 PM Radim Vansa wrote: > Since these tests use real server(s), many of them test not only the > client behaviour (generating correct commands according to the > protocol), but server, too. While this is practical (we need to test > server somehow, too), there's nothing all the tests across languages > will have physically in common and all comparison is prone to human error. > > If we want to test various implementations of the client, maybe it would > make sense to give the clients a fake server that will have just a > scenario of expected commands to receive and pre-defined responses. We > could use audit log to generate such scenario based on the actual Java > tests. > > But then we'd have to test the actual behaviour on server, and we'd need > a way to issue the commands. > > Just my 2c > > Radim > > On 04/11/2017 02:33 PM, Martin Gencur wrote: > > Hello all, > > we have been working on https://issues.jboss.org/browse/ISPN-7120. > > > > Anna has finished the first step from the JIRA - collecting information > > about tests in the Java HotRod client test suite (including server > > integration tests) and it is now prepared for wider review. > > > > She created a spreadsheet [1]. The spread sheet includes for each Java > > test its name, the suggested target package in the TCK, whether to > > include it in the TCK or not, and some other notes. The suggested > > package also poses grouping for the tests (e.g. tck.query, tck.near, > > tck.xsite, ...) > > > > Let me add that right now the goal is not to create a true TCK [2]. The > > goal is to make sure that all implementations of the HotRod protocol > > have sufficient test coverage and possibly the same server side of the > > client-server test (including the server version and configuration). > > > > What are the next step? > > > > * Please review the list (at least a quick look) and see if some of the > > tests which are NOT suggested for the TCK should be added or vice versa. > > * I suppose the next step would then be to check other implementations > > (C#, C++, NodeJS, ..) and identify tests which are missing there (there > > will surely be some). > > * Gradually implement the missing tests in the other implementations > > Note: Here we should ensure that the server is configured in the same > > way for all implementations. One way to achieve this (thanks Anna for > > suggestion!) is to have a shell/batch scripts for CLI which would be > > executed before the tests. This can probably be done for all impls. and > > both UNIX/WINDOWS. I also realize that my PR for ISPN [3] becomes > > useless because it uses Creaper (Java) and we need a language-neutral > > solution for configuring the server. > > > > Some other notes: > > * there are some duplicated tests in hotrod-client and server > > integration test suites, in this case it probably makes sense to only > > include in the TCK the server integration test > > * tests from the hotrod-client module which are supposed to be part of > > the TCK should be copied to the server integration test suite one day > > (possibly later) > > > > Please let us know what you think. > > > > Thanks, > > Martin > > > > > > [1] > > > https://docs.google.com/spreadsheets/d/1bZBBi5m4oLL4lBTZhdRbIC_EA0giQNDZWzFNPWrF5G4/edit#gid=0 > > [2] https://en.wikipedia.org/wiki/Technology_Compatibility_Kit > > [3] https://github.com/infinispan/infinispan/pull/5012 > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170414/1353c8f6/attachment.html From wfink at redhat.com Sat Apr 15 06:57:05 2017 From: wfink at redhat.com (Wolf Fink) Date: Sat, 15 Apr 2017 12:57:05 +0200 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> Message-ID: I would think a "switch" can have other impacts as you need to check it in the code - and might have security leaks here So what is wrong with some configurations which are the default and secured. and a "*-dev or *-unsecure" configuration to start easy. Also this can be used in production if there is no need for security On Thu, Apr 13, 2017 at 4:13 PM, Sebastian Laskawiec wrote: > I still think it would be better to create an extra switch to run > infinispan in "development mode". This means no authentication, no > encryption, possibly with JGroups stack tuned for fast discovery > (especially in Kubernetes) and a big warning saying "You are in development > mode, do not use this in production". > > Just something very easy to get you going. > > On Thu, Apr 13, 2017 at 12:16 PM Galder Zamarre?o > wrote: > >> >> -- >> Galder Zamarre?o >> Infinispan, Red Hat >> >> > On 13 Apr 2017, at 09:50, Gustavo Fernandes >> wrote: >> > >> > On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o >> wrote: >> > Hi all, >> > >> > As per some discussions we had yesterday on IRC w/ Tristan, Gustavo and >> Sebastian, I've created a docker image snapshot that reverts the change >> stop protected caches from requiring security enabled [1]. >> > >> > In other words, I've removed [2]. The reason for temporarily doing that >> is because with the change as is, the changes required for a default server >> distro require that the entire cache manager's security is enabled. This is >> in turn creates a lot of problems with health and running checks used by >> Kubernetes/OpenShift amongst other things. >> > >> > Judging from our discussions on IRC, the idea is for such change to be >> present in 9.0.1, but I'd like to get final confirmation from Tristan et al. >> > >> > >> > +1 >> > >> > Regarding the "security by default" discussion, I think we should ship >> configurations cloud.xml, clustered.xml and standalone.xml with security >> enabled and disabled variants, and let users >> > decide which one to pick based on the use case. >> >> I think that's a better idea. >> >> We could by default have a secured one, but switching to an insecure >> configuration should be doable with minimal effort, e.g. just switching >> config file. >> >> As highlighted above, any secured configuration should work >> out-of-the-box with our docker images, e.g. WRT healthy/running checks. >> >> Cheers, >> >> > >> > Gustavo. >> > >> > >> > Cheers, >> > >> > [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ >> (9.0.1-SNAPSHOT tag for anyone interested) >> > [2] https://github.com/infinispan/infinispan/blob/master/server/ >> hotrod/src/main/java/org/infinispan/server/hotrod/ >> CacheDecodeContext.java#L114-L118 >> > -- >> > Galder Zamarre?o >> > Infinispan, Red Hat >> > >> > > On 30 Mar 2017, at 14:25, Tristan Tarrant >> wrote: >> > > >> > > Dear all, >> > > >> > > after a mini chat on IRC, I wanted to bring this to everybody's >> attention. >> > > >> > > We should make the Hot Rod endpoint require authentication in the >> > > out-of-the-box configuration. >> > > The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL >> > > mechanism against the ApplicationRealm and require users to run the >> > > add-user script. >> > > This would achieve two goals: >> > > - secure out-of-the-box configuration, which is always a good idea >> > > - access to the "protected" schema and script caches which is >> prevented >> > > when not on loopback on non-authenticated endpoints. >> > > >> > > Tristan >> > > -- >> > > Tristan Tarrant >> > > Infinispan Lead >> > > JBoss, a division of Red Hat >> > > _______________________________________________ >> > > infinispan-dev mailing list >> > > infinispan-dev at lists.jboss.org >> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170415/24aed177/attachment-0001.html From rvansa at redhat.com Tue Apr 18 03:01:46 2017 From: rvansa at redhat.com (Radim Vansa) Date: Tue, 18 Apr 2017 09:01:46 +0200 Subject: [infinispan-dev] HotRod client TCK In-Reply-To: References: <7bba7850-52fa-1b98-45da-603f1443cc34@redhat.com> Message-ID: <6ef7e2a7-ac01-355f-ac93-1cbb32b41d5b@redhat.com> On 04/14/2017 03:50 PM, Sebastian Laskawiec wrote: > I think that's a very good point Radim. But on the other hand I think > we may treat it as an implementation detail. The TCK must run without > Infinispan server. So the tests in TCK won't start/configure the server, that will be managed elsewhere? R. > > I also believe it would be beneficial to define as least 2-3 levels of > support, e.g. basic (only basic operations, no consistent hash > support), advanced (all features). This way we could validate some > more exotic clients such as Go (https://github.com/ugol/infinispan-go). > > And in my opinion, a final version of this document go to infinispan > design repository: https://github.com/infinispan/infinispan-designs > > Apart from that, that's huge effort and really great job! > > On Tue, Apr 11, 2017 at 5:05 PM Radim Vansa > wrote: > > Since these tests use real server(s), many of them test not only the > client behaviour (generating correct commands according to the > protocol), but server, too. While this is practical (we need to test > server somehow, too), there's nothing all the tests across languages > will have physically in common and all comparison is prone to > human error. > > If we want to test various implementations of the client, maybe it > would > make sense to give the clients a fake server that will have just a > scenario of expected commands to receive and pre-defined responses. We > could use audit log to generate such scenario based on the actual Java > tests. > > But then we'd have to test the actual behaviour on server, and > we'd need > a way to issue the commands. > > Just my 2c > > Radim > > On 04/11/2017 02:33 PM, Martin Gencur wrote: > > Hello all, > > we have been working on https://issues.jboss.org/browse/ISPN-7120. > > > > Anna has finished the first step from the JIRA - collecting > information > > about tests in the Java HotRod client test suite (including server > > integration tests) and it is now prepared for wider review. > > > > She created a spreadsheet [1]. The spread sheet includes for > each Java > > test its name, the suggested target package in the TCK, whether to > > include it in the TCK or not, and some other notes. The suggested > > package also poses grouping for the tests (e.g. tck.query, tck.near, > > tck.xsite, ...) > > > > Let me add that right now the goal is not to create a true TCK > [2]. The > > goal is to make sure that all implementations of the HotRod protocol > > have sufficient test coverage and possibly the same server side > of the > > client-server test (including the server version and configuration). > > > > What are the next step? > > > > * Please review the list (at least a quick look) and see if some > of the > > tests which are NOT suggested for the TCK should be added or > vice versa. > > * I suppose the next step would then be to check other > implementations > > (C#, C++, NodeJS, ..) and identify tests which are missing there > (there > > will surely be some). > > * Gradually implement the missing tests in the other implementations > > Note: Here we should ensure that the server is configured in > the same > > way for all implementations. One way to achieve this (thanks > Anna for > > suggestion!) is to have a shell/batch scripts for CLI which would be > > executed before the tests. This can probably be done for all > impls. and > > both UNIX/WINDOWS. I also realize that my PR for ISPN [3] becomes > > useless because it uses Creaper (Java) and we need a > language-neutral > > solution for configuring the server. > > > > Some other notes: > > * there are some duplicated tests in hotrod-client and server > > integration test suites, in this case it probably makes sense to > only > > include in the TCK the server integration test > > * tests from the hotrod-client module which are supposed to be > part of > > the TCK should be copied to the server integration test suite > one day > > (possibly later) > > > > Please let us know what you think. > > > > Thanks, > > Martin > > > > > > [1] > > > https://docs.google.com/spreadsheets/d/1bZBBi5m4oLL4lBTZhdRbIC_EA0giQNDZWzFNPWrF5G4/edit#gid=0 > > [2] https://en.wikipedia.org/wiki/Technology_Compatibility_Kit > > [3] https://github.com/infinispan/infinispan/pull/5012 > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa > > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN?ASKAWIEC > > INFINISPAN DEVELOPER > > Red HatEMEA > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From galder at redhat.com Wed Apr 19 04:05:47 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Wed, 19 Apr 2017 10:05:47 +0200 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> Message-ID: Agree with Wolf. Let's keep it simple by just providing extra configuration files for dev/unsecure envs. Cheers, -- Galder Zamarre?o Infinispan, Red Hat > On 15 Apr 2017, at 12:57, Wolf Fink wrote: > > I would think a "switch" can have other impacts as you need to check it in the code - and might have security leaks here > > So what is wrong with some configurations which are the default and secured. > and a "*-dev or *-unsecure" configuration to start easy. > Also this can be used in production if there is no need for security > > On Thu, Apr 13, 2017 at 4:13 PM, Sebastian Laskawiec wrote: > I still think it would be better to create an extra switch to run infinispan in "development mode". This means no authentication, no encryption, possibly with JGroups stack tuned for fast discovery (especially in Kubernetes) and a big warning saying "You are in development mode, do not use this in production". > > Just something very easy to get you going. > > On Thu, Apr 13, 2017 at 12:16 PM Galder Zamarre?o wrote: > > -- > Galder Zamarre?o > Infinispan, Red Hat > > > On 13 Apr 2017, at 09:50, Gustavo Fernandes wrote: > > > > On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o wrote: > > Hi all, > > > > As per some discussions we had yesterday on IRC w/ Tristan, Gustavo and Sebastian, I've created a docker image snapshot that reverts the change stop protected caches from requiring security enabled [1]. > > > > In other words, I've removed [2]. The reason for temporarily doing that is because with the change as is, the changes required for a default server distro require that the entire cache manager's security is enabled. This is in turn creates a lot of problems with health and running checks used by Kubernetes/OpenShift amongst other things. > > > > Judging from our discussions on IRC, the idea is for such change to be present in 9.0.1, but I'd like to get final confirmation from Tristan et al. > > > > > > +1 > > > > Regarding the "security by default" discussion, I think we should ship configurations cloud.xml, clustered.xml and standalone.xml with security enabled and disabled variants, and let users > > decide which one to pick based on the use case. > > I think that's a better idea. > > We could by default have a secured one, but switching to an insecure configuration should be doable with minimal effort, e.g. just switching config file. > > As highlighted above, any secured configuration should work out-of-the-box with our docker images, e.g. WRT healthy/running checks. > > Cheers, > > > > > Gustavo. > > > > > > Cheers, > > > > [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ (9.0.1-SNAPSHOT tag for anyone interested) > > [2] https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/java/org/infinispan/server/hotrod/CacheDecodeContext.java#L114-L118 > > -- > > Galder Zamarre?o > > Infinispan, Red Hat > > > > > On 30 Mar 2017, at 14:25, Tristan Tarrant wrote: > > > > > > Dear all, > > > > > > after a mini chat on IRC, I wanted to bring this to everybody's attention. > > > > > > We should make the Hot Rod endpoint require authentication in the > > > out-of-the-box configuration. > > > The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL > > > mechanism against the ApplicationRealm and require users to run the > > > add-user script. > > > This would achieve two goals: > > > - secure out-of-the-box configuration, which is always a good idea > > > - access to the "protected" schema and script caches which is prevented > > > when not on loopback on non-authenticated endpoints. > > > > > > Tristan > > > -- > > > Tristan Tarrant > > > Infinispan Lead > > > JBoss, a division of Red Hat > > > _______________________________________________ > > > infinispan-dev mailing list > > > infinispan-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- > SEBASTIAN ?ASKAWIEC > INFINISPAN DEVELOPER > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From ttarrant at redhat.com Wed Apr 19 04:57:06 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Wed, 19 Apr 2017 10:57:06 +0200 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> Message-ID: <20264f25-5a92-f6b9-e8f9-a91d822b5c8f@redhat.com> Currently the "protected cache access" security is implemented as follows: - if authorization is enabled || client is on loopback allow The first check also implies that authentication needs to be in place, as the authorization checks need a valid Subject. Unfortunately authorization is very heavy-weight and actually overkill even for "normal" secure usage. My proposal is as follows: - the "default" configuration files are "secure" by default - provide clearly marked "unsecured" configuration files, which the user can use - drop the "protected cache" check completely And definitely NO to a dev switch. Tristan On 19/04/2017 10:05, Galder Zamarre?o wrote: > Agree with Wolf. Let's keep it simple by just providing extra configuration files for dev/unsecure envs. > > Cheers, > -- > Galder Zamarre?o > Infinispan, Red Hat > >> On 15 Apr 2017, at 12:57, Wolf Fink wrote: >> >> I would think a "switch" can have other impacts as you need to check it in the code - and might have security leaks here >> >> So what is wrong with some configurations which are the default and secured. >> and a "*-dev or *-unsecure" configuration to start easy. >> Also this can be used in production if there is no need for security >> >> On Thu, Apr 13, 2017 at 4:13 PM, Sebastian Laskawiec wrote: >> I still think it would be better to create an extra switch to run infinispan in "development mode". This means no authentication, no encryption, possibly with JGroups stack tuned for fast discovery (especially in Kubernetes) and a big warning saying "You are in development mode, do not use this in production". >> >> Just something very easy to get you going. >> >> On Thu, Apr 13, 2017 at 12:16 PM Galder Zamarre?o wrote: >> >> -- >> Galder Zamarre?o >> Infinispan, Red Hat >> >>> On 13 Apr 2017, at 09:50, Gustavo Fernandes wrote: >>> >>> On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o wrote: >>> Hi all, >>> >>> As per some discussions we had yesterday on IRC w/ Tristan, Gustavo and Sebastian, I've created a docker image snapshot that reverts the change stop protected caches from requiring security enabled [1]. >>> >>> In other words, I've removed [2]. The reason for temporarily doing that is because with the change as is, the changes required for a default server distro require that the entire cache manager's security is enabled. This is in turn creates a lot of problems with health and running checks used by Kubernetes/OpenShift amongst other things. >>> >>> Judging from our discussions on IRC, the idea is for such change to be present in 9.0.1, but I'd like to get final confirmation from Tristan et al. >>> >>> >>> +1 >>> >>> Regarding the "security by default" discussion, I think we should ship configurations cloud.xml, clustered.xml and standalone.xml with security enabled and disabled variants, and let users >>> decide which one to pick based on the use case. >> >> I think that's a better idea. >> >> We could by default have a secured one, but switching to an insecure configuration should be doable with minimal effort, e.g. just switching config file. >> >> As highlighted above, any secured configuration should work out-of-the-box with our docker images, e.g. WRT healthy/running checks. >> >> Cheers, >> >>> >>> Gustavo. >>> >>> >>> Cheers, >>> >>> [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ (9.0.1-SNAPSHOT tag for anyone interested) >>> [2] https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/java/org/infinispan/server/hotrod/CacheDecodeContext.java#L114-L118 >>> -- >>> Galder Zamarre?o >>> Infinispan, Red Hat >>> >>>> On 30 Mar 2017, at 14:25, Tristan Tarrant wrote: >>>> >>>> Dear all, >>>> >>>> after a mini chat on IRC, I wanted to bring this to everybody's attention. >>>> >>>> We should make the Hot Rod endpoint require authentication in the >>>> out-of-the-box configuration. >>>> The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL >>>> mechanism against the ApplicationRealm and require users to run the >>>> add-user script. >>>> This would achieve two goals: >>>> - secure out-of-the-box configuration, which is always a good idea >>>> - access to the "protected" schema and script caches which is prevented >>>> when not on loopback on non-authenticated endpoints. >>>> >>>> Tristan >>>> -- >>>> Tristan Tarrant >>>> Infinispan Lead >>>> JBoss, a division of Red Hat >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> -- >> SEBASTIAN ?ASKAWIEC >> INFINISPAN DEVELOPER >> Red Hat EMEA >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From slaskawi at redhat.com Wed Apr 19 05:34:54 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Wed, 19 Apr 2017 09:34:54 +0000 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: <20264f25-5a92-f6b9-e8f9-a91d822b5c8f@redhat.com> References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> <20264f25-5a92-f6b9-e8f9-a91d822b5c8f@redhat.com> Message-ID: The proposal look ok to me. But I would also like to highlight one thing - it seems you can't access secured cache properties using CLI. This seems wrong to me (if you can invoke the cli, in 99,99% of the cases you have access to the machine, so you can do whatever you want). It also breaks healthchecks in Docker image. I would like to make sure we will address those concerns. On Wed, Apr 19, 2017 at 10:59 AM Tristan Tarrant wrote: > Currently the "protected cache access" security is implemented as follows: > > - if authorization is enabled || client is on loopback > allow > > The first check also implies that authentication needs to be in place, > as the authorization checks need a valid Subject. > > Unfortunately authorization is very heavy-weight and actually overkill > even for "normal" secure usage. > > My proposal is as follows: > - the "default" configuration files are "secure" by default > - provide clearly marked "unsecured" configuration files, which the user > can use > - drop the "protected cache" check completely > > And definitely NO to a dev switch. > > Tristan > > On 19/04/2017 10:05, Galder Zamarre?o wrote: > > Agree with Wolf. Let's keep it simple by just providing extra > configuration files for dev/unsecure envs. > > > > Cheers, > > -- > > Galder Zamarre?o > > Infinispan, Red Hat > > > >> On 15 Apr 2017, at 12:57, Wolf Fink wrote: > >> > >> I would think a "switch" can have other impacts as you need to check it > in the code - and might have security leaks here > >> > >> So what is wrong with some configurations which are the default and > secured. > >> and a "*-dev or *-unsecure" configuration to start easy. > >> Also this can be used in production if there is no need for security > >> > >> On Thu, Apr 13, 2017 at 4:13 PM, Sebastian Laskawiec < > slaskawi at redhat.com> wrote: > >> I still think it would be better to create an extra switch to run > infinispan in "development mode". This means no authentication, no > encryption, possibly with JGroups stack tuned for fast discovery > (especially in Kubernetes) and a big warning saying "You are in development > mode, do not use this in production". > >> > >> Just something very easy to get you going. > >> > >> On Thu, Apr 13, 2017 at 12:16 PM Galder Zamarre?o > wrote: > >> > >> -- > >> Galder Zamarre?o > >> Infinispan, Red Hat > >> > >>> On 13 Apr 2017, at 09:50, Gustavo Fernandes > wrote: > >>> > >>> On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o > wrote: > >>> Hi all, > >>> > >>> As per some discussions we had yesterday on IRC w/ Tristan, Gustavo > and Sebastian, I've created a docker image snapshot that reverts the change > stop protected caches from requiring security enabled [1]. > >>> > >>> In other words, I've removed [2]. The reason for temporarily doing > that is because with the change as is, the changes required for a default > server distro require that the entire cache manager's security is enabled. > This is in turn creates a lot of problems with health and running checks > used by Kubernetes/OpenShift amongst other things. > >>> > >>> Judging from our discussions on IRC, the idea is for such change to be > present in 9.0.1, but I'd like to get final confirmation from Tristan et al. > >>> > >>> > >>> +1 > >>> > >>> Regarding the "security by default" discussion, I think we should ship > configurations cloud.xml, clustered.xml and standalone.xml with security > enabled and disabled variants, and let users > >>> decide which one to pick based on the use case. > >> > >> I think that's a better idea. > >> > >> We could by default have a secured one, but switching to an insecure > configuration should be doable with minimal effort, e.g. just switching > config file. > >> > >> As highlighted above, any secured configuration should work > out-of-the-box with our docker images, e.g. WRT healthy/running checks. > >> > >> Cheers, > >> > >>> > >>> Gustavo. > >>> > >>> > >>> Cheers, > >>> > >>> [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ > (9.0.1-SNAPSHOT tag for anyone interested) > >>> [2] > https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/java/org/infinispan/server/hotrod/CacheDecodeContext.java#L114-L118 > >>> -- > >>> Galder Zamarre?o > >>> Infinispan, Red Hat > >>> > >>>> On 30 Mar 2017, at 14:25, Tristan Tarrant > wrote: > >>>> > >>>> Dear all, > >>>> > >>>> after a mini chat on IRC, I wanted to bring this to everybody's > attention. > >>>> > >>>> We should make the Hot Rod endpoint require authentication in the > >>>> out-of-the-box configuration. > >>>> The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL > >>>> mechanism against the ApplicationRealm and require users to run the > >>>> add-user script. > >>>> This would achieve two goals: > >>>> - secure out-of-the-box configuration, which is always a good idea > >>>> - access to the "protected" schema and script caches which is > prevented > >>>> when not on loopback on non-authenticated endpoints. > >>>> > >>>> Tristan > >>>> -- > >>>> Tristan Tarrant > >>>> Infinispan Lead > >>>> JBoss, a division of Red Hat > >>>> _______________________________________________ > >>>> infinispan-dev mailing list > >>>> infinispan-dev at lists.jboss.org > >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> -- > >> SEBASTIAN ?ASKAWIEC > >> INFINISPAN DEVELOPER > >> Red Hat EMEA > >> > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170419/eeff2607/attachment-0001.html From galder at redhat.com Wed Apr 19 05:35:17 2017 From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=) Date: Wed, 19 Apr 2017 11:35:17 +0200 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: <20264f25-5a92-f6b9-e8f9-a91d822b5c8f@redhat.com> References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> <20264f25-5a92-f6b9-e8f9-a91d822b5c8f@redhat.com> Message-ID: +100 -- Galder Zamarre?o Infinispan, Red Hat > On 19 Apr 2017, at 10:57, Tristan Tarrant wrote: > > Currently the "protected cache access" security is implemented as follows: > > - if authorization is enabled || client is on loopback > allow > > The first check also implies that authentication needs to be in place, > as the authorization checks need a valid Subject. > > Unfortunately authorization is very heavy-weight and actually overkill > even for "normal" secure usage. > > My proposal is as follows: > - the "default" configuration files are "secure" by default > - provide clearly marked "unsecured" configuration files, which the user > can use > - drop the "protected cache" check completely > > And definitely NO to a dev switch. > > Tristan > > On 19/04/2017 10:05, Galder Zamarre?o wrote: >> Agree with Wolf. Let's keep it simple by just providing extra configuration files for dev/unsecure envs. >> >> Cheers, >> -- >> Galder Zamarre?o >> Infinispan, Red Hat >> >>> On 15 Apr 2017, at 12:57, Wolf Fink wrote: >>> >>> I would think a "switch" can have other impacts as you need to check it in the code - and might have security leaks here >>> >>> So what is wrong with some configurations which are the default and secured. >>> and a "*-dev or *-unsecure" configuration to start easy. >>> Also this can be used in production if there is no need for security >>> >>> On Thu, Apr 13, 2017 at 4:13 PM, Sebastian Laskawiec wrote: >>> I still think it would be better to create an extra switch to run infinispan in "development mode". This means no authentication, no encryption, possibly with JGroups stack tuned for fast discovery (especially in Kubernetes) and a big warning saying "You are in development mode, do not use this in production". >>> >>> Just something very easy to get you going. >>> >>> On Thu, Apr 13, 2017 at 12:16 PM Galder Zamarre?o wrote: >>> >>> -- >>> Galder Zamarre?o >>> Infinispan, Red Hat >>> >>>> On 13 Apr 2017, at 09:50, Gustavo Fernandes wrote: >>>> >>>> On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o wrote: >>>> Hi all, >>>> >>>> As per some discussions we had yesterday on IRC w/ Tristan, Gustavo and Sebastian, I've created a docker image snapshot that reverts the change stop protected caches from requiring security enabled [1]. >>>> >>>> In other words, I've removed [2]. The reason for temporarily doing that is because with the change as is, the changes required for a default server distro require that the entire cache manager's security is enabled. This is in turn creates a lot of problems with health and running checks used by Kubernetes/OpenShift amongst other things. >>>> >>>> Judging from our discussions on IRC, the idea is for such change to be present in 9.0.1, but I'd like to get final confirmation from Tristan et al. >>>> >>>> >>>> +1 >>>> >>>> Regarding the "security by default" discussion, I think we should ship configurations cloud.xml, clustered.xml and standalone.xml with security enabled and disabled variants, and let users >>>> decide which one to pick based on the use case. >>> >>> I think that's a better idea. >>> >>> We could by default have a secured one, but switching to an insecure configuration should be doable with minimal effort, e.g. just switching config file. >>> >>> As highlighted above, any secured configuration should work out-of-the-box with our docker images, e.g. WRT healthy/running checks. >>> >>> Cheers, >>> >>>> >>>> Gustavo. >>>> >>>> >>>> Cheers, >>>> >>>> [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ (9.0.1-SNAPSHOT tag for anyone interested) >>>> [2] https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/java/org/infinispan/server/hotrod/CacheDecodeContext.java#L114-L118 >>>> -- >>>> Galder Zamarre?o >>>> Infinispan, Red Hat >>>> >>>>> On 30 Mar 2017, at 14:25, Tristan Tarrant wrote: >>>>> >>>>> Dear all, >>>>> >>>>> after a mini chat on IRC, I wanted to bring this to everybody's attention. >>>>> >>>>> We should make the Hot Rod endpoint require authentication in the >>>>> out-of-the-box configuration. >>>>> The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL >>>>> mechanism against the ApplicationRealm and require users to run the >>>>> add-user script. >>>>> This would achieve two goals: >>>>> - secure out-of-the-box configuration, which is always a good idea >>>>> - access to the "protected" schema and script caches which is prevented >>>>> when not on loopback on non-authenticated endpoints. >>>>> >>>>> Tristan >>>>> -- >>>>> Tristan Tarrant >>>>> Infinispan Lead >>>>> JBoss, a division of Red Hat >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> -- >>> SEBASTIAN ?ASKAWIEC >>> INFINISPAN DEVELOPER >>> Red Hat EMEA >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From ttarrant at redhat.com Wed Apr 19 06:04:34 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Wed, 19 Apr 2017 12:04:34 +0200 Subject: [infinispan-dev] Hot Rod secured by default In-Reply-To: References: <78C8F389-2EBA-4E2F-8EFE-CDAAAD65F55D@redhat.com> <20264f25-5a92-f6b9-e8f9-a91d822b5c8f@redhat.com> Message-ID: <913ec0fd-46dd-57e5-0eee-cb190066ed2e@redhat.com> That is caused by not wrapping the calls in PrivilegedActions in all the correct places and is a bug. Tristan On 19/04/2017 11:34, Sebastian Laskawiec wrote: > The proposal look ok to me. > > But I would also like to highlight one thing - it seems you can't access > secured cache properties using CLI. This seems wrong to me (if you can > invoke the cli, in 99,99% of the cases you have access to the machine, > so you can do whatever you want). It also breaks healthchecks in Docker > image. > > I would like to make sure we will address those concerns. > > On Wed, Apr 19, 2017 at 10:59 AM Tristan Tarrant > wrote: > > Currently the "protected cache access" security is implemented as > follows: > > - if authorization is enabled || client is on loopback > allow > > The first check also implies that authentication needs to be in place, > as the authorization checks need a valid Subject. > > Unfortunately authorization is very heavy-weight and actually overkill > even for "normal" secure usage. > > My proposal is as follows: > - the "default" configuration files are "secure" by default > - provide clearly marked "unsecured" configuration files, which the user > can use > - drop the "protected cache" check completely > > And definitely NO to a dev switch. > > Tristan > > On 19/04/2017 10:05, Galder Zamarre?o wrote: > > Agree with Wolf. Let's keep it simple by just providing extra > configuration files for dev/unsecure envs. > > > > Cheers, > > -- > > Galder Zamarre?o > > Infinispan, Red Hat > > > >> On 15 Apr 2017, at 12:57, Wolf Fink > wrote: > >> > >> I would think a "switch" can have other impacts as you need to > check it in the code - and might have security leaks here > >> > >> So what is wrong with some configurations which are the default > and secured. > >> and a "*-dev or *-unsecure" configuration to start easy. > >> Also this can be used in production if there is no need for security > >> > >> On Thu, Apr 13, 2017 at 4:13 PM, Sebastian Laskawiec > > wrote: > >> I still think it would be better to create an extra switch to > run infinispan in "development mode". This means no authentication, > no encryption, possibly with JGroups stack tuned for fast discovery > (especially in Kubernetes) and a big warning saying "You are in > development mode, do not use this in production". > >> > >> Just something very easy to get you going. > >> > >> On Thu, Apr 13, 2017 at 12:16 PM Galder Zamarre?o > > wrote: > >> > >> -- > >> Galder Zamarre?o > >> Infinispan, Red Hat > >> > >>> On 13 Apr 2017, at 09:50, Gustavo Fernandes > > wrote: > >>> > >>> On Thu, Apr 13, 2017 at 6:38 AM, Galder Zamarre?o > > wrote: > >>> Hi all, > >>> > >>> As per some discussions we had yesterday on IRC w/ Tristan, > Gustavo and Sebastian, I've created a docker image snapshot that > reverts the change stop protected caches from requiring security > enabled [1]. > >>> > >>> In other words, I've removed [2]. The reason for temporarily > doing that is because with the change as is, the changes required > for a default server distro require that the entire cache manager's > security is enabled. This is in turn creates a lot of problems with > health and running checks used by Kubernetes/OpenShift amongst other > things. > >>> > >>> Judging from our discussions on IRC, the idea is for such > change to be present in 9.0.1, but I'd like to get final > confirmation from Tristan et al. > >>> > >>> > >>> +1 > >>> > >>> Regarding the "security by default" discussion, I think we > should ship configurations cloud.xml, clustered.xml and > standalone.xml with security enabled and disabled variants, and let > users > >>> decide which one to pick based on the use case. > >> > >> I think that's a better idea. > >> > >> We could by default have a secured one, but switching to an > insecure configuration should be doable with minimal effort, e.g. > just switching config file. > >> > >> As highlighted above, any secured configuration should work > out-of-the-box with our docker images, e.g. WRT healthy/running checks. > >> > >> Cheers, > >> > >>> > >>> Gustavo. > >>> > >>> > >>> Cheers, > >>> > >>> [1] https://hub.docker.com/r/galderz/infinispan-server/tags/ > (9.0.1-SNAPSHOT tag for anyone interested) > >>> [2] > https://github.com/infinispan/infinispan/blob/master/server/hotrod/src/main/java/org/infinispan/server/hotrod/CacheDecodeContext.java#L114-L118 > >>> -- > >>> Galder Zamarre?o > >>> Infinispan, Red Hat > >>> > >>>> On 30 Mar 2017, at 14:25, Tristan Tarrant > wrote: > >>>> > >>>> Dear all, > >>>> > >>>> after a mini chat on IRC, I wanted to bring this to > everybody's attention. > >>>> > >>>> We should make the Hot Rod endpoint require authentication in the > >>>> out-of-the-box configuration. > >>>> The proposal is to enable the PLAIN (or, preferably, DIGEST) SASL > >>>> mechanism against the ApplicationRealm and require users to > run the > >>>> add-user script. > >>>> This would achieve two goals: > >>>> - secure out-of-the-box configuration, which is always a good idea > >>>> - access to the "protected" schema and script caches which is > prevented > >>>> when not on loopback on non-authenticated endpoints. > >>>> > >>>> Tristan > >>>> -- > >>>> Tristan Tarrant > >>>> Infinispan Lead > >>>> JBoss, a division of Red Hat > >>>> _______________________________________________ > >>>> infinispan-dev mailing list > >>>> infinispan-dev at lists.jboss.org > > >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev at lists.jboss.org > > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev at lists.jboss.org > > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> -- > >> SEBASTIAN ?ASKAWIEC > >> INFINISPAN DEVELOPER > >> Red Hat EMEA > >> > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN?ASKAWIEC > > INFINISPAN DEVELOPER > > Red HatEMEA > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From ttarrant at redhat.com Thu Apr 20 08:08:36 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Thu, 20 Apr 2017 14:08:36 +0200 Subject: [infinispan-dev] Infinispan Query API simplification Message-ID: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> Querying an Infinispan cache is currently a bit cumbersome. There are two paths: Ickle: Search.getQueryFactory(cache).create("...").list(); DSL (one possible example): Search.getQueryFactory(cache).from(class).[filters].build().list(); Ideally we should have something like: Ickle: cache.query("...").list(); DSL: cache.query(class).[filters].list(); Additionally, the query module is separate from infinispan-core. While this made sense when we didn't have non-indexed query capabilities (and is somewhat mitigated by the uberjars), I feel that query should be a 1st class citizen API-wise. For this reason I propose that we extract the query API to infinispan-commons, put the query SPI in infinispan-core together with the non-indexed implementation and have the hibernate-search backend as a pluggable implementation. Thoughts ? Tristan -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From sanne at infinispan.org Thu Apr 20 08:35:25 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Thu, 20 Apr 2017 13:35:25 +0100 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> Message-ID: Looks good to me, at high level. This will have to focus on the capabilities of the non-indexed query engine though, as we won't be able to expose the more advanced capabilities over the DSL without introducing an hard dependency on Hibernate Search. Conversely, the method accepting an Ickle query will have to fail at runtime if it's using full-text capabilities and the indexing engine wasn't enabled. I think that's fine, as it's not wildly different than attempting a full-text query today when having the wrong fields indexed: we can only validate such things when the query is parsed. [N.B. I'd like to see a way to pre-register queries - like "prepared statement", for many other good reasons so that might improve eventually] So the tricky part will be to cleanly separate the query API from the full-text query API, and the downside will be that using full-text queries will still require some form of API unwrapping. I take it you'd like to see the same API which we'll make for Cache to be exposed on Remote Cache ? Thanks, Sanne On 20 April 2017 at 13:08, Tristan Tarrant wrote: > Querying an Infinispan cache is currently a bit cumbersome. There are > two paths: > > Ickle: > Search.getQueryFactory(cache).create("...").list(); > > DSL (one possible example): > Search.getQueryFactory(cache).from(class).[filters].build().list(); > > Ideally we should have something like: > > Ickle: > cache.query("...").list(); > > DSL: > cache.query(class).[filters].list(); > > Additionally, the query module is separate from infinispan-core. While > this made sense when we didn't have non-indexed query capabilities (and > is somewhat mitigated by the uberjars), I feel that query should be a > 1st class citizen API-wise. > For this reason I propose that we extract the query API to > infinispan-commons, put the query SPI in infinispan-core together with > the non-indexed implementation and have the hibernate-search backend as > a pluggable implementation. > > Thoughts ? > > Tristan > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From rvansa at redhat.com Thu Apr 20 08:56:27 2017 From: rvansa at redhat.com (Radim Vansa) Date: Thu, 20 Apr 2017 14:56:27 +0200 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> Message-ID: <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> That's completely opposite approach from the one outlined for distributed counters and other "on-top" functionality (the same approach was later suggested for conflict resolution manager, multimap and maybe others). Why is query 1st level citizen and those others are not? I am not opposing the idea but let's define the line between patriarchs and plebeians. How big is the DSL API surface (which will be brought into commons)? R. On 04/20/2017 02:08 PM, Tristan Tarrant wrote: > Querying an Infinispan cache is currently a bit cumbersome. There are > two paths: > > Ickle: > Search.getQueryFactory(cache).create("...").list(); > > DSL (one possible example): > Search.getQueryFactory(cache).from(class).[filters].build().list(); > > Ideally we should have something like: > > Ickle: > cache.query("...").list(); > > DSL: > cache.query(class).[filters].list(); > > Additionally, the query module is separate from infinispan-core. While > this made sense when we didn't have non-indexed query capabilities (and > is somewhat mitigated by the uberjars), I feel that query should be a > 1st class citizen API-wise. > For this reason I propose that we extract the query API to > infinispan-commons, put the query SPI in infinispan-core together with > the non-indexed implementation and have the hibernate-search backend as > a pluggable implementation. > > Thoughts ? > > Tristan > -- Radim Vansa JBoss Performance Team From dan.berindei at gmail.com Thu Apr 20 09:34:24 2017 From: dan.berindei at gmail.com (Dan Berindei) Date: Thu, 20 Apr 2017 16:34:24 +0300 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> Message-ID: On Thu, Apr 20, 2017 at 3:56 PM, Radim Vansa wrote: > That's completely opposite approach from the one outlined for > distributed counters and other "on-top" functionality (the same approach > was later suggested for conflict resolution manager, multimap and maybe > others). Why is query 1st level citizen and those others are not? > > I am not opposing the idea but let's define the line between patriarchs > and plebeians. I think you meant patricians, not patriarchs :) But your question makes sense, especially as Sanne pointed out that some query functionality will not be available via the commons/core API. > > How big is the DSL API surface (which will be brought into commons)? -1 from me to add anything in commons, I don't think allowing the users to query both embedded caches and remote caches with the same code is that important. I'd rather go the opposite way and remove the BasicCache interface completely. Dan > > R. > > On 04/20/2017 02:08 PM, Tristan Tarrant wrote: >> Querying an Infinispan cache is currently a bit cumbersome. There are >> two paths: >> >> Ickle: >> Search.getQueryFactory(cache).create("...").list(); >> >> DSL (one possible example): >> Search.getQueryFactory(cache).from(class).[filters].build().list(); >> >> Ideally we should have something like: >> >> Ickle: >> cache.query("...").list(); >> >> DSL: >> cache.query(class).[filters].list(); >> >> Additionally, the query module is separate from infinispan-core. While >> this made sense when we didn't have non-indexed query capabilities (and >> is somewhat mitigated by the uberjars), I feel that query should be a >> 1st class citizen API-wise. >> For this reason I propose that we extract the query API to >> infinispan-commons, put the query SPI in infinispan-core together with >> the non-indexed implementation and have the hibernate-search backend as >> a pluggable implementation. >> >> Thoughts ? >> >> Tristan >> > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From ttarrant at redhat.com Thu Apr 20 09:53:36 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Thu, 20 Apr 2017 15:53:36 +0200 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> Message-ID: Actually, the API for counters et alia is going into commons (so that it can be shared between embedded and remote). Additionally, something like a counter has no API relationship with a Cache, whereas query does. Tristan On 20/04/2017 14:56, Radim Vansa wrote: > That's completely opposite approach from the one outlined for > distributed counters and other "on-top" functionality (the same approach > was later suggested for conflict resolution manager, multimap and maybe > others). Why is query 1st level citizen and those others are not? > > I am not opposing the idea but let's define the line between patriarchs > and plebeians. > > How big is the DSL API surface (which will be brought into commons)? > > R. > > On 04/20/2017 02:08 PM, Tristan Tarrant wrote: >> Querying an Infinispan cache is currently a bit cumbersome. There are >> two paths: >> >> Ickle: >> Search.getQueryFactory(cache).create("...").list(); >> >> DSL (one possible example): >> Search.getQueryFactory(cache).from(class).[filters].build().list(); >> >> Ideally we should have something like: >> >> Ickle: >> cache.query("...").list(); >> >> DSL: >> cache.query(class).[filters].list(); >> >> Additionally, the query module is separate from infinispan-core. While >> this made sense when we didn't have non-indexed query capabilities (and >> is somewhat mitigated by the uberjars), I feel that query should be a >> 1st class citizen API-wise. >> For this reason I propose that we extract the query API to >> infinispan-commons, put the query SPI in infinispan-core together with >> the non-indexed implementation and have the hibernate-search backend as >> a pluggable implementation. >> >> Thoughts ? >> >> Tristan >> > > -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From sanne at infinispan.org Thu Apr 20 10:04:01 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Thu, 20 Apr 2017 15:04:01 +0100 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> Message-ID: On 20 April 2017 at 13:56, Radim Vansa wrote: > That's completely opposite approach from the one outlined for > distributed counters and other "on-top" functionality (the same approach > was later suggested for conflict resolution manager, multimap and maybe > others). Why is query 1st level citizen and those others are not? I agree this needs to be clarified. Be aware that this is the main reason for which it didn't happen earlier ;-) It seems what changed recently is the demand for nicer Query API is quite popular, but I'm skeptical that our existing helper is so hard to use; are we sure people won't just be fine with us improving that API but still keeping the helper? Maybe just a lack of straight forward examples? Either way, it would still be good to draft some guidelines for future API development. Maybe even just having a quick reference sheet with all extension using the unwrap would make Query users happier. ## Alternative direction: what about we actually simplify the Cache API in Infinispan Core, and move anything which isn't strictly key/value store access to its own module - to be accessed via unwrap. This would allow people who need just the Cache to run that directly, so good for integrators - I'm thinking use cases like Hibernate's simple caching and WildFly's clustering needs. Then we introduce a new module which depends on all of these and exposes a nicer API. People using Infinispan "directly" would use this, and specific system integrators could use the composition they prefer by picking only the fragments they want/need. > I am not opposing the idea but let's define the line between patriarchs > and plebeians. > > How big is the DSL API surface (which will be brought into commons)? It's one method on Cache to get started ;) I have no idea how large it is going to be: surely not trivial but I don't see that as a problem as in term of "fluent API" the IDE will guide you from the single starting point. It needs to support several options, hints, flags and not just "list()" but also streams, and reusing a Query definition as filter for other purposes. Finally the fact that one might want to query for projections, keys and combinations makes even the return type non-trivial. > > R. > > On 04/20/2017 02:08 PM, Tristan Tarrant wrote: >> Querying an Infinispan cache is currently a bit cumbersome. There are >> two paths: >> >> Ickle: >> Search.getQueryFactory(cache).create("...").list(); >> >> DSL (one possible example): >> Search.getQueryFactory(cache).from(class).[filters].build().list(); >> >> Ideally we should have something like: >> >> Ickle: >> cache.query("...").list(); >> >> DSL: >> cache.query(class).[filters].list(); >> >> Additionally, the query module is separate from infinispan-core. While >> this made sense when we didn't have non-indexed query capabilities (and >> is somewhat mitigated by the uberjars), I feel that query should be a >> 1st class citizen API-wise. >> For this reason I propose that we extract the query API to >> infinispan-commons, put the query SPI in infinispan-core together with >> the non-indexed implementation and have the hibernate-search backend as >> a pluggable implementation. >> >> Thoughts ? >> >> Tristan >> > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From ttarrant at redhat.com Thu Apr 20 10:06:39 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Thu, 20 Apr 2017 16:06:39 +0200 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> Message-ID: <88ad4f32-1d91-e0ef-fff5-c8cf7906dc95@redhat.com> On 20/04/2017 15:34, Dan Berindei wrote: >> How big is the DSL API surface (which will be brought into commons)? > > -1 from me to add anything in commons, I don't think allowing the > users to query both embedded caches and remote caches with the same > code is that important. I'd rather go the opposite way and remove the > BasicCache interface completely. Actually, we've had requests for interchangeable APIs... So, according to your strategy we either have each feature implemented with a divergent specific embedded or remote API, or each feature has its own feature-api with two separate feature-embedded and feature-remote implementations. Both plans sound terrible. Alternatively, we could go with an infinispan-api package (which Paul has been advocating for a long time) which would contain the various interfaces. Tristan -- Tristan Tarrant Infinispan Lead JBoss, a division of Red Hat From karesti at redhat.com Thu Apr 20 10:06:23 2017 From: karesti at redhat.com (Katia Aresti) Date: Thu, 20 Apr 2017 16:06:23 +0200 Subject: [infinispan-dev] to be a command, or not to be a command, that is the question In-Reply-To: References: Message-ID: Hi all Well, nobody spoke, so I consider that everybody agrees that I can take a decision like a big girl by myself ! :) I'm going to add 3 new commands, for merge, compute&computeIfPresent and computeIfAbsent. So I won't use the actual existing commands for the implementation : ReadWriteKeyCommand and ReadWriteKeyValueCommand even if I'm a DRY person and I love reusing code, I'm a KISS person too. I tested the implementation using these functional commands and IMHO : - merge and compute methods worth their own commands, they are very useful and we might want to adjust/optimize them individually - there are some technical issues related to the TypeConverterDelegatingAdvancedCache that makes me modify these existing functional commands with some hacky code that, for me, should be kept in commands like merge or compute with the correct documentation. They don't belong to a generic command. - Functional API is experimental right now. It might be non experimental in the near future, but we might decide to move to another thing. The 3 commands are already "coded" in my branches (not everything reviewed yet but soon). If one day we decide to change/simplify or we find a nice way to get rid of commands with a more generic one, removing and simplifying should be less painful than adding commands for these methods. That's all ! Cheers Katia On Wed, Apr 12, 2017 at 12:11 PM, Katia Aresti wrote: > Hi all, > > As you might know I'm working since my arrival, among other things, on > ISPN-5728 Jira [1], where the idea is to override the default ConcurrentMap > methods that are missing in CacheImpl (merge, replaceAll, compute ... ) > > I've created a pull-request [2] for compute, computeIfAbsent and > computeIfPresent methods, creating two new commands. By the way, I did the > same thing for the merge method in a branch that I haven't pull requested > yet. > > There is an opposite view between Radim and Will concerning the > implementation of these methods. To make it short : > In one side Will considers compute/merge best implementation should be as > a new Command (so what is already done) > In the other side, Radim considers adding another command is not necessary > as we could simple implement these methods using ReadWriteKeyCommand > > The detailed discussion and arguments of both sides is on GitHub [2] > > Before moving forward and making any choice by myself, I would like to > hear your opinions. For the record, it doesn't bother me redoing everything > if most people think like Radim because working on commands has helped me > to learn and understand more about infinispan internals, so this hasn't > been a waste of time for me. > > Katia > > [1] https://issues.jboss.org/browse/ISPN-5728 > [2] https://github.com/infinispan/infinispan/pull/5046 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170420/8f8b6c1a/attachment-0001.html From anistor at redhat.com Thu Apr 20 12:50:04 2017 From: anistor at redhat.com (Adrian Nistor) Date: Thu, 20 Apr 2017 19:50:04 +0300 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <88ad4f32-1d91-e0ef-fff5-c8cf7906dc95@redhat.com> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> <88ad4f32-1d91-e0ef-fff5-c8cf7906dc95@redhat.com> Message-ID: +1 for interchangeable apis +1 for infinispan-api module On 04/20/2017 05:06 PM, Tristan Tarrant wrote: > On 20/04/2017 15:34, Dan Berindei wrote: >>> How big is the DSL API surface (which will be brought into commons)? >> -1 from me to add anything in commons, I don't think allowing the >> users to query both embedded caches and remote caches with the same >> code is that important. I'd rather go the opposite way and remove the >> BasicCache interface completely. > Actually, we've had requests for interchangeable APIs... > > So, according to your strategy we either have each feature implemented > with a divergent specific embedded or remote API, or each feature has > its own feature-api with two separate feature-embedded and > feature-remote implementations. Both plans sound terrible. > > Alternatively, we could go with an infinispan-api package (which Paul > has been advocating for a long time) which would contain the various > interfaces. > > Tristan > From dan.berindei at gmail.com Thu Apr 20 13:35:33 2017 From: dan.berindei at gmail.com (Dan Berindei) Date: Thu, 20 Apr 2017 20:35:33 +0300 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <88ad4f32-1d91-e0ef-fff5-c8cf7906dc95@redhat.com> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> <88ad4f32-1d91-e0ef-fff5-c8cf7906dc95@redhat.com> Message-ID: On Thu, Apr 20, 2017 at 5:06 PM, Tristan Tarrant wrote: > On 20/04/2017 15:34, Dan Berindei wrote: >>> >>> How big is the DSL API surface (which will be brought into commons)? >> >> >> -1 from me to add anything in commons, I don't think allowing the >> users to query both embedded caches and remote caches with the same >> code is that important. I'd rather go the opposite way and remove the >> BasicCache interface completely. > > > Actually, we've had requests for interchangeable APIs... > > So, according to your strategy we either have each feature implemented with > a divergent specific embedded or remote API, or each feature has its own > feature-api with two separate feature-embedded and feature-remote > implementations. Both plans sound terrible. > Would a divergent embedded vs remote API be that bad? If the functionality really is different, then I'd rather have different APIs then force 2 different things use the same API. E.g. with BasicCache, IMO it would have been better to focus on the versioned conditional write operations, and remove all the non-versioned conditional write operations from RemoteCache. I'm sure we could have improved the versioned API a lot, but instead we worked mainly on the non-versioned API that we got from BasicCache. > Alternatively, we could go with an infinispan-api package (which Paul has > been advocating for a long time) which would contain the various interfaces. > > > Tristan > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat From anistor at redhat.com Thu Apr 20 14:16:01 2017 From: anistor at redhat.com (Adrian Nistor) Date: Thu, 20 Apr 2017 21:16:01 +0300 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> <0373f5c0-34a7-d5ca-6664-d71579aabd58@redhat.com> <88ad4f32-1d91-e0ef-fff5-c8cf7906dc95@redhat.com> Message-ID: <90f0ba42-ac72-0dc9-7406-96ff0ee8a1f3@redhat.com> If we are discussing how an Ickle query is defined, I insist we need identical APIs. I don't want to go over the long list of benefits of that. Let's just say it's trivial to implement because It's done already :). And now we also have a string dsl, which makes it even easier. We do not have ATM a unified way of executing an Ickle query. As Tristan showed in the mail starting this thread, the incantations are slightly different. And I'd like to have that unified too. The BasicCache/RemoteCache mishap is a textbook demonstration of a leaky abstraction, and is unrelated. Should not stop us from unifying query execution. But I would probably not add a 'query' method to the cache interfaces and would rather go with something similar to what Sanne proposed in a previous email in this thread (the so called 'Alternative direction'). On 04/20/2017 08:35 PM, Dan Berindei wrote: > On Thu, Apr 20, 2017 at 5:06 PM, Tristan Tarrant wrote: >> On 20/04/2017 15:34, Dan Berindei wrote: >>>> How big is the DSL API surface (which will be brought into commons)? >>> >>> -1 from me to add anything in commons, I don't think allowing the >>> users to query both embedded caches and remote caches with the same >>> code is that important. I'd rather go the opposite way and remove the >>> BasicCache interface completely. >> >> Actually, we've had requests for interchangeable APIs... >> >> So, according to your strategy we either have each feature implemented with >> a divergent specific embedded or remote API, or each feature has its own >> feature-api with two separate feature-embedded and feature-remote >> implementations. Both plans sound terrible. >> > Would a divergent embedded vs remote API be that bad? If the > functionality really is different, then I'd rather have different APIs > then force 2 different things use the same API. > > E.g. with BasicCache, IMO it would have been better to focus on the > versioned conditional write operations, and remove all the > non-versioned conditional write operations from RemoteCache. I'm sure > we could have improved the versioned API a lot, but instead we worked > mainly on the non-versioned API that we got from BasicCache. > >> Alternatively, we could go with an infinispan-api package (which Paul has >> been advocating for a long time) which would contain the various interfaces. >> >> >> Tristan >> >> -- >> Tristan Tarrant >> Infinispan Lead >> JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From emmanuel at hibernate.org Thu Apr 20 16:31:54 2017 From: emmanuel at hibernate.org (Emmanuel Bernard) Date: Thu, 20 Apr 2017 22:31:54 +0200 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> Message-ID: <75CF2311-B32C-4D13-B13C-62A41FEABCE8@hibernate.org> We did discuss something like cache.using(QueryableCache.class).createQuery("...").[build].list(); With some service locator system to have extensible and pluggable modules. We did discuss a potential problem but I forgot what it was. Emmanuel > On 20 Apr 2017, at 14:08, Tristan Tarrant wrote: > > Querying an Infinispan cache is currently a bit cumbersome. There are > two paths: > > Ickle: > Search.getQueryFactory(cache).create("...").list(); > > DSL (one possible example): > Search.getQueryFactory(cache).from(class).[filters].build().list(); > > Ideally we should have something like: > > Ickle: > cache.query("...").list(); > > DSL: > cache.query(class).[filters].list(); > > Additionally, the query module is separate from infinispan-core. While > this made sense when we didn't have non-indexed query capabilities (and > is somewhat mitigated by the uberjars), I feel that query should be a > 1st class citizen API-wise. > For this reason I propose that we extract the query API to > infinispan-commons, put the query SPI in infinispan-core together with > the non-indexed implementation and have the hibernate-search backend as > a pluggable implementation. > > Thoughts ? > > Tristan > > -- > Tristan Tarrant > Infinispan Lead > JBoss, a division of Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From sanne at infinispan.org Thu Apr 20 17:41:28 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Thu, 20 Apr 2017 22:41:28 +0100 Subject: [infinispan-dev] Infinispan Query API simplification In-Reply-To: <75CF2311-B32C-4D13-B13C-62A41FEABCE8@hibernate.org> References: <0432f3fb-994d-c219-2a3a-5dc6a8c59db6@redhat.com> <75CF2311-B32C-4D13-B13C-62A41FEABCE8@hibernate.org> Message-ID: On 20 April 2017 at 21:31, Emmanuel Bernard wrote: > We did discuss something like > > cache.using(QueryableCache.class).createQuery("...").[build].list(); > > With some service locator system to have extensible and pluggable modules. > > We did discuss a potential problem but I forgot what it was. The problem is that apparently people don't like having to know that. It's very similar to today's Search helper? In that case we might as well keep things as they are. Thanks, Sanne > > Emmanuel > >> On 20 Apr 2017, at 14:08, Tristan Tarrant wrote: >> >> Querying an Infinispan cache is currently a bit cumbersome. There are >> two paths: >> >> Ickle: >> Search.getQueryFactory(cache).create("...").list(); >> >> DSL (one possible example): >> Search.getQueryFactory(cache).from(class).[filters].build().list(); >> >> Ideally we should have something like: >> >> Ickle: >> cache.query("...").list(); >> >> DSL: >> cache.query(class).[filters].list(); >> >> Additionally, the query module is separate from infinispan-core. While >> this made sense when we didn't have non-indexed query capabilities (and >> is somewhat mitigated by the uberjars), I feel that query should be a >> 1st class citizen API-wise. >> For this reason I propose that we extract the query API to >> infinispan-commons, put the query SPI in infinispan-core together with >> the non-indexed implementation and have the hibernate-search backend as >> a pluggable implementation. >> >> Thoughts ? >> >> Tristan >> >> -- >> Tristan Tarrant >> Infinispan Lead >> JBoss, a division of Red Hat >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From slaskawi at redhat.com Fri Apr 21 09:55:22 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Fri, 21 Apr 2017 13:55:22 +0000 Subject: [infinispan-dev] Jenkins migration Message-ID: Hey! As you probably have heard I'm migrating our TeamCity installation [1] into Jenkins (temporarily in [2]). So far I've managed to migrate all Infinispan builds (with pull requests), C++/C# clients, JGroups and JGroups Kubernetes. I decided to use the new Pipeline [3] approach for the builds and keep the configuration along with the code (here's an example [4]). The configuration builds *refs/pull//head* for Pull Requests at the moment. I will switch it back to *refs/pull//merge* as soon as our PR queue size is ~20. Current pain points are: - Blue Ocean UI doesn't show tests. It has been reported in [5]. The workaround is to use the old Jenkins UI. - Windows VM doesn't start on demand (together with Vittorio we will be working on this) The rough plan is: - Apr 24th, move other 2 agents from TeamCity to Jenkins - Apr 24th, redirect ci.infinispan.org domain - May 4th, remove TeamCity Please let me know if you have any questions or concerns. Thanks, Sebastian [1] http://ci.infinispan.org/ [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com [3] https://jenkins.io/doc/book/pipeline/ [4] https://github.com/infinispan/infinispan/blob/master/Jenkinsfile [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170421/0b726864/attachment-0001.html From dan.berindei at gmail.com Fri Apr 21 11:11:39 2017 From: dan.berindei at gmail.com (Dan Berindei) Date: Fri, 21 Apr 2017 18:11:39 +0300 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: Message-ID: Looks like the invalid "control characters from U+0000 through U+001F" are the ANSI escape codes used by WildFly to color output. So we might be able to work around this by disabling the color output in WildFly in our integration tests. OTOH I'm fine with removing the Blue Ocean plugin for now, because its usability is sometime worse than the default UI's. E.g. when I click on the build results link in GitHub, 99.999% of the time I want to see the test results, but Blue Ocean thinks it's much better to show me some circles with question marks and exclamation points instead, and then keep me waiting for half a minute after I click on the tests link :) Cheers Dan On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec wrote: > Hey! > > As you probably have heard I'm migrating our TeamCity installation [1] > into Jenkins (temporarily in [2]). > > So far I've managed to migrate all Infinispan builds (with pull requests), > C++/C# clients, JGroups and JGroups Kubernetes. I decided to use the new > Pipeline [3] approach for the builds and keep the configuration along with > the code (here's an example [4]). > > The configuration builds *refs/pull//head* for Pull Requests at the > moment. I will switch it back to *refs/pull//merge* as soon as our PR > queue size is ~20. > > Current pain points are: > > - Blue Ocean UI doesn't show tests. It has been reported in [5]. The > workaround is to use the old Jenkins UI. > - Windows VM doesn't start on demand (together with Vittorio we will > be working on this) > > The rough plan is: > > - Apr 24th, move other 2 agents from TeamCity to Jenkins > - Apr 24th, redirect ci.infinispan.org domain > - May 4th, remove TeamCity > > Please let me know if you have any questions or concerns. > > Thanks, > Sebastian > > [1] http://ci.infinispan.org/ > [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com > [3] https://jenkins.io/doc/book/pipeline/ > [4] https://github.com/infinispan/infinispan/blob/master/Jenkinsfile > [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170421/91660573/attachment.html From anistor at redhat.com Sun Apr 23 13:14:20 2017 From: anistor at redhat.com (Adrian Nistor) Date: Sun, 23 Apr 2017 20:14:20 +0300 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: Message-ID: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> I also do not see much value in the current state of Blue Ocean. Better stick with the default ui. On 04/21/2017 06:11 PM, Dan Berindei wrote: > Looks like the invalid "control characters from U+0000 through U+001F" > are the ANSI escape codes used by WildFly to color output. So we > might be able to work around this by disabling the color output in > WildFly in our integration tests. > > OTOH I'm fine with removing the Blue Ocean plugin for now, because its > usability is sometime worse than the default UI's. E.g. when I click > on the build results link in GitHub, 99.999% of the time I want to see > the test results, but Blue Ocean thinks it's much better to show me > some circles with question marks and exclamation points instead, and > then keep me waiting for half a minute after I click on the tests link :) > > Cheers > Dan > > > On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec > > wrote: > > Hey! > > As you probably have heard I'm migrating our TeamCity installation > [1] into Jenkins (temporarily in [2]). > > So far I've managed to migrate all Infinispan builds (with pull > requests), C++/C# clients, JGroups and JGroups Kubernetes. I > decided to use the new Pipeline [3] approach for the builds and > keep the configuration along with the code (here's an example [4]). > > The configuration builds /refs/pull//head/ for Pull Requests > at the moment. I will switch it back to /refs/pull//merge/ as > soon as our PR queue size is ~20. > > Current pain points are: > > * Blue Ocean UI doesn't show tests. It has been reported in [5]. > The workaround is to use the old Jenkins UI. > * Windows VM doesn't start on demand (together with Vittorio we > will be working on this) > > The rough plan is: > > * Apr 24th, move other 2 agents from TeamCity to Jenkins > * Apr 24th, redirect ci.infinispan.org > domain > * May 4th, remove TeamCity > > Please let me know if you have any questions or concerns. > > Thanks, > Sebastian > > [1] http://ci.infinispan.org/ > [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com > > [3] https://jenkins.io/doc/book/pipeline/ > > [4] > https://github.com/infinispan/infinispan/blob/master/Jenkinsfile > > [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 > > -- > > SEBASTIAN?ASKAWIEC > > INFINISPAN DEVELOPER > > Red HatEMEA > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170423/20cc8ec5/attachment.html From rvansa at redhat.com Mon Apr 24 03:49:51 2017 From: rvansa at redhat.com (Radim Vansa) Date: Mon, 24 Apr 2017 09:49:51 +0200 Subject: [infinispan-dev] Jenkins migration In-Reply-To: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: I've heard that the default UI in Jenkins was the reason why we went with TC, and Blue Ocean was supposed to be the cure. Why was the default UI dismissed in the first place? R. On 04/23/2017 07:14 PM, Adrian Nistor wrote: > I also do not see much value in the current state of Blue Ocean. > Better stick with the default ui. > > On 04/21/2017 06:11 PM, Dan Berindei wrote: >> Looks like the invalid "control characters from U+0000 through >> U+001F" are the ANSI escape codes used by WildFly to color output. >> So we might be able to work around this by disabling the color output >> in WildFly in our integration tests. >> >> OTOH I'm fine with removing the Blue Ocean plugin for now, because >> its usability is sometime worse than the default UI's. E.g. when I >> click on the build results link in GitHub, 99.999% of the time I want >> to see the test results, but Blue Ocean thinks it's much better to >> show me some circles with question marks and exclamation points >> instead, and then keep me waiting for half a minute after I click on >> the tests link :) >> >> Cheers >> Dan >> >> >> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >> > wrote: >> >> Hey! >> >> As you probably have heard I'm migrating our TeamCity >> installation [1] into Jenkins (temporarily in [2]). >> >> So far I've managed to migrate all Infinispan builds (with pull >> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >> decided to use the new Pipeline [3] approach for the builds and >> keep the configuration along with the code (here's an example [4]). >> >> The configuration builds /refs/pull//head/ for Pull Requests >> at the moment. I will switch it back to /refs/pull//merge/ as >> soon as our PR queue size is ~20. >> >> Current pain points are: >> >> * Blue Ocean UI doesn't show tests. It has been reported in >> [5]. The workaround is to use the old Jenkins UI. >> * Windows VM doesn't start on demand (together with Vittorio we >> will be working on this) >> >> The rough plan is: >> >> * Apr 24th, move other 2 agents from TeamCity to Jenkins >> * Apr 24th, redirect ci.infinispan.org >> domain >> * May 4th, remove TeamCity >> >> Please let me know if you have any questions or concerns. >> >> Thanks, >> Sebastian >> >> [1] http://ci.infinispan.org/ >> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >> >> [3] https://jenkins.io/doc/book/pipeline/ >> >> [4] >> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >> >> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >> >> -- >> >> SEBASTIAN?ASKAWIEC >> >> INFINISPAN DEVELOPER >> >> Red HatEMEA >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa JBoss Performance Team From slaskawi at redhat.com Mon Apr 24 04:27:13 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 24 Apr 2017 08:27:13 +0000 Subject: [infinispan-dev] REST server, dealing with Serializable Message-ID: Hey Guys! I'm currently reworking REST interface and I'm scratching my head looking how we deal with Serializable [1][2]. The scenario assumes that server knows that cache stores a Serializable instance and moreover, it knows how to deserialize it (and convert it to XML/JSON, but that's the trivial part). I might be wrong, but I think both assumptions are questionable if not wrong. At first, how to distinguish a serialized instance of a class the server received [3] from a standard byte array? I can imagine someone using "Content-type: application/x-java-serialized-object" but it's very error prone. It also leads to the question number two - how the server will know that type of instance it is? This knowledge is essential for deserialization. I think the serialization/deserialization should be really done on the client side (but as I mentioned before, maybe I don't see some important use cases). I would like to remove it from refactored REST server. What do you think? Thanks, Sebastian [1] https://github.com/infinispan/infinispan/blob/master/server/rest/src/main/java/org/infinispan/rest/Server.java#L267-L313 [2] https://github.com/infinispan/infinispan/blob/master/server/rest/src/test/java/org/infinispan/rest/IntegrationTest.java#L636-L639 [3] Note that the input param here is byte[] https://github.com/infinispan/infinispan/blob/master/server/rest/src/main/java/org/infinispan/rest/Server.java#L458 -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/70265c97/attachment.html From gustavo at infinispan.org Mon Apr 24 04:45:20 2017 From: gustavo at infinispan.org (Gustavo Fernandes) Date: Mon, 24 Apr 2017 09:45:20 +0100 Subject: [infinispan-dev] REST server, dealing with Serializable In-Reply-To: References: Message-ID: With the upcoming transcoding support [1], each cache will have a MimeType configured for key and value. This will allow, for example, to differentiate from a byte[] that is a JPEG image, a gzipped text, some UTF-8 bytes, Jboss marshalled object, protostream marshalled message and so on. At runtime, it'll be possible to read and write to/from the cache specifying a different MimeType and the appropriate transcoder will convert the values on demand. Gustavo [1] https://issues.jboss.org/browse/ISPN-7417 On Mon, Apr 24, 2017 at 9:27 AM, Sebastian Laskawiec wrote: > Hey Guys! > > I'm currently reworking REST interface and I'm scratching my head looking > how we deal with Serializable [1][2]. > > The scenario assumes that server knows that cache stores a Serializable > instance and moreover, it knows how to deserialize it (and convert it to > XML/JSON, but that's the trivial part). I might be wrong, but I think both > assumptions are questionable if not wrong. At first, how to distinguish a > serialized instance of a class the server received [3] from a standard byte > array? I can imagine someone using "Content-type: > application/x-java-serialized-object" but it's very error prone. It also > leads to the question number two - how the server will know that type of > instance it is? This knowledge is essential for deserialization. > > I think the serialization/deserialization should be really done on the > client side (but as I mentioned before, maybe I don't see some important > use cases). I would like to remove it from refactored REST server. > > What do you think? > > Thanks, > Sebastian > > [1] https://github.com/infinispan/infinispan/blob/ > master/server/rest/src/main/java/org/infinispan/rest/Server.java#L267-L313 > [2] https://github.com/infinispan/infinispan/blob/ > master/server/rest/src/test/java/org/infinispan/rest/ > IntegrationTest.java#L636-L639 > [3] Note that the input param here is byte[] > https://github.com/infinispan/infinispan/blob/master/server/ > rest/src/main/java/org/infinispan/rest/Server.java#L458 > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/9ef6f4b7/attachment.html From sanne at infinispan.org Mon Apr 24 05:36:00 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Apr 2017 10:36:00 +0100 Subject: [infinispan-dev] REST server, dealing with Serializable In-Reply-To: References: Message-ID: On 24 April 2017 at 09:27, Sebastian Laskawiec wrote: > Hey Guys! > > I'm currently reworking REST interface and I'm scratching my head looking > how we deal with Serializable [1][2]. > > The scenario assumes that server knows that cache stores a Serializable > instance and moreover, it knows how to deserialize it (and convert it to > XML/JSON, but that's the trivial part). I might be wrong, but I think both > assumptions are questionable if not wrong. At first, how to distinguish a > serialized instance of a class the server received [3] from a standard byte > array? I can imagine someone using "Content-type: > application/x-java-serialized-object" but it's very error prone. It also > leads to the question number two - how the server will know that type of > instance it is? This knowledge is essential for deserialization. > > I think the serialization/deserialization should be really done on the > client side (but as I mentioned before, maybe I don't see some important > use cases). I would like to remove it from refactored REST server. > ?To answer this specific question: the server needs to be able to serialize / deserialize the objects for many reasons. Among these we have: - compatibility mode - "transcoding" among different clients expecting different types - indexless queries (read and compare specific fields) - indexed queries (index specific fields) - firing events? (on anything more useful else than "blob was changed") - avoid losing the data when encoding schemes are updated (e.g. an Infinispan update which includes improvements / fixes on the serialized representation ) It's probably not an exhaustive list as I didn't follow this subject closely, but I guess there are enough reasons there to clarify why just encoding/decoding at the client side is not good enough. Thanks, Sanne > > What do you think? > > Thanks, > Sebastian > > [1] https://github.com/infinispan/infinispan/blob/ > master/server/rest/src/main/java/org/infinispan/rest/Server.java#L267-L313 > [2] https://github.com/infinispan/infinispan/blob/ > master/server/rest/src/test/java/org/infinispan/rest/ > IntegrationTest.java#L636-L639 > [3] Note that the input param here is byte[] > https://github.com/infinispan/infinispan/blob/master/server/ > rest/src/main/java/org/infinispan/rest/Server.java#L458 > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/4332cd4d/attachment-0001.html From sanne at infinispan.org Mon Apr 24 06:25:09 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Apr 2017 11:25:09 +0100 Subject: [infinispan-dev] SLF4J shaded into infinispan-query-embedded ? Message-ID: When inspecting the stuff included in the "shaded" jar infinispan-query-embedded, I even found a copy of SLF4J in there. It's being relocated to the package "/infinispan/org/slf4j/" Is that intentional? Does it even work in this way? I doubt it would be able to bind to the right appenders, or be configured as intended. Thanks, Sanne From remerson at redhat.com Mon Apr 24 06:40:23 2017 From: remerson at redhat.com (Ryan Emerson) Date: Mon, 24 Apr 2017 06:40:23 -0400 (EDT) Subject: [infinispan-dev] SLF4J shaded into infinispan-query-embedded ? In-Reply-To: References: Message-ID: <1632880487.338891.1493030423002.JavaMail.zimbra@redhat.com> Previously slf4j was included as a dependency in infinispan-parent, so I assume this relocation was related to that. However, [1] removed slf4j from the parent so we should be to remove this relocation as well. Unless anybody else can provide more insight, I will issue a PR removing the relocation. [1] https://github.com/infinispan/infinispan/pull/5058/files Cheers Ryan ----- Original Message ----- > When inspecting the stuff included in the "shaded" jar > infinispan-query-embedded, I even found a copy of SLF4J in there. > > It's being relocated to the package "/infinispan/org/slf4j/" > > Is that intentional? Does it even work in this way? I doubt it would > be able to bind to the right appenders, or be configured as intended. > > Thanks, > Sanne > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > From sanne at infinispan.org Mon Apr 24 06:43:18 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Apr 2017 11:43:18 +0100 Subject: [infinispan-dev] SLF4J shaded into infinispan-query-embedded ? In-Reply-To: <1632880487.338891.1493030423002.JavaMail.zimbra@redhat.com> References: <1632880487.338891.1493030423002.JavaMail.zimbra@redhat.com> Message-ID: Thanks Ryan. Beyond removing the relocation, I also doubt if it should be included at all? I'm sincerely puzzled so I think we really need to hear from the others if this was intentional, or what the plan is. On 24 April 2017 at 11:40, Ryan Emerson wrote: > Previously slf4j was included as a dependency in infinispan-parent, so I assume this relocation was related to that. However, [1] removed slf4j from the parent so we should be to remove this relocation as well. Unless anybody else can provide more insight, I will issue a PR removing the relocation. > > [1] https://github.com/infinispan/infinispan/pull/5058/files > > Cheers > Ryan > > ----- Original Message ----- >> When inspecting the stuff included in the "shaded" jar >> infinispan-query-embedded, I even found a copy of SLF4J in there. >> >> It's being relocated to the package "/infinispan/org/slf4j/" >> >> Is that intentional? Does it even work in this way? I doubt it would >> be able to bind to the right appenders, or be configured as intended. >> >> Thanks, >> Sanne >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From sanne at infinispan.org Mon Apr 24 06:53:55 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Apr 2017 11:53:55 +0100 Subject: [infinispan-dev] SLF4J shaded into infinispan-query-embedded ? In-Reply-To: References: <1632880487.338891.1493030423002.JavaMail.zimbra@redhat.com> Message-ID: Just to add, I see now that JBoss Logging is included as well. On 24 April 2017 at 11:43, Sanne Grinovero wrote: > Thanks Ryan. Beyond removing the relocation, I also doubt if it should > be included at all? I'm sincerely puzzled so I think we really need to > hear from the others if this was intentional, or what the plan is. > > On 24 April 2017 at 11:40, Ryan Emerson wrote: >> Previously slf4j was included as a dependency in infinispan-parent, so I assume this relocation was related to that. However, [1] removed slf4j from the parent so we should be to remove this relocation as well. Unless anybody else can provide more insight, I will issue a PR removing the relocation. >> >> [1] https://github.com/infinispan/infinispan/pull/5058/files >> >> Cheers >> Ryan >> >> ----- Original Message ----- >>> When inspecting the stuff included in the "shaded" jar >>> infinispan-query-embedded, I even found a copy of SLF4J in there. >>> >>> It's being relocated to the package "/infinispan/org/slf4j/" >>> >>> Is that intentional? Does it even work in this way? I doubt it would >>> be able to bind to the right appenders, or be configured as intended. >>> >>> Thanks, >>> Sanne >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev From slaskawi at redhat.com Mon Apr 24 07:14:11 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 24 Apr 2017 11:14:11 +0000 Subject: [infinispan-dev] SLF4J shaded into infinispan-query-embedded ? In-Reply-To: References: <1632880487.338891.1493030423002.JavaMail.zimbra@redhat.com> Message-ID: The reason the SLF4J api was shaded was that some of our dependencies used it directly. Therefore client apps could experience ClassNotFoundExceptions. As the time passed we migrated many of them to JBoss Logging so that the problem naturally went away and Ryan removed it completely. On Mon, Apr 24, 2017 at 12:57 PM Sanne Grinovero wrote: > Just to add, I see now that JBoss Logging is included as well. > > > On 24 April 2017 at 11:43, Sanne Grinovero wrote: > > Thanks Ryan. Beyond removing the relocation, I also doubt if it should > > be included at all? I'm sincerely puzzled so I think we really need to > > hear from the others if this was intentional, or what the plan is. > > > > On 24 April 2017 at 11:40, Ryan Emerson wrote: > >> Previously slf4j was included as a dependency in infinispan-parent, so > I assume this relocation was related to that. However, [1] removed slf4j > from the parent so we should be to remove this relocation as well. Unless > anybody else can provide more insight, I will issue a PR removing the > relocation. > >> > >> [1] https://github.com/infinispan/infinispan/pull/5058/files > >> > >> Cheers > >> Ryan > >> > >> ----- Original Message ----- > >>> When inspecting the stuff included in the "shaded" jar > >>> infinispan-query-embedded, I even found a copy of SLF4J in there. > >>> > >>> It's being relocated to the package "/infinispan/org/slf4j/" > >>> > >>> Is that intentional? Does it even work in this way? I doubt it would > >>> be able to bind to the right appenders, or be configured as intended. > >>> > >>> Thanks, > >>> Sanne > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/dab8c3bd/attachment.html From slaskawi at redhat.com Mon Apr 24 07:19:23 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Mon, 24 Apr 2017 11:19:23 +0000 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: Hey! I uninstalled Blue Ocean plugin. I think it's worth to have another look at it as soon as 1.1.0 is released [1]. I also plan to migrate 2 TeamCity Agents into Jenkins very shortly (in 30 mins). @Tristan - may I ask you to redirect ci.infinispan.org to our new installation: http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com/ More comments inlined. Thanks, Sebastian [1] https://issues.jenkins-ci.org/browse/JENKINS-43751?focusedCommentId=296703 On Mon, Apr 24, 2017 at 9:50 AM Radim Vansa wrote: > I've heard that the default UI in Jenkins was the reason why we went > with TC, and Blue Ocean was supposed to be the cure. Why was the default > UI dismissed in the first place? > Once BlueOcean was installed (I think by Dan or Gustavo), it replaced the default UI without asking. > > R. > > On 04/23/2017 07:14 PM, Adrian Nistor wrote: > > I also do not see much value in the current state of Blue Ocean. > > Better stick with the default ui. > +1 > > > > On 04/21/2017 06:11 PM, Dan Berindei wrote: > >> Looks like the invalid "control characters from U+0000 through > >> U+001F" are the ANSI escape codes used by WildFly to color output. > >> So we might be able to work around this by disabling the color output > >> in WildFly in our integration tests. > I think it's not worth to invest more time in this. Let's switch to default UI and then try out BlueOcean once 1.1.0 is out. > >> > >> OTOH I'm fine with removing the Blue Ocean plugin for now, because > >> its usability is sometime worse than the default UI's. E.g. when I > >> click on the build results link in GitHub, 99.999% of the time I want > >> to see the test results, but Blue Ocean thinks it's much better to > >> show me some circles with question marks and exclamation points > >> instead, and then keep me waiting for half a minute after I click on > >> the tests link :) > +1 > >> > >> Cheers > >> Dan > >> > >> > >> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec > >> > wrote: > >> > >> Hey! > >> > >> As you probably have heard I'm migrating our TeamCity > >> installation [1] into Jenkins (temporarily in [2]). > >> > >> So far I've managed to migrate all Infinispan builds (with pull > >> requests), C++/C# clients, JGroups and JGroups Kubernetes. I > >> decided to use the new Pipeline [3] approach for the builds and > >> keep the configuration along with the code (here's an example [4]). > >> > >> The configuration builds /refs/pull//head/ for Pull Requests > >> at the moment. I will switch it back to /refs/pull//merge/ as > >> soon as our PR queue size is ~20. > >> > >> Current pain points are: > >> > >> * Blue Ocean UI doesn't show tests. It has been reported in > >> [5]. The workaround is to use the old Jenkins UI. > >> * Windows VM doesn't start on demand (together with Vittorio we > >> will be working on this) > >> > >> The rough plan is: > >> > >> * Apr 24th, move other 2 agents from TeamCity to Jenkins > >> * Apr 24th, redirect ci.infinispan.org > >> domain > >> * May 4th, remove TeamCity > >> > >> Please let me know if you have any questions or concerns. > >> > >> Thanks, > >> Sebastian > >> > >> [1] http://ci.infinispan.org/ > >> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com > >> > >> [3] https://jenkins.io/doc/book/pipeline/ > >> > >> [4] > >> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile > >> > >> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 > >> > >> -- > >> > >> SEBASTIAN?ASKAWIEC > >> > >> INFINISPAN DEVELOPER > >> > >> Red HatEMEA > >> > >> > >> > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> > >> > >> > >> > >> > >> _______________________________________________ > >> infinispan-dev mailing list > >> infinispan-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > > > > > > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/b5ae0140/attachment-0001.html From sanne at infinispan.org Mon Apr 24 07:29:02 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Apr 2017 12:29:02 +0100 Subject: [infinispan-dev] SLF4J shaded into infinispan-query-embedded ? In-Reply-To: References: <1632880487.338891.1493030423002.JavaMail.zimbra@redhat.com> Message-ID: On 24 April 2017 at 12:14, Sebastian Laskawiec wrote: > The reason the SLF4J api was shaded was that some of our dependencies used > it directly. Therefore client apps could experience > ClassNotFoundExceptions. > ?But frameworks like SLF4J don't work when shaded?? It's unable to pick up its extension points, and I'm not even sure if you're able to configure the logging categories of Infinispan this way - as these classes are probably binding to the shaded copy. > > As the time passed we migrated many of them to JBoss Logging so that the > problem naturally went away and Ryan removed it completely. > > On Mon, Apr 24, 2017 at 12:57 PM Sanne Grinovero > wrote: > >> Just to add, I see now that JBoss Logging is included as well. >> >> >> On 24 April 2017 at 11:43, Sanne Grinovero wrote: >> > Thanks Ryan. Beyond removing the relocation, I also doubt if it should >> > be included at all? I'm sincerely puzzled so I think we really need to >> > hear from the others if this was intentional, or what the plan is. >> > >> > On 24 April 2017 at 11:40, Ryan Emerson wrote: >> >> Previously slf4j was included as a dependency in infinispan-parent, so >> I assume this relocation was related to that. However, [1] removed slf4j >> from the parent so we should be to remove this relocation as well. Unless >> anybody else can provide more insight, I will issue a PR removing the >> relocation. >> >> >> >> [1] https://github.com/infinispan/infinispan/pull/5058/files >> >> >> >> Cheers >> >> Ryan >> >> >> >> ----- Original Message ----- >> >>> When inspecting the stuff included in the "shaded" jar >> >>> infinispan-query-embedded, I even found a copy of SLF4J in there. >> >>> >> >>> It's being relocated to the package "/infinispan/org/slf4j/" >> >>> >> >>> Is that intentional? Does it even work in this way? I doubt it would >> >>> be able to bind to the right appenders, or be configured as intended. >> >>> >> >>> Thanks, >> >>> Sanne >> >>> _______________________________________________ >> >>> infinispan-dev mailing list >> >>> infinispan-dev at lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >>> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/abf977a6/attachment-0001.html From sanne at infinispan.org Mon Apr 24 07:31:03 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Apr 2017 12:31:03 +0100 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: On 24 April 2017 at 12:19, Sebastian Laskawiec wrote: > Hey! > > I uninstalled Blue Ocean plugin. I think it's worth to have another look > at it as soon as 1.1.0 is released [1]. > > I also plan to migrate 2 TeamCity Agents into Jenkins very shortly (in 30 > mins). > > @Tristan - may I ask you to redirect ci.infinispan.org to our new > installation: http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com/ > ?Please don't assign a redirect to the volatile EC2 machine names as you'll regret it as soon as you have to do some maintenance on AWS. Assign a public floating IP to that machine first, then bind the ci.infinispan.org domain to that IP.? > > More comments inlined. > > Thanks, > Sebastian > > [1] https://issues.jenkins-ci.org/browse/JENKINS-43751? > focusedCommentId=296703 > > On Mon, Apr 24, 2017 at 9:50 AM Radim Vansa wrote: > >> I've heard that the default UI in Jenkins was the reason why we went >> with TC, and Blue Ocean was supposed to be the cure. Why was the default >> UI dismissed in the first place? >> > > Once BlueOcean was installed (I think by Dan or Gustavo), it replaced the > default UI without asking. > > >> >> R. >> >> On 04/23/2017 07:14 PM, Adrian Nistor wrote: >> > I also do not see much value in the current state of Blue Ocean. >> > Better stick with the default ui. >> > > +1 > > >> > >> > On 04/21/2017 06:11 PM, Dan Berindei wrote: >> >> Looks like the invalid "control characters from U+0000 through >> >> U+001F" are the ANSI escape codes used by WildFly to color output. >> >> So we might be able to work around this by disabling the color output >> >> in WildFly in our integration tests. >> > > I think it's not worth to invest more time in this. Let's switch to > default UI and then try out BlueOcean once 1.1.0 is out. > > >> >> >> >> OTOH I'm fine with removing the Blue Ocean plugin for now, because >> >> its usability is sometime worse than the default UI's. E.g. when I >> >> click on the build results link in GitHub, 99.999% of the time I want >> >> to see the test results, but Blue Ocean thinks it's much better to >> >> show me some circles with question marks and exclamation points >> >> instead, and then keep me waiting for half a minute after I click on >> >> the tests link :) >> > > +1 > > >> >> >> >> Cheers >> >> Dan >> >> >> >> >> >> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >> >> > wrote: >> >> >> >> Hey! >> >> >> >> As you probably have heard I'm migrating our TeamCity >> >> installation [1] into Jenkins (temporarily in [2]). >> >> >> >> So far I've managed to migrate all Infinispan builds (with pull >> >> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >> >> decided to use the new Pipeline [3] approach for the builds and >> >> keep the configuration along with the code (here's an example [4]). >> >> >> >> The configuration builds /refs/pull//head/ for Pull Requests >> >> at the moment. I will switch it back to /refs/pull//merge/ as >> >> soon as our PR queue size is ~20. >> >> >> >> Current pain points are: >> >> >> >> * Blue Ocean UI doesn't show tests. It has been reported in >> >> [5]. The workaround is to use the old Jenkins UI. >> >> * Windows VM doesn't start on demand (together with Vittorio we >> >> will be working on this) >> >> >> >> The rough plan is: >> >> >> >> * Apr 24th, move other 2 agents from TeamCity to Jenkins >> >> * Apr 24th, redirect ci.infinispan.org >> >> domain >> >> * May 4th, remove TeamCity >> >> >> >> Please let me know if you have any questions or concerns. >> >> >> >> Thanks, >> >> Sebastian >> >> >> >> [1] http://ci.infinispan.org/ >> >> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >> >> >> >> [3] https://jenkins.io/doc/book/pipeline/ >> >> >> >> [4] >> >> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >> >> >> >> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >> >> >> >> -- >> >> >> >> SEBASTIAN?ASKAWIEC >> >> >> >> INFINISPAN DEVELOPER >> >> >> >> Red HatEMEA >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev at lists.jboss.org >> >> >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > >> > >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> -- >> Radim Vansa >> JBoss Performance Team >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/35a7c0b0/attachment-0001.html From ttarrant at redhat.com Mon Apr 24 07:31:38 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Mon, 24 Apr 2017 13:31:38 +0200 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: Tristan is on PTO. He'll fix DNS on Wednesday :-) On 24 Apr 2017 13:26, "Sebastian Laskawiec" wrote: > Hey! > > I uninstalled Blue Ocean plugin. I think it's worth to have another look > at it as soon as 1.1.0 is released [1]. > > I also plan to migrate 2 TeamCity Agents into Jenkins very shortly (in 30 > mins). > > @Tristan - may I ask you to redirect ci.infinispan.org to our new > installation: http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com/ > > More comments inlined. > > Thanks, > Sebastian > > [1] https://issues.jenkins-ci.org/browse/JENKINS-43751? > focusedCommentId=296703 > > On Mon, Apr 24, 2017 at 9:50 AM Radim Vansa wrote: > >> I've heard that the default UI in Jenkins was the reason why we went >> with TC, and Blue Ocean was supposed to be the cure. Why was the default >> UI dismissed in the first place? >> > > Once BlueOcean was installed (I think by Dan or Gustavo), it replaced the > default UI without asking. > > >> >> R. >> >> On 04/23/2017 07:14 PM, Adrian Nistor wrote: >> > I also do not see much value in the current state of Blue Ocean. >> > Better stick with the default ui. >> > > +1 > > >> > >> > On 04/21/2017 06:11 PM, Dan Berindei wrote: >> >> Looks like the invalid "control characters from U+0000 through >> >> U+001F" are the ANSI escape codes used by WildFly to color output. >> >> So we might be able to work around this by disabling the color output >> >> in WildFly in our integration tests. >> > > I think it's not worth to invest more time in this. Let's switch to > default UI and then try out BlueOcean once 1.1.0 is out. > > >> >> >> >> OTOH I'm fine with removing the Blue Ocean plugin for now, because >> >> its usability is sometime worse than the default UI's. E.g. when I >> >> click on the build results link in GitHub, 99.999% of the time I want >> >> to see the test results, but Blue Ocean thinks it's much better to >> >> show me some circles with question marks and exclamation points >> >> instead, and then keep me waiting for half a minute after I click on >> >> the tests link :) >> > > +1 > > >> >> >> >> Cheers >> >> Dan >> >> >> >> >> >> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >> >> > wrote: >> >> >> >> Hey! >> >> >> >> As you probably have heard I'm migrating our TeamCity >> >> installation [1] into Jenkins (temporarily in [2]). >> >> >> >> So far I've managed to migrate all Infinispan builds (with pull >> >> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >> >> decided to use the new Pipeline [3] approach for the builds and >> >> keep the configuration along with the code (here's an example [4]). >> >> >> >> The configuration builds /refs/pull//head/ for Pull Requests >> >> at the moment. I will switch it back to /refs/pull//merge/ as >> >> soon as our PR queue size is ~20. >> >> >> >> Current pain points are: >> >> >> >> * Blue Ocean UI doesn't show tests. It has been reported in >> >> [5]. The workaround is to use the old Jenkins UI. >> >> * Windows VM doesn't start on demand (together with Vittorio we >> >> will be working on this) >> >> >> >> The rough plan is: >> >> >> >> * Apr 24th, move other 2 agents from TeamCity to Jenkins >> >> * Apr 24th, redirect ci.infinispan.org >> >> domain >> >> * May 4th, remove TeamCity >> >> >> >> Please let me know if you have any questions or concerns. >> >> >> >> Thanks, >> >> Sebastian >> >> >> >> [1] http://ci.infinispan.org/ >> >> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >> >> >> >> [3] https://jenkins.io/doc/book/pipeline/ >> >> >> >> [4] >> >> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >> >> >> >> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >> >> >> >> -- >> >> >> >> SEBASTIAN?ASKAWIEC >> >> >> >> INFINISPAN DEVELOPER >> >> >> >> Red HatEMEA >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev at lists.jboss.org >> >> >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > >> > >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> -- >> Radim Vansa >> JBoss Performance Team >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/c07b9a94/attachment.html From mudokonman at gmail.com Mon Apr 24 07:35:57 2017 From: mudokonman at gmail.com (William Burns) Date: Mon, 24 Apr 2017 11:35:57 +0000 Subject: [infinispan-dev] Weekly IRC meeting Message-ID: I will be unable to make it to the meeting today. Last week: ISPN-7479 Add Failover and Execution Policy to ClusterExecutor This has been completed and integrated. I also was looking at CI to find some stuff that could be fixed, I submitted ISPN-7743 DistributedStreamIteratorTest.testNode* fails randomly The fix may not work, but it should ease the test at a minimum. Will have to watch it later to see if it resurfaces. I also completed various backport work required and tried to relieve some of the PR queue stress and added comments to others. This week: I will most likely have to go on leave very soon, to be honest I am surprised I am not yet :D But I need to add documentation for both wiki and forums regarding ISPN-7479 and deprecate distributed execution service. The new cluster executor is completely detached from keys though so we will have to see if that is an issue (distributed streams should handle the bulk of that now). If time permits I also have some design stuff I need to add regarding stream for each with lock. Have a good week! - Will -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/968b5064/attachment.html From ttarrant at redhat.com Mon Apr 24 08:08:58 2017 From: ttarrant at redhat.com (Tristan Tarrant) Date: Mon, 24 Apr 2017 14:08:58 +0200 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: You mean exactly like the current CI is being handled? :-) On 24 Apr 2017 13:33, "Sanne Grinovero" wrote: > > > On 24 April 2017 at 12:19, Sebastian Laskawiec > wrote: > >> Hey! >> >> I uninstalled Blue Ocean plugin. I think it's worth to have another look >> at it as soon as 1.1.0 is released [1]. >> >> I also plan to migrate 2 TeamCity Agents into Jenkins very shortly (in 30 >> mins). >> >> @Tristan - may I ask you to redirect ci.infinispan.org to our new >> installation: http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com/ >> > > ?Please don't assign a redirect to the volatile EC2 machine names as > you'll regret it as soon as you have to do some maintenance on AWS. Assign > a public floating IP to that machine first, then bind the > ci.infinispan.org domain to that IP.? > > > >> >> More comments inlined. >> >> Thanks, >> Sebastian >> >> [1] https://issues.jenkins-ci.org/browse/JENKINS-43751?focus >> edCommentId=296703 >> >> On Mon, Apr 24, 2017 at 9:50 AM Radim Vansa wrote: >> >>> I've heard that the default UI in Jenkins was the reason why we went >>> with TC, and Blue Ocean was supposed to be the cure. Why was the default >>> UI dismissed in the first place? >>> >> >> Once BlueOcean was installed (I think by Dan or Gustavo), it replaced the >> default UI without asking. >> >> >>> >>> R. >>> >>> On 04/23/2017 07:14 PM, Adrian Nistor wrote: >>> > I also do not see much value in the current state of Blue Ocean. >>> > Better stick with the default ui. >>> >> >> +1 >> >> >>> > >>> > On 04/21/2017 06:11 PM, Dan Berindei wrote: >>> >> Looks like the invalid "control characters from U+0000 through >>> >> U+001F" are the ANSI escape codes used by WildFly to color output. >>> >> So we might be able to work around this by disabling the color output >>> >> in WildFly in our integration tests. >>> >> >> I think it's not worth to invest more time in this. Let's switch to >> default UI and then try out BlueOcean once 1.1.0 is out. >> >> >>> >> >>> >> OTOH I'm fine with removing the Blue Ocean plugin for now, because >>> >> its usability is sometime worse than the default UI's. E.g. when I >>> >> click on the build results link in GitHub, 99.999% of the time I want >>> >> to see the test results, but Blue Ocean thinks it's much better to >>> >> show me some circles with question marks and exclamation points >>> >> instead, and then keep me waiting for half a minute after I click on >>> >> the tests link :) >>> >> >> +1 >> >> >>> >> >>> >> Cheers >>> >> Dan >>> >> >>> >> >>> >> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >>> >> > wrote: >>> >> >>> >> Hey! >>> >> >>> >> As you probably have heard I'm migrating our TeamCity >>> >> installation [1] into Jenkins (temporarily in [2]). >>> >> >>> >> So far I've managed to migrate all Infinispan builds (with pull >>> >> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >>> >> decided to use the new Pipeline [3] approach for the builds and >>> >> keep the configuration along with the code (here's an example >>> [4]). >>> >> >>> >> The configuration builds /refs/pull//head/ for Pull Requests >>> >> at the moment. I will switch it back to /refs/pull//merge/ as >>> >> soon as our PR queue size is ~20. >>> >> >>> >> Current pain points are: >>> >> >>> >> * Blue Ocean UI doesn't show tests. It has been reported in >>> >> [5]. The workaround is to use the old Jenkins UI. >>> >> * Windows VM doesn't start on demand (together with Vittorio we >>> >> will be working on this) >>> >> >>> >> The rough plan is: >>> >> >>> >> * Apr 24th, move other 2 agents from TeamCity to Jenkins >>> >> * Apr 24th, redirect ci.infinispan.org >>> >> domain >>> >> * May 4th, remove TeamCity >>> >> >>> >> Please let me know if you have any questions or concerns. >>> >> >>> >> Thanks, >>> >> Sebastian >>> >> >>> >> [1] http://ci.infinispan.org/ >>> >> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >>> >> >>> >> [3] https://jenkins.io/doc/book/pipeline/ >>> >> >>> >> [4] >>> >> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >>> >> >> > >>> >> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >>> >> >>> >> -- >>> >> >>> >> SEBASTIAN?ASKAWIEC >>> >> >>> >> INFINISPAN DEVELOPER >>> >> >>> >> Red HatEMEA >>> >> >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> infinispan-dev mailing list >>> >> infinispan-dev at lists.jboss.org >>> >> >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> infinispan-dev mailing list >>> >> infinispan-dev at lists.jboss.org >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> > >>> > >>> > >>> > >>> > _______________________________________________ >>> > infinispan-dev mailing list >>> > infinispan-dev at lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> -- >>> Radim Vansa >>> JBoss Performance Team >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> >> SEBASTIAN ?ASKAWIEC >> >> INFINISPAN DEVELOPER >> >> Red Hat EMEA >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/4e7f8d38/attachment-0001.html From sanne at infinispan.org Mon Apr 24 08:24:47 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Apr 2017 13:24:47 +0100 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: On 24 April 2017 at 13:08, Tristan Tarrant wrote: > You mean exactly like the current CI is being handled? :-) > ?Cool, that other statement sounded like it could have misled you. > > On 24 Apr 2017 13:33, "Sanne Grinovero" wrote: > >> >> >> On 24 April 2017 at 12:19, Sebastian Laskawiec >> wrote: >> >>> Hey! >>> >>> I uninstalled Blue Ocean plugin. I think it's worth to have another look >>> at it as soon as 1.1.0 is released [1]. >>> >>> I also plan to migrate 2 TeamCity Agents into Jenkins very shortly (in >>> 30 mins). >>> >>> @Tristan - may I ask you to redirect ci.infinispan.org to our new >>> installation: http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com/ >>> >> >> ?Please don't assign a redirect to the volatile EC2 machine names as >> you'll regret it as soon as you have to do some maintenance on AWS. Assign >> a public floating IP to that machine first, then bind the >> ci.infinispan.org domain to that IP.? >> >> >> >>> >>> More comments inlined. >>> >>> Thanks, >>> Sebastian >>> >>> [1] https://issues.jenkins-ci.org/browse/JENKINS-43751?focus >>> edCommentId=296703 >>> >>> On Mon, Apr 24, 2017 at 9:50 AM Radim Vansa wrote: >>> >>>> I've heard that the default UI in Jenkins was the reason why we went >>>> with TC, and Blue Ocean was supposed to be the cure. Why was the default >>>> UI dismissed in the first place? >>>> >>> >>> Once BlueOcean was installed (I think by Dan or Gustavo), it replaced >>> the default UI without asking. >>> >>> >>>> >>>> R. >>>> >>>> On 04/23/2017 07:14 PM, Adrian Nistor wrote: >>>> > I also do not see much value in the current state of Blue Ocean. >>>> > Better stick with the default ui. >>>> >>> >>> +1 >>> >>> >>>> > >>>> > On 04/21/2017 06:11 PM, Dan Berindei wrote: >>>> >> Looks like the invalid "control characters from U+0000 through >>>> >> U+001F" are the ANSI escape codes used by WildFly to color output. >>>> >> So we might be able to work around this by disabling the color output >>>> >> in WildFly in our integration tests. >>>> >>> >>> I think it's not worth to invest more time in this. Let's switch to >>> default UI and then try out BlueOcean once 1.1.0 is out. >>> >>> >>>> >> >>>> >> OTOH I'm fine with removing the Blue Ocean plugin for now, because >>>> >> its usability is sometime worse than the default UI's. E.g. when I >>>> >> click on the build results link in GitHub, 99.999% of the time I want >>>> >> to see the test results, but Blue Ocean thinks it's much better to >>>> >> show me some circles with question marks and exclamation points >>>> >> instead, and then keep me waiting for half a minute after I click on >>>> >> the tests link :) >>>> >>> >>> +1 >>> >>> >>>> >> >>>> >> Cheers >>>> >> Dan >>>> >> >>>> >> >>>> >> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >>>> >> > wrote: >>>> >> >>>> >> Hey! >>>> >> >>>> >> As you probably have heard I'm migrating our TeamCity >>>> >> installation [1] into Jenkins (temporarily in [2]). >>>> >> >>>> >> So far I've managed to migrate all Infinispan builds (with pull >>>> >> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >>>> >> decided to use the new Pipeline [3] approach for the builds and >>>> >> keep the configuration along with the code (here's an example >>>> [4]). >>>> >> >>>> >> The configuration builds /refs/pull//head/ for Pull Requests >>>> >> at the moment. I will switch it back to /refs/pull//merge/ as >>>> >> soon as our PR queue size is ~20. >>>> >> >>>> >> Current pain points are: >>>> >> >>>> >> * Blue Ocean UI doesn't show tests. It has been reported in >>>> >> [5]. The workaround is to use the old Jenkins UI. >>>> >> * Windows VM doesn't start on demand (together with Vittorio we >>>> >> will be working on this) >>>> >> >>>> >> The rough plan is: >>>> >> >>>> >> * Apr 24th, move other 2 agents from TeamCity to Jenkins >>>> >> * Apr 24th, redirect ci.infinispan.org >>>> >> domain >>>> >> * May 4th, remove TeamCity >>>> >> >>>> >> Please let me know if you have any questions or concerns. >>>> >> >>>> >> Thanks, >>>> >> Sebastian >>>> >> >>>> >> [1] http://ci.infinispan.org/ >>>> >> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >>>> >> >>>> >> [3] https://jenkins.io/doc/book/pipeline/ >>>> >> >>>> >> [4] >>>> >> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >>>> >> >>> nsfile> >>>> >> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >>>> >> >>>> >> -- >>>> >> >>>> >> SEBASTIAN?ASKAWIEC >>>> >> >>>> >> INFINISPAN DEVELOPER >>>> >> >>>> >> Red HatEMEA >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> infinispan-dev mailing list >>>> >> infinispan-dev at lists.jboss.org >>>> >> >>>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> infinispan-dev mailing list >>>> >> infinispan-dev at lists.jboss.org >>>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> > >>>> > >>>> > >>>> > >>>> > _______________________________________________ >>>> > infinispan-dev mailing list >>>> > infinispan-dev at lists.jboss.org >>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> >>>> -- >>>> Radim Vansa >>>> JBoss Performance Team >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> -- >>> >>> SEBASTIAN ?ASKAWIEC >>> >>> INFINISPAN DEVELOPER >>> >>> Red Hat EMEA >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/28e0de20/attachment.html From gustavo at infinispan.org Mon Apr 24 10:43:22 2017 From: gustavo at infinispan.org (Gustavo Fernandes) Date: Mon, 24 Apr 2017 15:43:22 +0100 Subject: [infinispan-dev] Weekly IRC meeting logs 2017-04-24 Message-ID: Hello all, here are the logs for our meeting this week: http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2017/infinispan.2017-04-24-14.03.html Gustavo -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/ce67aa15/attachment-0001.html From dan.berindei at gmail.com Mon Apr 24 11:43:51 2017 From: dan.berindei at gmail.com (Dan Berindei) Date: Mon, 24 Apr 2017 18:43:51 +0300 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: Radim, it's not like the default UI became much better since we evaluated it against TeamCity, it's just that we no longer have a choice :) And I'm sure in the Blue Ocean UI will get more development effort than the default UI in the future, but at this time it's clearly unfinished. Cheers Dan On Mon, Apr 24, 2017 at 10:49 AM, Radim Vansa wrote: > I've heard that the default UI in Jenkins was the reason why we went > with TC, and Blue Ocean was supposed to be the cure. Why was the default > UI dismissed in the first place? > > R. > > On 04/23/2017 07:14 PM, Adrian Nistor wrote: >> I also do not see much value in the current state of Blue Ocean. >> Better stick with the default ui. >> >> On 04/21/2017 06:11 PM, Dan Berindei wrote: >>> Looks like the invalid "control characters from U+0000 through >>> U+001F" are the ANSI escape codes used by WildFly to color output. >>> So we might be able to work around this by disabling the color output >>> in WildFly in our integration tests. >>> >>> OTOH I'm fine with removing the Blue Ocean plugin for now, because >>> its usability is sometime worse than the default UI's. E.g. when I >>> click on the build results link in GitHub, 99.999% of the time I want >>> to see the test results, but Blue Ocean thinks it's much better to >>> show me some circles with question marks and exclamation points >>> instead, and then keep me waiting for half a minute after I click on >>> the tests link :) >>> >>> Cheers >>> Dan >>> >>> >>> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >>> > wrote: >>> >>> Hey! >>> >>> As you probably have heard I'm migrating our TeamCity >>> installation [1] into Jenkins (temporarily in [2]). >>> >>> So far I've managed to migrate all Infinispan builds (with pull >>> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >>> decided to use the new Pipeline [3] approach for the builds and >>> keep the configuration along with the code (here's an example [4]). >>> >>> The configuration builds /refs/pull//head/ for Pull Requests >>> at the moment. I will switch it back to /refs/pull//merge/ as >>> soon as our PR queue size is ~20. >>> >>> Current pain points are: >>> >>> * Blue Ocean UI doesn't show tests. It has been reported in >>> [5]. The workaround is to use the old Jenkins UI. >>> * Windows VM doesn't start on demand (together with Vittorio we >>> will be working on this) >>> >>> The rough plan is: >>> >>> * Apr 24th, move other 2 agents from TeamCity to Jenkins >>> * Apr 24th, redirect ci.infinispan.org >>> domain >>> * May 4th, remove TeamCity >>> >>> Please let me know if you have any questions or concerns. >>> >>> Thanks, >>> Sebastian >>> >>> [1] http://ci.infinispan.org/ >>> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >>> >>> [3] https://jenkins.io/doc/book/pipeline/ >>> >>> [4] >>> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >>> >>> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >>> >>> -- >>> >>> SEBASTIAN?ASKAWIEC >>> >>> INFINISPAN DEVELOPER >>> >>> Red HatEMEA >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From sanne at infinispan.org Mon Apr 24 12:11:10 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Mon, 24 Apr 2017 17:11:10 +0100 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: Personally I've always used Jenkins and for that reason I didn't like Infinispan to use TeamCity - mainly as I'm not used to its UI - but I'll admit that its capability to focus on the test history, highlighting the first failure in time and its "stability" over time on a per-test focus rather than a testsuite-focus has been very useful for the specific Infinispan needs. On 24 April 2017 at 16:43, Dan Berindei wrote: > Radim, it's not like the default UI became much better since we > evaluated it against TeamCity, it's just that we no longer have a > choice :) > > And I'm sure in the Blue Ocean UI will get more development effort > than the default UI in the future, but at this time it's clearly > unfinished. > > Cheers > Dan > > > On Mon, Apr 24, 2017 at 10:49 AM, Radim Vansa wrote: >> I've heard that the default UI in Jenkins was the reason why we went >> with TC, and Blue Ocean was supposed to be the cure. Why was the default >> UI dismissed in the first place? >> >> R. >> >> On 04/23/2017 07:14 PM, Adrian Nistor wrote: >>> I also do not see much value in the current state of Blue Ocean. >>> Better stick with the default ui. >>> >>> On 04/21/2017 06:11 PM, Dan Berindei wrote: >>>> Looks like the invalid "control characters from U+0000 through >>>> U+001F" are the ANSI escape codes used by WildFly to color output. >>>> So we might be able to work around this by disabling the color output >>>> in WildFly in our integration tests. >>>> >>>> OTOH I'm fine with removing the Blue Ocean plugin for now, because >>>> its usability is sometime worse than the default UI's. E.g. when I >>>> click on the build results link in GitHub, 99.999% of the time I want >>>> to see the test results, but Blue Ocean thinks it's much better to >>>> show me some circles with question marks and exclamation points >>>> instead, and then keep me waiting for half a minute after I click on >>>> the tests link :) >>>> >>>> Cheers >>>> Dan >>>> >>>> >>>> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >>>> > wrote: >>>> >>>> Hey! >>>> >>>> As you probably have heard I'm migrating our TeamCity >>>> installation [1] into Jenkins (temporarily in [2]). >>>> >>>> So far I've managed to migrate all Infinispan builds (with pull >>>> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >>>> decided to use the new Pipeline [3] approach for the builds and >>>> keep the configuration along with the code (here's an example [4]). >>>> >>>> The configuration builds /refs/pull//head/ for Pull Requests >>>> at the moment. I will switch it back to /refs/pull//merge/ as >>>> soon as our PR queue size is ~20. >>>> >>>> Current pain points are: >>>> >>>> * Blue Ocean UI doesn't show tests. It has been reported in >>>> [5]. The workaround is to use the old Jenkins UI. >>>> * Windows VM doesn't start on demand (together with Vittorio we >>>> will be working on this) >>>> >>>> The rough plan is: >>>> >>>> * Apr 24th, move other 2 agents from TeamCity to Jenkins >>>> * Apr 24th, redirect ci.infinispan.org >>>> domain >>>> * May 4th, remove TeamCity >>>> >>>> Please let me know if you have any questions or concerns. >>>> >>>> Thanks, >>>> Sebastian >>>> >>>> [1] http://ci.infinispan.org/ >>>> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >>>> >>>> [3] https://jenkins.io/doc/book/pipeline/ >>>> >>>> [4] >>>> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >>>> >>>> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >>>> >>>> -- >>>> >>>> SEBASTIAN?ASKAWIEC >>>> >>>> INFINISPAN DEVELOPER >>>> >>>> Red HatEMEA >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> -- >> Radim Vansa >> JBoss Performance Team >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From dan.berindei at gmail.com Mon Apr 24 16:42:23 2017 From: dan.berindei at gmail.com (Dan Berindei) Date: Mon, 24 Apr 2017 23:42:23 +0300 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: Thanks Sebastian! Looks like we can remove the claim plugin, too, because it doesn't work with pipelines: https://issues.jenkins-ci.org/browse/JENKINS-33969 Cheers Dan PS: I'm the one who installed the Blue Ocean plugin, but I thought it was opt-in for every user. Sorry about that! On Mon, Apr 24, 2017 at 2:19 PM, Sebastian Laskawiec wrote: > Hey! > > I uninstalled Blue Ocean plugin. I think it's worth to have another look > at it as soon as 1.1.0 is released [1]. > > I also plan to migrate 2 TeamCity Agents into Jenkins very shortly (in 30 > mins). > > @Tristan - may I ask you to redirect ci.infinispan.org to our new > installation: http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com/ > > More comments inlined. > > Thanks, > Sebastian > > [1] https://issues.jenkins-ci.org/browse/JENKINS-43751? > focusedCommentId=296703 > > On Mon, Apr 24, 2017 at 9:50 AM Radim Vansa wrote: > >> I've heard that the default UI in Jenkins was the reason why we went >> with TC, and Blue Ocean was supposed to be the cure. Why was the default >> UI dismissed in the first place? >> > > Once BlueOcean was installed (I think by Dan or Gustavo), it replaced the > default UI without asking. > > >> >> R. >> >> On 04/23/2017 07:14 PM, Adrian Nistor wrote: >> > I also do not see much value in the current state of Blue Ocean. >> > Better stick with the default ui. >> > > +1 > > >> > >> > On 04/21/2017 06:11 PM, Dan Berindei wrote: >> >> Looks like the invalid "control characters from U+0000 through >> >> U+001F" are the ANSI escape codes used by WildFly to color output. >> >> So we might be able to work around this by disabling the color output >> >> in WildFly in our integration tests. >> > > I think it's not worth to invest more time in this. Let's switch to > default UI and then try out BlueOcean once 1.1.0 is out. > > >> >> >> >> OTOH I'm fine with removing the Blue Ocean plugin for now, because >> >> its usability is sometime worse than the default UI's. E.g. when I >> >> click on the build results link in GitHub, 99.999% of the time I want >> >> to see the test results, but Blue Ocean thinks it's much better to >> >> show me some circles with question marks and exclamation points >> >> instead, and then keep me waiting for half a minute after I click on >> >> the tests link :) >> > > +1 > > >> >> >> >> Cheers >> >> Dan >> >> >> >> >> >> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >> >> > wrote: >> >> >> >> Hey! >> >> >> >> As you probably have heard I'm migrating our TeamCity >> >> installation [1] into Jenkins (temporarily in [2]). >> >> >> >> So far I've managed to migrate all Infinispan builds (with pull >> >> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >> >> decided to use the new Pipeline [3] approach for the builds and >> >> keep the configuration along with the code (here's an example [4]). >> >> >> >> The configuration builds /refs/pull//head/ for Pull Requests >> >> at the moment. I will switch it back to /refs/pull//merge/ as >> >> soon as our PR queue size is ~20. >> >> >> >> Current pain points are: >> >> >> >> * Blue Ocean UI doesn't show tests. It has been reported in >> >> [5]. The workaround is to use the old Jenkins UI. >> >> * Windows VM doesn't start on demand (together with Vittorio we >> >> will be working on this) >> >> >> >> The rough plan is: >> >> >> >> * Apr 24th, move other 2 agents from TeamCity to Jenkins >> >> * Apr 24th, redirect ci.infinispan.org >> >> domain >> >> * May 4th, remove TeamCity >> >> >> >> Please let me know if you have any questions or concerns. >> >> >> >> Thanks, >> >> Sebastian >> >> >> >> [1] http://ci.infinispan.org/ >> >> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >> >> >> >> [3] https://jenkins.io/doc/book/pipeline/ >> >> >> >> [4] >> >> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >> >> >> >> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >> >> >> >> -- >> >> >> >> SEBASTIAN?ASKAWIEC >> >> >> >> INFINISPAN DEVELOPER >> >> >> >> Red HatEMEA >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev at lists.jboss.org >> >> >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> > >> > >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> -- >> Radim Vansa >> JBoss Performance Team >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170424/d54db6ac/attachment-0001.html From galder at redhat.com Tue Apr 25 07:41:18 2017 From: galder at redhat.com (Galder Zamarreno) Date: Tue, 25 Apr 2017 07:41:18 -0400 (EDT) Subject: [infinispan-dev] RemoteCache putAll javadoc outdated? In-Reply-To: <1968302460.1182274.1493120425518.JavaMail.zimbra@redhat.com> Message-ID: <867143645.1182388.1493120478494.JavaMail.zimbra@redhat.com> Hey Will, Have we forgotten to update the RemoteCache.putAll javadoc after implementing ISPN-5266 and related jiras? https://github.com/infinispan/infinispan/blob/master/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/RemoteCache.java#L248 We're definitely not doing a remote call for each entry in the map anymore ;) Cheers, -- Galder Zamarre?o Infinispan, Red Hat From slaskawi at redhat.com Tue Apr 25 08:33:57 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Tue, 25 Apr 2017 12:33:57 +0000 Subject: [infinispan-dev] REST server, dealing with Serializable In-Reply-To: References: Message-ID: Ok, understood. The bottom line is - yes, we need to support Serializable. Thanks for clarification! Sebastian On Mon, Apr 24, 2017 at 11:38 AM Sanne Grinovero wrote: > On 24 April 2017 at 09:27, Sebastian Laskawiec > wrote: > >> Hey Guys! >> >> I'm currently reworking REST interface and I'm scratching my head looking >> how we deal with Serializable [1][2]. >> >> The scenario assumes that server knows that cache stores a Serializable >> instance and moreover, it knows how to deserialize it (and convert it to >> XML/JSON, but that's the trivial part). I might be wrong, but I think both >> assumptions are questionable if not wrong. At first, how to distinguish a >> serialized instance of a class the server received [3] from a standard byte >> array? I can imagine someone using "Content-type: >> application/x-java-serialized-object" but it's very error prone. It also >> leads to the question number two - how the server will know that type of >> instance it is? This knowledge is essential for deserialization. >> >> I think the serialization/deserialization should be really done on the >> client side (but as I mentioned before, maybe I don't see some important >> use cases). I would like to remove it from refactored REST server. >> > > ?To answer this specific question: the server needs to be able to > serialize / deserialize the objects for many reasons. > Among these we have: > - compatibility mode > - "transcoding" among different clients expecting different types > - indexless queries (read and compare specific fields) > - indexed queries (index specific fields) > - firing events? (on anything more useful else than "blob was changed") > - avoid losing the data when encoding schemes are updated (e.g. an > Infinispan update which includes improvements / fixes on the serialized > representation ) > > It's probably not an exhaustive list as I didn't follow this subject > closely, but I guess there are enough reasons there to clarify why just > encoding/decoding at the client side is not good enough. > > Thanks, > Sanne > > > > >> >> What do you think? >> >> Thanks, >> Sebastian >> >> [1] >> https://github.com/infinispan/infinispan/blob/master/server/rest/src/main/java/org/infinispan/rest/Server.java#L267-L313 >> [2] >> https://github.com/infinispan/infinispan/blob/master/server/rest/src/test/java/org/infinispan/rest/IntegrationTest.java#L636-L639 >> [3] Note that the input param here is byte[] >> https://github.com/infinispan/infinispan/blob/master/server/rest/src/main/java/org/infinispan/rest/Server.java#L458 >> -- >> >> SEBASTIAN ?ASKAWIEC >> >> INFINISPAN DEVELOPER >> >> Red Hat EMEA >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170425/b67092af/attachment.html From slaskawi at redhat.com Tue Apr 25 09:04:39 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Tue, 25 Apr 2017 13:04:39 +0000 Subject: [infinispan-dev] TLS/SNI support for Relay protocol Message-ID: Hey Bela! I've been thinking about Cross Site Replication using Relay protocol on Kubernetes/OpenShift. Most of the installations should use Federation [1] but I can also imagine a custom installation with two sites (let's call them X and Y) and totally separate networks. In that case, the flow through Kubernetes/OpenShift might look like the following: Site X, Pod 1 (sending relay message) ---> sending packets ---> the Internet ---> Site Y, Ingress/Route ---> Service ---> Site Y, Pod 1 Ingress/Routes and Services are Kubernetes/OpenShift "things". The former acts as a reverse proxy and the latter as a load balancer. Unfortunately Ingress/Routes don't have good support for custom protocols using TCP (they were designed with HTTP in mind). The only way to make it work is to use TLS with SNI [2][3]. So we would need to encrypt all traffic with TLS and use Application FQDN (a fully qualified application name, so something like this: infinispan-app-2-myproject.*site-x*.com) as SNI Hostname. Note that FQDN for both sites might be slightly different - Infinispan on site X might want to use FQDN containing site Y in its name and vice versa. I was wondering if it is possible to configure JGroups this way. If not, are there any plans to do so? Thanks, Sebastian [1] https://kubernetes.io/docs/concepts/cluster-administration/federation/ [2] https://www.ietf.org/rfc/rfc3546.txt [3] Look for "Passthrough Termination" https://docs.openshift.com/enterprise/3.2/architecture/core_concepts/routes.html#secured-routes -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170425/6f146577/attachment.html From slaskawi at redhat.com Wed Apr 26 06:17:31 2017 From: slaskawi at redhat.com (Sebastian Laskawiec) Date: Wed, 26 Apr 2017 10:17:31 +0000 Subject: [infinispan-dev] Jenkins migration In-Reply-To: References: <1d083600-e721-a2bc-bb0c-7a9539262f1b@redhat.com> Message-ID: Final update (I hope so)... Phew, it was a busy time.... http://ci.infinispan.org now points to Jenkins and all agents are up and running. Since all links on Pull Requests were broken (they were pointing to old Jenkins address), I retriggered all builds. The next things I would like to do (but no necessarily in closest future) are: 1. Add a new slave for "small jobs". It will be a slower machine but will perform all JGroups, C++ etc builds. 2. Add some monitoring. Thanks, Sebastian On Mon, Apr 24, 2017 at 11:49 PM Dan Berindei wrote: > Thanks Sebastian! > > Looks like we can remove the claim plugin, too, because it doesn't work > with pipelines: > > https://issues.jenkins-ci.org/browse/JENKINS-33969 > > Cheers > > Dan > > PS: I'm the one who installed the Blue Ocean plugin, but I thought it was > opt-in for every user. Sorry about that! > > > On Mon, Apr 24, 2017 at 2:19 PM, Sebastian Laskawiec > wrote: > >> Hey! >> >> I uninstalled Blue Ocean plugin. I think it's worth to have another look >> at it as soon as 1.1.0 is released [1]. >> >> I also plan to migrate 2 TeamCity Agents into Jenkins very shortly (in 30 >> mins). >> >> @Tristan - may I ask you to redirect ci.infinispan.org to our new >> installation: http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com/ >> >> More comments inlined. >> >> Thanks, >> Sebastian >> >> [1] >> https://issues.jenkins-ci.org/browse/JENKINS-43751?focusedCommentId=296703 >> >> On Mon, Apr 24, 2017 at 9:50 AM Radim Vansa wrote: >> >>> I've heard that the default UI in Jenkins was the reason why we went >>> with TC, and Blue Ocean was supposed to be the cure. Why was the default >>> UI dismissed in the first place? >>> >> >> Once BlueOcean was installed (I think by Dan or Gustavo), it replaced the >> default UI without asking. >> >> >>> >>> R. >>> >>> On 04/23/2017 07:14 PM, Adrian Nistor wrote: >>> > I also do not see much value in the current state of Blue Ocean. >>> > Better stick with the default ui. >>> >> >> +1 >> >> >>> > >>> > On 04/21/2017 06:11 PM, Dan Berindei wrote: >>> >> Looks like the invalid "control characters from U+0000 through >>> >> U+001F" are the ANSI escape codes used by WildFly to color output. >>> >> So we might be able to work around this by disabling the color output >>> >> in WildFly in our integration tests. >>> >> >> I think it's not worth to invest more time in this. Let's switch to >> default UI and then try out BlueOcean once 1.1.0 is out. >> >> >>> >> >>> >> OTOH I'm fine with removing the Blue Ocean plugin for now, because >>> >> its usability is sometime worse than the default UI's. E.g. when I >>> >> click on the build results link in GitHub, 99.999% of the time I want >>> >> to see the test results, but Blue Ocean thinks it's much better to >>> >> show me some circles with question marks and exclamation points >>> >> instead, and then keep me waiting for half a minute after I click on >>> >> the tests link :) >>> >> >> +1 >> >> >>> >> >>> >> Cheers >>> >> Dan >>> >> >>> >> >>> >> On Fri, Apr 21, 2017 at 4:55 PM, Sebastian Laskawiec >>> >> > wrote: >>> >> >>> >> Hey! >>> >> >>> >> As you probably have heard I'm migrating our TeamCity >>> >> installation [1] into Jenkins (temporarily in [2]). >>> >> >>> >> So far I've managed to migrate all Infinispan builds (with pull >>> >> requests), C++/C# clients, JGroups and JGroups Kubernetes. I >>> >> decided to use the new Pipeline [3] approach for the builds and >>> >> keep the configuration along with the code (here's an example >>> [4]). >>> >> >>> >> The configuration builds /refs/pull//head/ for Pull Requests >>> >> at the moment. I will switch it back to /refs/pull//merge/ as >>> >> soon as our PR queue size is ~20. >>> >> >>> >> Current pain points are: >>> >> >>> >> * Blue Ocean UI doesn't show tests. It has been reported in >>> >> [5]. The workaround is to use the old Jenkins UI. >>> >> * Windows VM doesn't start on demand (together with Vittorio we >>> >> will be working on this) >>> >> >>> >> The rough plan is: >>> >> >>> >> * Apr 24th, move other 2 agents from TeamCity to Jenkins >>> >> * Apr 24th, redirect ci.infinispan.org >>> >> domain >>> >> * May 4th, remove TeamCity >>> >> >>> >> Please let me know if you have any questions or concerns. >>> >> >>> >> Thanks, >>> >> Sebastian >>> >> >>> >> [1] http://ci.infinispan.org/ >>> >> [2] http://ec2-52-215-14-157.eu-west-1.compute.amazonaws.com >>> >> >>> >> [3] https://jenkins.io/doc/book/pipeline/ >>> >> >>> >> [4] >>> >> https://github.com/infinispan/infinispan/blob/master/Jenkinsfile >>> >> >> > >>> >> [5] https://issues.jenkins-ci.org/browse/JENKINS-43751 >>> >> >>> >> -- >>> >> >>> >> SEBASTIAN?ASKAWIEC >>> >> >>> >> INFINISPAN DEVELOPER >>> >> >>> >> Red HatEMEA >>> >> >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> infinispan-dev mailing list >>> >> infinispan-dev at lists.jboss.org >>> >> >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> infinispan-dev mailing list >>> >> infinispan-dev at lists.jboss.org >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> > >>> > >>> > >>> > >>> > _______________________________________________ >>> > infinispan-dev mailing list >>> > infinispan-dev at lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> -- >>> Radim Vansa >>> JBoss Performance Team >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> >> SEBASTIAN ?ASKAWIEC >> >> INFINISPAN DEVELOPER >> >> Red Hat EMEA >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- SEBASTIAN ?ASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170426/c78f58da/attachment-0001.html From rory.odonnell at oracle.com Fri Apr 28 05:48:42 2017 From: rory.odonnell at oracle.com (Rory O'Donnell) Date: Fri, 28 Apr 2017 10:48:42 +0100 Subject: [infinispan-dev] JDK 9 EA Build 167 and JDK 8u152 build 03 are available on jdk.java.net Message-ID: Hi Galder, *JDK 9 Early Access* build 167 is available at the new location : - jdk.java.net/9/ A summary of all the changes in this build are listed here . One change that maybe of interest is : * JEP 291: Deprecate the Concurrent Mark Sweep (CMS) Garbage Collector [1] * * *JDK 8u152 Early Access* build 03 is available at the new location : - jdk.java.net/8/ More information on the change of location for Early Access builds. [2] NOTE: - Oracle's JRE and JDK Cryptographic Roadmap has been updated since last availability email [3] Rgds,Rory [1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2017-April/005766.html [2] http://mail.openjdk.java.net/pipermail/adoption-discuss/2017-April/001610.html [3] https://www.java.com/en/jre-jdk-cryptoroadmap.html -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170428/f35dfcd0/attachment.html From karesti at redhat.com Fri Apr 28 06:38:24 2017 From: karesti at redhat.com (Katia Aresti) Date: Fri, 28 Apr 2017 12:38:24 +0200 Subject: [infinispan-dev] Unwrapping exceptions Message-ID: Hi all ! Radim pointed me to this thread discussing the exceptions launched by the lambda executed by the user. So, I've came accros this problem right now with the compute method. ComputeIfAbsent is used by the QueryCache [1] This method is now a Command, so when the wrapped lambda throws an exception, [2], the expected exception is the one raised by the lambda. But with my modifications, this exception is wrapped in a CacheException. I discussed with Adrien yesterday, and IMHO and his, a CacheException is not the same thing as the exception raised inside the lambda. Moreover, in this particular case, I don't know if users some code could be broken if we make the user get a CacheException that wrappes the ParseException instead of the ParseException itself. How can I fix the problem ? Should we correct the tests and say that, from now on, CacheException will be raised ? Should we handle this CacheException in the QueryCache class when computeIfAbsent is called ? Should we propagate the lambda's exception as it is ? Katia [1] https://github.com/infinispan/infinispan/blob/master/query/src/main/java/org/infinispan/query/dsl/embedded/impl/QueryCache.java#L79 [2] https://github.com/infinispan/infinispan/blob/master/query/src/test/java/org/infinispan/query/dsl/embedded/QueryDslConditionsTest.java#L1913 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170428/ce6b1d49/attachment.html From sanne at infinispan.org Fri Apr 28 06:56:36 2017 From: sanne at infinispan.org (Sanne Grinovero) Date: Fri, 28 Apr 2017 11:56:36 +0100 Subject: [infinispan-dev] Unwrapping exceptions In-Reply-To: References: Message-ID: Personally as a user I'd expect to have a CacheException raised if and only if it's not caused by my own code. Imagine my own lambda is explicitly throwing an exception of a type of my choice, it would be nice to receive that error and not a different one. On 28 April 2017 at 11:38, Katia Aresti wrote: > Hi all ! > > Radim pointed me to this thread discussing the exceptions launched by the > lambda executed by the user. > > So, I've came accros this problem right now with the compute method. > > ComputeIfAbsent is used by the QueryCache [1] > > This method is now a Command, so when the wrapped lambda throws an > exception, [2], the expected exception is the one raised by the lambda. But > with my modifications, this exception is wrapped in a CacheException. > > I discussed with Adrien yesterday, and IMHO and his, a CacheException is not > the same thing as the exception raised inside the lambda. Moreover, in this > particular case, I don't know if users some code could be broken if we make > the user get a CacheException that wrappes the ParseException instead of the > ParseException itself. > > How can I fix the problem ? > Should we correct the tests and say that, from now on, CacheException will > be raised ? > Should we handle this CacheException in the QueryCache class when > computeIfAbsent is called ? > Should we propagate the lambda's exception as it is ? > > Katia > > [1] > https://github.com/infinispan/infinispan/blob/master/query/src/main/java/org/infinispan/query/dsl/embedded/impl/QueryCache.java#L79 > [2] > https://github.com/infinispan/infinispan/blob/master/query/src/test/java/org/infinispan/query/dsl/embedded/QueryDslConditionsTest.java#L1913 > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev From belaban at mailbox.org Fri Apr 28 09:01:52 2017 From: belaban at mailbox.org (Bela Ban) Date: Fri, 28 Apr 2017 15:01:52 +0200 Subject: [infinispan-dev] TLS/SNI support for Relay protocol In-Reply-To: References: Message-ID: <7e0e460b-1a9b-5295-bcc8-fa09dd2545f2@mailbox.org> Hi Sebastian, I must confess I understood about 30% of your email (I understood "Bela" and "JGroups" LOL :-))... Cross-site replication works by bridging local clusters with a 'global' cluster. The endpoints (IP addresses:ports) of this global cluster need to be listed (or found dynamically), and at the end of the day, I don't care how we get them as long as we can establish (TCP) connections to them. TCP, TCP_NIO2 and UDP are currently the only options, but if this only works with HTTP, we could think about an HTTP protocol which sends and receives serialized (binary) JGroups messages. OTOH if we have site masters which have addresses that are accessible from any of the local cluster nodes plus the other site masters, then I don't see why we would need routes. So if we can use Federation to (1) find endpoints of the global cluster and (2) and SNI/TLS to exchange messages between site masters, I'm all for building a specialized setup for Kubernetes/Openshift. Although, as I mentioned above, I don't currently see what the value-add of (2) is. Let's discuss this in a chat. Cheers, On 25/04/17 15:04, Sebastian Laskawiec wrote: > Hey Bela! > > I've been thinking about Cross Site Replication using Relay protocol on > Kubernetes/OpenShift. Most of the installations should use Federation > [1] but I can also imagine a custom installation with two sites (let's > call them X and Y) and totally separate networks. In that case, the flow > through Kubernetes/OpenShift might look like the following: > > Site X, Pod 1 (sending relay message) ---> sending packets ---> the > Internet ---> Site Y, Ingress/Route ---> Service ---> Site Y, Pod 1 > > Ingress/Routes and Services are Kubernetes/OpenShift "things". The > former acts as a reverse proxy and the latter as a load balancer. > > Unfortunately Ingress/Routes don't have good support for custom > protocols using TCP (they were designed with HTTP in mind). The only way > to make it work is to use TLS with SNI [2][3]. So we would need to > encrypt all traffic with TLS and use Application FQDN (a fully qualified > application name, so something like > this: infinispan-app-2-myproject.*site-x*.com) as SNI Hostname. Note > that FQDN for both sites might be slightly different - Infinispan on > site X might want to use FQDN containing site Y in its name and vice versa. > > I was wondering if it is possible to configure JGroups this way. If not, > are there any plans to do so? > > Thanks, > Sebastian > > [1] https://kubernetes.io/docs/concepts/cluster-administration/federation/ > [2] https://www.ietf.org/rfc/rfc3546.txt > [3] Look for "Passthrough Termination" > https://docs.openshift.com/enterprise/3.2/architecture/core_concepts/routes.html#secured-routes > -- > > SEBASTIAN ?ASKAWIEC > > INFINISPAN DEVELOPER > > Red Hat EMEA > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- Bela Ban | http://www.jgroups.org