From mmarkus at redhat.com  Mon Jun  2 05:42:03 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Mon, 2 Jun 2014 10:42:03 +0100
Subject: [infinispan-dev] book on Infinispan
In-Reply-To: <D4AD1AC5-6760-4180-8D11-BEFB193EB5F5@redhat.com>
References: <D4AD1AC5-6760-4180-8D11-BEFB193EB5F5@redhat.com>
Message-ID: <7DF2D821-E83E-4209-877B-3F915BD44B60@redhat.com>

Hi Wagner,

How's the book progressing? If you want someone to review it happy to take a look.

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)




From sanne at infinispan.org  Mon Jun  2 10:13:00 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Mon, 2 Jun 2014 15:13:00 +0100
Subject: [infinispan-dev] JDK8 master failures
In-Reply-To: <9EE85C54-CEC6-4D29-9AD4-D6EDB703097D@redhat.com>
References: <9EE85C54-CEC6-4D29-9AD4-D6EDB703097D@redhat.com>
Message-ID: <CAFm4XO18RwBCK6uwg3i-bidPqx=XYJ-+O9rOwY3FJ74-GNKmFQ@mail.gmail.com>

I also have a failure in infinispan-core, so not getting beyond to
test other modules:

Failed tests:
  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
locals st...


testCheckThreadLocalLeaks(org.infinispan.util.ThreadLocalLeakTest)
Time elapsed: 0.544 sec  <<< FAILURE!
java.lang.IllegalStateException: Thread locals still present:
{ForkThread-1,ThreadLocalLeakTest={java.lang.ThreadLocal at 2978515=org.infinispan.commons.util.concurrent.jdk8backported.EquivalentConcurrentHashMapV8$CounterHashCode at 29a22707}}
at org.infinispan.util.ThreadLocalLeakTest.testCheckThreadLocalLeaks(ThreadLocalLeakTest.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)


Results :

Failed tests:
  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
locals st...

Tests run: 5389, Failures: 1, Errors: 0, Skipped: 0


On 2 June 2014 15:07, Galder Zamarre?o <galder at redhat.com> wrote:
> Seems like there are tons of JDK8 failures on master:
>
> Caused by: java.lang.NoSuchMethodException: org.infinispan.security.SecureCacheTestDriver.testReplaceAll_BiFunction(org.infinispan.security.SecureCache)
>
> @Tristan, ring a bell?
>
> Cheers,
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
>


From mmarkus at redhat.com  Mon Jun  2 12:00:22 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Mon, 2 Jun 2014 17:00:22 +0100
Subject: [infinispan-dev] running ISPN in Karaf - documentation
In-Reply-To: <538874E9.5060901@redhat.com>
References: <996C94CA-3552-46F5-A012-2B0DB4B14324@redhat.com>
	<538874E9.5060901@redhat.com>
Message-ID: <D29FB25E-339B-473C-B125-250B2DDE484D@redhat.com>


On May 30, 2014, at 13:09, Ion Savin <isavin at redhat.com> wrote:

> Hi Mircea,
> 
> On 05/29/2014 01:38 PM, Mircea Markus wrote:
>> Hi Ion,
>> 
>> Would be good to have a documentation page + blog on how to install ISPN in Karaf: where to take the features file from, how to install the jars, what are the limitations (e.g. query not supported yet) etc. Martin has written a nice blog for the hotrod client already: http://blog.infinispan.org/search/label/osgi
>> 
>> Cheers,
>> 
> 
> The process is the same for the other modules also.
> 
> At the moment each bundle/module when installed using Karaf Features will pull in the modules it depends on (e.g. installing infinispan-cachestore-remote will also install commons, core and hotrod-client).
> 
> Perhaps in addition to this it might help to have an additional feature file which would install the bundles frequently used together (similar to [2])
> 
> I'll add this and document the process though it's not different from what Martin described.


Thanks, would be be very helpful for someone who just googles to see if the integration is possible.


> 
> [1] https://github.com/infinispan/infinispan/pull/2540
> [2] https://issues.jboss.org/browse/ISPN-4333
> 
> Regards,
> Ion Savin

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From ttarrant at redhat.com  Tue Jun  3 03:53:22 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Tue, 03 Jun 2014 09:53:22 +0200
Subject: [infinispan-dev] Uber Jars
Message-ID: <538D7EF2.5060103@redhat.com>

Dear all,

on Thursday I issued a PR [1] to introduce Uber Jars, i.e. single jars 
which wrap our multitude of jars and some of the transitive 
dependencies, but it was (rightly) pointed out that we should have a 
little discussion here first.

Firstly, I'm using the maven shade plugin which repackages multiple jars 
into one with:
- automatic transitive resolution
- the ability to include/exclude certain jars
- move classes if necessary to other packages to avoid conflicts
- rewrite the POM with the new dependencies.

Here is my global strategy:
- define a set of uber-jars (see below)
- include all non-optional dependencies in each uber-jar except for the 
specification Jars (e.g. javax.transaction and javax.persistence)
- rename some internal-only dependencies to avoid conflicts
- uber jars MUST NOT inherit from infinispan-parent (too much cruft in 
there) but only from infinispan-bom.

The Uber Jars
- infinispan-embedded-all (infinispan-commons, infinispan-core, jgroups, 
jboss-marshalling-osgi, jboss-logging, infinispan-cachestore-jdbc, 
infinispan-cachestore-jpa, infinispan-cachestore-leveldb)
- infinispan-remote-all (infinispan-commons, infinispan-client-hotrod, 
commons-pool, jboss-marshalling-osgi, jboss-logging, 
infinispan-remote-query-client, infinispan-query-dsl, 
infinispan-protostream, protobuf-java)
- infinispan-query-all (infinispan-query, infinispan-query-dsl, 
hibernate-hql-parser, antlr, stringtemplate, hibernate-hql-lucene, 
hibernate-search-engine, infinispan-lucene-v4, 
hibernate-search-analyzers, lucene*, solr*, avro, jackson-core, 
jackson-mapper, paranamer, apache-compress, infinispan-lucene-directory, 
hibernate-search-infinispan, hibernate-commons-annotations). This 
package will depend on infinispan-embedded-all and we should only make a 
Lucene v4 version).

Discuss :)


Tristan



[1] https://github.com/infinispan/infinispan/pull/2589

From mmarkus at redhat.com  Tue Jun  3 04:57:27 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Tue, 3 Jun 2014 09:57:27 +0100
Subject: [infinispan-dev] Uber Jars
In-Reply-To: <538D7EF2.5060103@redhat.com>
References: <538D7EF2.5060103@redhat.com>
Message-ID: <1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>

On Jun 3, 2014, at 8:53, Tristan Tarrant <ttarrant at redhat.com> wrote:

> Dear all,
> 
> on Thursday I issued a PR [1] to introduce Uber Jars, i.e. single jars 
> which wrap our multitude of jars and some of the transitive 
> dependencies, but it was (rightly) pointed out that we should have a 
> little discussion here first.
> 
> Firstly, I'm using the maven shade plugin which repackages multiple jars 
> into one with:
> - automatic transitive resolution
> - the ability to include/exclude certain jars
> - move classes if necessary to other packages to avoid conflicts
> - rewrite the POM with the new dependencies.
> 
> Here is my global strategy:
> - define a set of uber-jars (see below)
> - include all non-optional dependencies in each uber-jar except for the 
> specification Jars (e.g. javax.transaction and javax.persistence)
> - rename some internal-only dependencies to avoid conflicts
> - uber jars MUST NOT inherit from infinispan-parent (too much cruft in 
> there) but only from infinispan-bom.

Just for my own understanding, what is the rationale behind this? 

> 
> The Uber Jars
> - infinispan-embedded-all (infinispan-commons, infinispan-core, jgroups, 
> jboss-marshalling-osgi, jboss-logging, infinispan-cachestore-jdbc, 
> infinispan-cachestore-jpa, infinispan-cachestore-leveldb)
> - infinispan-remote-all (infinispan-commons, infinispan-client-hotrod, 
> commons-pool, jboss-marshalling-osgi, jboss-logging, 
> infinispan-remote-query-client, infinispan-query-dsl, 
> infinispan-protostream, protobuf-java)
> - infinispan-query-all (infinispan-query, infinispan-query-dsl, 
> hibernate-hql-parser, antlr, stringtemplate, hibernate-hql-lucene, 
> hibernate-search-engine, infinispan-lucene-v4, 
> hibernate-search-analyzers, lucene*, solr*, avro, jackson-core, 
> jackson-mapper, paranamer, apache-compress, infinispan-lucene-directory, 
> hibernate-search-infinispan, hibernate-commons-annotations). This 
> package will depend on infinispan-embedded-all and we should only make a 
> Lucene v4 version).
> 
> Discuss :)

Much simpler this way, so why not making this the default packaging? 

> 
> 
> Tristan
> 
> 
> 
> [1] https://github.com/infinispan/infinispan/pull/2589
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)





From mmarkus at redhat.com  Tue Jun  3 04:59:11 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Tue, 3 Jun 2014 09:59:11 +0100
Subject: [infinispan-dev] book on Infinispan
In-Reply-To: <CAL7v1EsHgzAL-KUS=LppN+x=wZCtNU1gTkrnD-dNdpKztVBpDg@mail.gmail.com>
References: <D4AD1AC5-6760-4180-8D11-BEFB193EB5F5@redhat.com>
	<7DF2D821-E83E-4209-877B-3F915BD44B60@redhat.com>
	<CAL7v1EsHgzAL-KUS=LppN+x=wZCtNU1gTkrnD-dNdpKztVBpDg@mail.gmail.com>
Message-ID: <CEE93D5C-DBAC-4D4B-A342-D2850CC8C8FF@redhat.com>

Thanks, really looking forward to it!

On Jun 3, 2014, at 2:38, Wagner Santos <wrsconsulting at gmail.com> wrote:

> Hi Mircea, 
> 
> Nice to meet you, 
> 
> It's a honor for me to know you, I have to congratulate you for the awesome work with Infinispan.
> ?
> I sent the draft of the last chapter today, and I think that it would be great if you (or anyone that you may be working with.) could review it!
> 
> If you have any query, just let me know!
> 
> Thanks and Kind Regards,

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From sanne at infinispan.org  Tue Jun  3 05:35:01 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 3 Jun 2014 10:35:01 +0100
Subject: [infinispan-dev] Uber Jars
In-Reply-To: <1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>
References: <538D7EF2.5060103@redhat.com>
	<1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>
Message-ID: <CAFm4XO32xSMyoY12GY7oSscim4T=3RL6vr+OwQWx8CNh9-MMKQ@mail.gmail.com>

On 3 June 2014 09:57, Mircea Markus <mmarkus at redhat.com> wrote:
> On Jun 3, 2014, at 8:53, Tristan Tarrant <ttarrant at redhat.com> wrote:
>
>> Dear all,
>>
>> on Thursday I issued a PR [1] to introduce Uber Jars, i.e. single jars
>> which wrap our multitude of jars and some of the transitive
>> dependencies, but it was (rightly) pointed out that we should have a
>> little discussion here first.

thanks!

>>
>> Firstly, I'm using the maven shade plugin which repackages multiple jars
>> into one with:
>> - automatic transitive resolution
>> - the ability to include/exclude certain jars
>> - move classes if necessary to other packages to avoid conflicts
>> - rewrite the POM with the new dependencies.
>>
>> Here is my global strategy:
>> - define a set of uber-jars (see below)
>> - include all non-optional dependencies in each uber-jar except for the
>> specification Jars (e.g. javax.transaction and javax.persistence)
>> - rename some internal-only dependencies to avoid conflicts
>> - uber jars MUST NOT inherit from infinispan-parent (too much cruft in
>> there) but only from infinispan-bom.

What is the definition of an "internal-only" dependency?
I really like the intention but renaming seems quite dangerous.

For example, JGroups is quite internal but doing so it means the user
could not create a custom Protocol and define it in his configuration
file.
Also I suspect several frameworks rely on reflection to "wire-up" at
boot time (like JGroups does with Protocol class names), so if we're
in the business of relocating classes, this should be followed by
integration tests.

> Just for my own understanding, what is the rationale behind this?
>
>>
>> The Uber Jars
>> - infinispan-embedded-all (infinispan-commons, infinispan-core, jgroups,
>> jboss-marshalling-osgi, jboss-logging, infinispan-cachestore-jdbc,
>> infinispan-cachestore-jpa, infinispan-cachestore-leveldb)

I wouldn't call a package "-all" if it's missing some things. For
example, without Query functionality this "All" is actually a very
limited subset ;-)

WDYT "infinispan-embedded"

>> - infinispan-remote-all (infinispan-commons, infinispan-client-hotrod,
>> commons-pool, jboss-marshalling-osgi, jboss-logging,
>> infinispan-remote-query-client, infinispan-query-dsl,
>> infinispan-protostream, protobuf-java)

Same objection on the "-all" naming.
I'd suggest "infinispan-client".

>> - infinispan-query-all (infinispan-query, infinispan-query-dsl,
>> hibernate-hql-parser, antlr, stringtemplate, hibernate-hql-lucene,
>> hibernate-search-engine, infinispan-lucene-v4,
>> hibernate-search-analyzers, lucene*, solr*, avro, jackson-core,
>> jackson-mapper, paranamer, apache-compress, infinispan-lucene-directory,
>> hibernate-search-infinispan, hibernate-commons-annotations). This
>> package will depend on infinispan-embedded-all and we should only make a
>> Lucene v4 version).

If you make only a Lucene V4 version that's not going to work with
master as it's today, my patches to upgade the Query engine to use
Lucene V4 are not integrated yet. (And if it's meant for a Lucene4
target, then you don't need Solr)

More importantly: is the user meant to use these as Either/OR or as
additive packages? From the dependencies you listed it looks like
additive packages, but I think the name suggests that using
"infinispan-query-all" you have all what's needed.

Very glad to see progress in this area :-)

Cheers,
Sanne

>>
>> Discuss :)
>
> Much simpler this way, so why not making this the default packaging?
>
>>
>>
>> Tristan
>>
>>
>>
>> [1] https://github.com/infinispan/infinispan/pull/2589
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From ttarrant at redhat.com  Tue Jun  3 06:30:40 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Tue, 03 Jun 2014 12:30:40 +0200
Subject: [infinispan-dev] Uber Jars
In-Reply-To: <CAFm4XO32xSMyoY12GY7oSscim4T=3RL6vr+OwQWx8CNh9-MMKQ@mail.gmail.com>
References: <538D7EF2.5060103@redhat.com>	<1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>
	<CAFm4XO32xSMyoY12GY7oSscim4T=3RL6vr+OwQWx8CNh9-MMKQ@mail.gmail.com>
Message-ID: <538DA3D0.2020701@redhat.com>

On 03/06/2014 11:35, Sanne Grinovero wrote:
> On 3 June 2014 09:57, Mircea Markus <mmarkus at redhat.com> wrote:
>> On Jun 3, 2014, at 8:53, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>
>>> Dear all,
>>>
>>> on Thursday I issued a PR [1] to introduce Uber Jars, i.e. single jars
>>> which wrap our multitude of jars and some of the transitive
>>> dependencies, but it was (rightly) pointed out that we should have a
>>> little discussion here first.
> thanks!
>
>>> Firstly, I'm using the maven shade plugin which repackages multiple jars
>>> into one with:
>>> - automatic transitive resolution
>>> - the ability to include/exclude certain jars
>>> - move classes if necessary to other packages to avoid conflicts
>>> - rewrite the POM with the new dependencies.
>>>
>>> Here is my global strategy:
>>> - define a set of uber-jars (see below)
>>> - include all non-optional dependencies in each uber-jar except for the
>>> specification Jars (e.g. javax.transaction and javax.persistence)
>>> - rename some internal-only dependencies to avoid conflicts
>>> - uber jars MUST NOT inherit from infinispan-parent (too much cruft in
>>> there) but only from infinispan-bom.
> What is the definition of an "internal-only" dependency?
> I really like the intention but renaming seems quite dangerous.
>
> For example, JGroups is quite internal but doing so it means the user
> could not create a custom Protocol and define it in his configuration
> file.
> Also I suspect several frameworks rely on reflection to "wire-up" at
> boot time (like JGroups does with Protocol class names), so if we're
> in the business of relocating classes, this should be followed by
> integration tests.
The only dependency which currently "fits" my definitions is 
jboss-logging. It is only used internally and its APIs are not 
"re-exported".
My PR also includes a simple integration test for the "embedded" case. I 
will obviously integrate it with tests with the others once we agree on 
the structure.
>> Just for my own understanding, what is the rationale behind this?
Users who don't use a dependency management system (i.e. Maven, Ivy, 
etc) can write an application just by adding a "single" jar.
>>
>>> The Uber Jars
>>> - infinispan-embedded-all (infinispan-commons, infinispan-core, jgroups,
>>> jboss-marshalling-osgi, jboss-logging, infinispan-cachestore-jdbc,
>>> infinispan-cachestore-jpa, infinispan-cachestore-leveldb)
> I wouldn't call a package "-all" if it's missing some things. For
> example, without Query functionality this "All" is actually a very
> limited subset ;-)
>
> WDYT "infinispan-embedded"
Ok.
>>> - infinispan-remote-all (infinispan-commons, infinispan-client-hotrod,
>>> commons-pool, jboss-marshalling-osgi, jboss-logging,
>>> infinispan-remote-query-client, infinispan-query-dsl,
>>> infinispan-protostream, protobuf-java)
> Same objection on the "-all" naming.
> I'd suggest "infinispan-client".
Ok
>>> - infinispan-query-all (infinispan-query, infinispan-query-dsl,
>>> hibernate-hql-parser, antlr, stringtemplate, hibernate-hql-lucene,
>>> hibernate-search-engine, infinispan-lucene-v4,
>>> hibernate-search-analyzers, lucene*, solr*, avro, jackson-core,
>>> jackson-mapper, paranamer, apache-compress, infinispan-lucene-directory,
>>> hibernate-search-infinispan, hibernate-commons-annotations). This
>>> package will depend on infinispan-embedded-all and we should only make a
>>> Lucene v4 version).
> If you make only a Lucene V4 version that's not going to work with
> master as it's today, my patches to upgade the Query engine to use
> Lucene V4 are not integrated yet. (And if it's meant for a Lucene4
> target, then you don't need Solr)
I added solr since that is what my aging neuron remembered. Shade 
actually pulls in the dependency tree automatically so I don't have to 
think about these things (and the Infinispan Query dependency hierarchy 
is hairier than a sea otter).
> More importantly: is the user meant to use these as Either/OR or as
> additive packages? From the dependencies you listed it looks like
> additive packages, but I think the name suggests that using
> "infinispan-query-all" you have all what's needed.

infinispan-embedded-query would still depend on infinispan-embedded, so 
the user would require both jars. For the maven user, just depending on 
infinispan-embedded-query will do the trick, since infinispan-embedded 
would be a transitive.

> Much simpler this way, so why not making this the default packaging?
>
The "uber" jars are what we should advertise in our quickstarts, docs, 
etc. Obviously the single underlying jars would still be deployed to maven.

Tristan

From ales.justin at gmail.com  Tue Jun  3 08:58:11 2014
From: ales.justin at gmail.com (Ales Justin)
Date: Tue, 3 Jun 2014 14:58:11 +0200
Subject: [infinispan-dev] CD datastore inserts slow
In-Reply-To: <CAFm4XO2gL0pcKyH0F0OTN2VATdjsOPMG+704CNgAXGCO-u9T-Q@mail.gmail.com>
References: <CAFm4XO2gL0pcKyH0F0OTN2VATdjsOPMG+704CNgAXGCO-u9T-Q@mail.gmail.com>
Message-ID: <878E3B1C-6CD2-411C-A4A5-8F379F9A4AFA@gmail.com>

> So, storing some thousand entities in a single transaction takes a
> little more than 1 second, while running each put operation in a
> single transaction takes ~50 seconds.
> 
> For the record, I'm more concerned that storing "a few thousand"
> entities in a single TX takes  a full second: that's not expected, but
> my guess is that in this specific case you're not warming up the JVM
> nor repeating the test. Having batching enabled, I'd expect this to be
> in the order of millions / second.

Uh, this was actually my bad -- with single Tx it also takes long time ...
(@Sanne -- be concerned, be very concerned ... ;-)

14:46:14,669 INFO  [com.google.appengine.tck.benchmark.ObjectifyBenchmarkTest] (default task-1) >>>> N = 6000
14:46:17,418 INFO  [com.google.appengine.tck.benchmark.ObjectifyBenchmarkTest] (default task-1) >>>> Save [6000] time: 2461ms
14:47:53,450 INFO  [com.google.appengine.tck.benchmark.ObjectifyBenchmarkTest] (default task-1) >>>> Full Tx  time: 98576ms
14:50:19,721 INFO  [com.google.appengine.tck.benchmark.ObjectifyBenchmarkTest] (default task-1) >>>> Save [6000] time: 146246ms

The 1st "Save" is fast, as this is just pilling-up the entities within same Tx.
It's the "Full Tx" that is relevant here -- which is ~100sec :-(.

The 2nd "Save" is w/o Tx.

-Ales

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140603/3b76a659/attachment.html 

From dan.berindei at gmail.com  Tue Jun  3 10:40:30 2014
From: dan.berindei at gmail.com (Dan Berindei)
Date: Tue, 3 Jun 2014 17:40:30 +0300
Subject: [infinispan-dev] Uber Jars
In-Reply-To: <538DA3D0.2020701@redhat.com>
References: <538D7EF2.5060103@redhat.com>
	<1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>
	<CAFm4XO32xSMyoY12GY7oSscim4T=3RL6vr+OwQWx8CNh9-MMKQ@mail.gmail.com>
	<538DA3D0.2020701@redhat.com>
Message-ID: <CA+nfvwRYWxiV+MF2t+iPFUtfb_N9zqw=csVogAjDq6_EZUcsTw@mail.gmail.com>

On Tue, Jun 3, 2014 at 1:30 PM, Tristan Tarrant <ttarrant at redhat.com> wrote:

> On 03/06/2014 11:35, Sanne Grinovero wrote:
> > On 3 June 2014 09:57, Mircea Markus <mmarkus at redhat.com> wrote:
> >> On Jun 3, 2014, at 8:53, Tristan Tarrant <ttarrant at redhat.com> wrote:
> >>
> >>> Dear all,
> >>>
> >>> on Thursday I issued a PR [1] to introduce Uber Jars, i.e. single jars
> >>> which wrap our multitude of jars and some of the transitive
> >>> dependencies, but it was (rightly) pointed out that we should have a
> >>> little discussion here first.
> > thanks!
> >
> >>> Firstly, I'm using the maven shade plugin which repackages multiple
> jars
> >>> into one with:
> >>> - automatic transitive resolution
> >>> - the ability to include/exclude certain jars
> >>> - move classes if necessary to other packages to avoid conflicts
> >>> - rewrite the POM with the new dependencies.
> >>>
> >>> Here is my global strategy:
> >>> - define a set of uber-jars (see below)
> >>> - include all non-optional dependencies in each uber-jar except for the
> >>> specification Jars (e.g. javax.transaction and javax.persistence)
> >>> - rename some internal-only dependencies to avoid conflicts
> >>> - uber jars MUST NOT inherit from infinispan-parent (too much cruft in
> >>> there) but only from infinispan-bom.
> > What is the definition of an "internal-only" dependency?
> > I really like the intention but renaming seems quite dangerous.
> >
> > For example, JGroups is quite internal but doing so it means the user
> > could not create a custom Protocol and define it in his configuration
> > file.
> > Also I suspect several frameworks rely on reflection to "wire-up" at
> > boot time (like JGroups does with Protocol class names), so if we're
> > in the business of relocating classes, this should be followed by
> > integration tests.
> The only dependency which currently "fits" my definitions is
> jboss-logging. It is only used internally and its APIs are not
> "re-exported".
> My PR also includes a simple integration test for the "embedded" case. I
> will obviously integrate it with tests with the others once we agree on
> the structure.
> >> Just for my own understanding, what is the rationale behind this?
> Users who don't use a dependency management system (i.e. Maven, Ivy,
> etc) can write an application just by adding a "single" jar.
> >>
> >>> The Uber Jars
> >>> - infinispan-embedded-all (infinispan-commons, infinispan-core,
> jgroups,
> >>> jboss-marshalling-osgi, jboss-logging, infinispan-cachestore-jdbc,
> >>> infinispan-cachestore-jpa, infinispan-cachestore-leveldb)
> > I wouldn't call a package "-all" if it's missing some things. For
> > example, without Query functionality this "All" is actually a very
> > limited subset ;-)
> >
> > WDYT "infinispan-embedded"
> Ok.
> >>> - infinispan-remote-all (infinispan-commons, infinispan-client-hotrod,
> >>> commons-pool, jboss-marshalling-osgi, jboss-logging,
> >>> infinispan-remote-query-client, infinispan-query-dsl,
> >>> infinispan-protostream, protobuf-java)
> > Same objection on the "-all" naming.
> > I'd suggest "infinispan-client".
> Ok
> >>> - infinispan-query-all (infinispan-query, infinispan-query-dsl,
> >>> hibernate-hql-parser, antlr, stringtemplate, hibernate-hql-lucene,
> >>> hibernate-search-engine, infinispan-lucene-v4,
> >>> hibernate-search-analyzers, lucene*, solr*, avro, jackson-core,
> >>> jackson-mapper, paranamer, apache-compress,
> infinispan-lucene-directory,
> >>> hibernate-search-infinispan, hibernate-commons-annotations). This
> >>> package will depend on infinispan-embedded-all and we should only make
> a
> >>> Lucene v4 version).
> > If you make only a Lucene V4 version that's not going to work with
> > master as it's today, my patches to upgade the Query engine to use
> > Lucene V4 are not integrated yet. (And if it's meant for a Lucene4
> > target, then you don't need Solr)
> I added solr since that is what my aging neuron remembered. Shade
> actually pulls in the dependency tree automatically so I don't have to
> think about these things (and the Infinispan Query dependency hierarchy
> is hairier than a sea otter).
> > More importantly: is the user meant to use these as Either/OR or as
> > additive packages? From the dependencies you listed it looks like
> > additive packages, but I think the name suggests that using
> > "infinispan-query-all" you have all what's needed.
>
> infinispan-embedded-query would still depend on infinispan-embedded, so
> the user would require both jars. For the maven user, just depending on
> infinispan-embedded-query will do the trick, since infinispan-embedded
> would be a transitive.
>

Interesting, I thought the uber jars would only be relevant for non-Maven
users.
So we'd also have a POM in the Maven repo that Maven users can reference,
for each uber jar?


>
> > Much simpler this way, so why not making this the default packaging?
> >
> The "uber" jars are what we should advertise in our quickstarts, docs,
> etc. Obviously the single underlying jars would still be deployed to maven.
>
> Tristan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140603/126a10ab/attachment-0001.html 

From ttarrant at redhat.com  Tue Jun  3 10:44:21 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Tue, 03 Jun 2014 16:44:21 +0200
Subject: [infinispan-dev] Uber Jars
In-Reply-To: <CA+nfvwRYWxiV+MF2t+iPFUtfb_N9zqw=csVogAjDq6_EZUcsTw@mail.gmail.com>
References: <538D7EF2.5060103@redhat.com>	<1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>	<CAFm4XO32xSMyoY12GY7oSscim4T=3RL6vr+OwQWx8CNh9-MMKQ@mail.gmail.com>	<538DA3D0.2020701@redhat.com>
	<CA+nfvwRYWxiV+MF2t+iPFUtfb_N9zqw=csVogAjDq6_EZUcsTw@mail.gmail.com>
Message-ID: <538DDF45.8030503@redhat.com>

On 03/06/2014 16:40, Dan Berindei wrote:
>
>
>
> On Tue, Jun 3, 2014 at 1:30 PM, Tristan Tarrant <ttarrant at redhat.com 
> <mailto:ttarrant at redhat.com>> wrote:
>
>     On 03/06/2014 11:35, Sanne Grinovero wrote:
>     > On 3 June 2014 09:57, Mircea Markus <mmarkus at redhat.com
>     <mailto:mmarkus at redhat.com>> wrote:
>     >> On Jun 3, 2014, at 8:53, Tristan Tarrant <ttarrant at redhat.com
>     <mailto:ttarrant at redhat.com>> wrote:
>     >>
>     >>> Dear all,
>     >>>
>     >>> on Thursday I issued a PR [1] to introduce Uber Jars, i.e.
>     single jars
>     >>> which wrap our multitude of jars and some of the transitive
>     >>> dependencies, but it was (rightly) pointed out that we should
>     have a
>     >>> little discussion here first.
>     > thanks!
>     >
>     >>> Firstly, I'm using the maven shade plugin which repackages
>     multiple jars
>     >>> into one with:
>     >>> - automatic transitive resolution
>     >>> - the ability to include/exclude certain jars
>     >>> - move classes if necessary to other packages to avoid conflicts
>     >>> - rewrite the POM with the new dependencies.
>     >>>
>     >>> Here is my global strategy:
>     >>> - define a set of uber-jars (see below)
>     >>> - include all non-optional dependencies in each uber-jar
>     except for the
>     >>> specification Jars (e.g. javax.transaction and javax.persistence)
>     >>> - rename some internal-only dependencies to avoid conflicts
>     >>> - uber jars MUST NOT inherit from infinispan-parent (too much
>     cruft in
>     >>> there) but only from infinispan-bom.
>     > What is the definition of an "internal-only" dependency?
>     > I really like the intention but renaming seems quite dangerous.
>     >
>     > For example, JGroups is quite internal but doing so it means the
>     user
>     > could not create a custom Protocol and define it in his
>     configuration
>     > file.
>     > Also I suspect several frameworks rely on reflection to "wire-up" at
>     > boot time (like JGroups does with Protocol class names), so if we're
>     > in the business of relocating classes, this should be followed by
>     > integration tests.
>     The only dependency which currently "fits" my definitions is
>     jboss-logging. It is only used internally and its APIs are not
>     "re-exported".
>     My PR also includes a simple integration test for the "embedded"
>     case. I
>     will obviously integrate it with tests with the others once we
>     agree on
>     the structure.
>     >> Just for my own understanding, what is the rationale behind this?
>     Users who don't use a dependency management system (i.e. Maven, Ivy,
>     etc) can write an application just by adding a "single" jar.
>     >>
>     >>> The Uber Jars
>     >>> - infinispan-embedded-all (infinispan-commons,
>     infinispan-core, jgroups,
>     >>> jboss-marshalling-osgi, jboss-logging, infinispan-cachestore-jdbc,
>     >>> infinispan-cachestore-jpa, infinispan-cachestore-leveldb)
>     > I wouldn't call a package "-all" if it's missing some things. For
>     > example, without Query functionality this "All" is actually a very
>     > limited subset ;-)
>     >
>     > WDYT "infinispan-embedded"
>     Ok.
>     >>> - infinispan-remote-all (infinispan-commons,
>     infinispan-client-hotrod,
>     >>> commons-pool, jboss-marshalling-osgi, jboss-logging,
>     >>> infinispan-remote-query-client, infinispan-query-dsl,
>     >>> infinispan-protostream, protobuf-java)
>     > Same objection on the "-all" naming.
>     > I'd suggest "infinispan-client".
>     Ok
>     >>> - infinispan-query-all (infinispan-query, infinispan-query-dsl,
>     >>> hibernate-hql-parser, antlr, stringtemplate, hibernate-hql-lucene,
>     >>> hibernate-search-engine, infinispan-lucene-v4,
>     >>> hibernate-search-analyzers, lucene*, solr*, avro, jackson-core,
>     >>> jackson-mapper, paranamer, apache-compress,
>     infinispan-lucene-directory,
>     >>> hibernate-search-infinispan, hibernate-commons-annotations). This
>     >>> package will depend on infinispan-embedded-all and we should
>     only make a
>     >>> Lucene v4 version).
>     > If you make only a Lucene V4 version that's not going to work with
>     > master as it's today, my patches to upgade the Query engine to use
>     > Lucene V4 are not integrated yet. (And if it's meant for a Lucene4
>     > target, then you don't need Solr)
>     I added solr since that is what my aging neuron remembered. Shade
>     actually pulls in the dependency tree automatically so I don't have to
>     think about these things (and the Infinispan Query dependency
>     hierarchy
>     is hairier than a sea otter).
>     > More importantly: is the user meant to use these as Either/OR or as
>     > additive packages? From the dependencies you listed it looks like
>     > additive packages, but I think the name suggests that using
>     > "infinispan-query-all" you have all what's needed.
>
>     infinispan-embedded-query would still depend on
>     infinispan-embedded, so
>     the user would require both jars. For the maven user, just
>     depending on
>     infinispan-embedded-query will do the trick, since infinispan-embedded
>     would be a transitive.
>
>
> Interesting, I thought the uber jars would only be relevant for 
> non-Maven users.
> So we'd also have a POM in the Maven repo that Maven users can 
> reference, for each uber jar?
Why not. A side of effect of Maven Shade is that it generates a 
"dependency-reduced-pom.xml" which is then "installed" or "deployed" to 
the repo in place of the original "pom.xml"

Tristan

From sanne at infinispan.org  Tue Jun  3 11:40:07 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 3 Jun 2014 16:40:07 +0100
Subject: [infinispan-dev] CD datastore inserts slow
In-Reply-To: <878E3B1C-6CD2-411C-A4A5-8F379F9A4AFA@gmail.com>
References: <CAFm4XO2gL0pcKyH0F0OTN2VATdjsOPMG+704CNgAXGCO-u9T-Q@mail.gmail.com>
	<878E3B1C-6CD2-411C-A4A5-8F379F9A4AFA@gmail.com>
Message-ID: <CAFm4XO3jy3i0NUycZYJ4wAQ7XsW-3VaKRiV73-jWVFrDiqfqpg@mail.gmail.com>

Hi Ales,
I don't have CD installed anymore but will try it out. Do you have
some pointers to set up and run this benchmark?

Sanne

On 3 June 2014 13:58, Ales Justin <ales.justin at gmail.com> wrote:
> So, storing some thousand entities in a single transaction takes a
> little more than 1 second, while running each put operation in a
> single transaction takes ~50 seconds.
>
> For the record, I'm more concerned that storing "a few thousand"
> entities in a single TX takes  a full second: that's not expected, but
> my guess is that in this specific case you're not warming up the JVM
> nor repeating the test. Having batching enabled, I'd expect this to be
> in the order of millions / second.
>
>
> Uh, this was actually my bad -- with single Tx it also takes long time ...
> (@Sanne -- be concerned, be very concerned ... ;-)
>
> 14:46:14,669 INFO
> [com.google.appengine.tck.benchmark.ObjectifyBenchmarkTest] (default task-1)
>>>>> N = 6000
> 14:46:17,418 INFO
> [com.google.appengine.tck.benchmark.ObjectifyBenchmarkTest] (default task-1)
>>>>> Save [6000] time: 2461ms
> 14:47:53,450 INFO
> [com.google.appengine.tck.benchmark.ObjectifyBenchmarkTest] (default task-1)
>>>>> Full Tx  time: 98576ms
> 14:50:19,721 INFO
> [com.google.appengine.tck.benchmark.ObjectifyBenchmarkTest] (default task-1)
>>>>> Save [6000] time: 146246ms
>
> The 1st "Save" is fast, as this is just pilling-up the entities within same
> Tx.
> It's the "Full Tx" that is relevant here -- which is ~100sec :-(.
>
> The 2nd "Save" is w/o Tx.
>
> -Ales
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From ales.justin at gmail.com  Tue Jun  3 14:39:09 2014
From: ales.justin at gmail.com (Ales Justin)
Date: Tue, 3 Jun 2014 20:39:09 +0200
Subject: [infinispan-dev] CD datastore inserts slow
In-Reply-To: <CAFm4XO3jy3i0NUycZYJ4wAQ7XsW-3VaKRiV73-jWVFrDiqfqpg@mail.gmail.com>
References: <CAFm4XO2gL0pcKyH0F0OTN2VATdjsOPMG+704CNgAXGCO-u9T-Q@mail.gmail.com>
	<878E3B1C-6CD2-411C-A4A5-8F379F9A4AFA@gmail.com>
	<CAFm4XO3jy3i0NUycZYJ4wAQ7XsW-3VaKRiV73-jWVFrDiqfqpg@mail.gmail.com>
Message-ID: <7A5D7DA2-77F5-4710-9091-7A76C8A8DA40@gmail.com>

Here:

---

I've added this test to GAE TCK:
* https://github.com/GoogleCloudPlatform/appengine-tck/blob/master/core/benchmark/src/test/java/com/google/appengine/tck/benchmark/ObjectifyBenchmarkTest.java

You need CapeDwarf 2.0.0.CR2:
* http://capedwarf.org/downloads/

And clone GAE TCK:
* https://github.com/GoogleCloudPlatform/appengine-tck

Run CapeDwarf:
* https://github.com/capedwarf/capedwarf-blue/blob/master/README.md (see (5))

Then run the ObjectifyBenchmarkTest:
* cd <GAE_TCK_HOME>
* mvn clean install
* cd core/benchmark
* mvn clean install -Pcapedwarf,benchmark

Ping me for any issues.

Thanks!

-Ales

On 03 Jun 2014, at 17:40, Sanne Grinovero <sanne at infinispan.org> wrote:

> Hi Ales,
> I don't have CD installed anymore but will try it out. Do you have
> some pointers to set up and run this benchmark?
> 
> Sanne

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140603/327d1154/attachment.html 

From manik at infinispan.org  Tue Jun  3 19:02:26 2014
From: manik at infinispan.org (Manik Surtani)
Date: Tue, 3 Jun 2014 16:02:26 -0700
Subject: [infinispan-dev] "Unknown Yet"
Message-ID: <CAKUx19rfm4L3AinTBGOdQ=J139k7Fw+HTSq18Tqk23wEmtkNQA@mail.gmail.com>

Really?  Do we not like beer anymore?  :-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140603/e6131eed/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2014-06-03 at 3.58.49 PM.png
Type: image/png
Size: 24631 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140603/e6131eed/attachment-0001.png 

From tsykora at redhat.com  Wed Jun  4 03:08:18 2014
From: tsykora at redhat.com (Tomas Sykora)
Date: Wed, 4 Jun 2014 03:08:18 -0400 (EDT)
Subject: [infinispan-dev] Policy: PR with failing test as a reproducer
In-Reply-To: <1594367917.15986589.1401865186732.JavaMail.zimbra@redhat.com>
Message-ID: <1470145088.15989650.1401865698264.JavaMail.zimbra@redhat.com>

Hello all,
I'd like to know what is our policy in a following matter:

I've wrote a new test which is failing. (local branch)

1) Do we want to integrate also failing test into our test-suite? To see the test failing regularly until the issue is fixed? I suppose no.
2) The other and clearly better "solution" is to push failing test into my own remote branch, create JIRA, let others to try out the issue from my remote branch and wait for fix, then, integrate (already passing) test into upstream.

Is here any possible place for 1) as well? Or we strictly follow 2)?
The only reason which I can see for a policy 1) is that the test would by failing regularly and wouldn't be easily overlooked. 

Thank you for your thoughts!
Tomas




From rvansa at redhat.com  Wed Jun  4 03:31:32 2014
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 04 Jun 2014 09:31:32 +0200
Subject: [infinispan-dev] "Unknown Yet"
In-Reply-To: <CAKUx19rfm4L3AinTBGOdQ=J139k7Fw+HTSq18Tqk23wEmtkNQA@mail.gmail.com>
References: <CAKUx19rfm4L3AinTBGOdQ=J139k7Fw+HTSq18Tqk23wEmtkNQA@mail.gmail.com>
Message-ID: <538ECB54.40102@redhat.com>

Has there been any poll? I can't see/remember any. Or call for name 
suggestions?

Radim

On 06/04/2014 01:02 AM, Manik Surtani wrote:
> Really?  Do we not like beer anymore?  :-)
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss DataGrid QA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140604/7092f33f/attachment.html 

From rory.odonnell at oracle.com  Wed Jun  4 04:02:35 2014
From: rory.odonnell at oracle.com (Rory O'Donnell Oracle, Dublin Ireland)
Date: Wed, 04 Jun 2014 09:02:35 +0100
Subject: [infinispan-dev] Early Access builds for JDK 9 b15,
 JDK 8u20 b16 are available on java.net
Message-ID: <538ED29B.9080300@oracle.com>

Hi Galder,

Early Access builds for JDK 9 b15 <https://jdk9.java.net/download/>, JDK 
8u20 b16 <https://jdk8.java.net/download.html>  are available on java.net.

As we enter the later phases of development for JDK 8u20 , please log 
any show
stoppers as soon as possible.

JDK 7u60 is available for download [0] .

Rgds, Rory

[0] http://www.oracle.com/technetwork/java/javase/downloads/index.html

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140604/b160a43a/attachment.html 

From sanne at infinispan.org  Wed Jun  4 04:08:21 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 4 Jun 2014 09:08:21 +0100
Subject: [infinispan-dev] Policy: PR with failing test as a reproducer
In-Reply-To: <1470145088.15989650.1401865698264.JavaMail.zimbra@redhat.com>
References: <1594367917.15986589.1401865186732.JavaMail.zimbra@redhat.com>
	<1470145088.15989650.1401865698264.JavaMail.zimbra@redhat.com>
Message-ID: <CAFm4XO0uUUwRUWiEXZ0v9OL0zpraWMaYYW=O+hqRiTfhKBQDWg@mail.gmail.com>

On 4 June 2014 08:08, Tomas Sykora <tsykora at redhat.com> wrote:
> Hello all,
> I'd like to know what is our policy in a following matter:
>
> I've wrote a new test which is failing. (local branch)
>
> 1) Do we want to integrate also failing test into our test-suite? To see the test failing regularly until the issue is fixed? I suppose no.
> 2) The other and clearly better "solution" is to push failing test into my own remote branch, create JIRA, let others to try out the issue from my remote branch and wait for fix, then, integrate (already passing) test into upstream.
>
> Is here any possible place for 1) as well? Or we strictly follow 2)?

We strictly follow 2, as otherwise it gets very hard to tell if any
change is introducing regressions.
Any "fix" we make is surely well intentioned, but wathever you do, you
want to make sure the project is evolving in a better direction.

> The only reason which I can see for a policy 1) is that the test would by failing regularly and wouldn't be easily overlooked.

Issues are tracked on the issue tracker -> JIRA.
Traditionally faling tests have been attached as patch files on the
issue, pointing to a branch is much nicer of course..

>
> Thank you for your thoughts!
> Tomas
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From mmarkus at redhat.com  Wed Jun  4 05:32:03 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Wed, 4 Jun 2014 10:32:03 +0100
Subject: [infinispan-dev] release name for Infinispan 7.0
Message-ID: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>

A bit late in the game, but let the beer flow!
I'll start: guinness 


Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)




From gustavonalle at gmail.com  Wed Jun  4 05:35:37 2014
From: gustavonalle at gmail.com (Gustavo Fernandes)
Date: Wed, 4 Jun 2014 10:35:37 +0100
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
Message-ID: <ED5E9468-82D1-48B0-AAD5-D12FC395121D@gmail.com>

My candidate: Blue Moon 

Cheers,
Gustavo


On 4 Jun 2014, at 10:32, Mircea Markus <mmarkus at redhat.com> wrote:

> A bit late in the game, but let the beer flow!
> I'll start: guinness 
> 
> 
> Cheers,
> -- 
> Mircea Markus
> Infinispan lead (www.infinispan.org)
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From borisha.zivkovic at gmail.com  Wed Jun  4 05:48:34 2014
From: borisha.zivkovic at gmail.com (Borisa Zivkovic)
Date: Wed, 4 Jun 2014 10:48:34 +0100
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <ED5E9468-82D1-48B0-AAD5-D12FC395121D@gmail.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
	<ED5E9468-82D1-48B0-AAD5-D12FC395121D@gmail.com>
Message-ID: <CANuqw7vh1gb2yxmx79ZRpapUJLyCQ9Xmq6v_=d9GbaZVkwNhFQ@mail.gmail.com>

Golden dragon (gulden draak)

Great beer btw, a bit strong taste :)
On 4 Jun 2014 10:35, "Gustavo Fernandes" <gustavonalle at gmail.com> wrote:

> My candidate: Blue Moon
>
> Cheers,
> Gustavo
>
>
> On 4 Jun 2014, at 10:32, Mircea Markus <mmarkus at redhat.com> wrote:
>
> > A bit late in the game, but let the beer flow!
> > I'll start: guinness
> >
> >
> > Cheers,
> > --
> > Mircea Markus
> > Infinispan lead (www.infinispan.org)
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140604/04a969db/attachment.html 

From afield at redhat.com  Wed Jun  4 07:17:40 2014
From: afield at redhat.com (Alan Field)
Date: Wed, 4 Jun 2014 07:17:40 -0400 (EDT)
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
Message-ID: <284469780.16082616.1401880660121.JavaMail.zimbra@redhat.com>

Chocolate Rain [1]

*That name also comes with a theme song! [2]

Alan

[1] http://www.beeradvocate.com/beer/profile/16866/53728/
[2] https://www.youtube.com/watch?v=EwTZ2xpQwpA&feature=kp

----- Original Message -----
> From: "Mircea Markus" <mmarkus at redhat.com>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Wednesday, June 4, 2014 11:32:03 AM
> Subject: [infinispan-dev] release name for Infinispan 7.0
> 
> A bit late in the game, but let the beer flow!
> I'll start: guinness
> 
> 
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From ttarrant at redhat.com  Wed Jun  4 07:36:10 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Wed, 04 Jun 2014 13:36:10 +0200
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
Message-ID: <538F04AA.2010803@redhat.com>

On 04/06/2014 11:32, Mircea Markus wrote:
> A bit late in the game, but let the beer flow!
> I'll start: guinness
>
>
> Cheers,
Aventinus, one of my favourites:

http://www.beeradvocate.com/beer/profile/72/224/

Tristan


From dan.berindei at gmail.com  Wed Jun  4 08:04:42 2014
From: dan.berindei at gmail.com (Dan Berindei)
Date: Wed, 4 Jun 2014 15:04:42 +0300
Subject: [infinispan-dev] Policy: PR with failing test as a reproducer
In-Reply-To: <CAFm4XO0uUUwRUWiEXZ0v9OL0zpraWMaYYW=O+hqRiTfhKBQDWg@mail.gmail.com>
References: <1594367917.15986589.1401865186732.JavaMail.zimbra@redhat.com>
	<1470145088.15989650.1401865698264.JavaMail.zimbra@redhat.com>
	<CAFm4XO0uUUwRUWiEXZ0v9OL0zpraWMaYYW=O+hqRiTfhKBQDWg@mail.gmail.com>
Message-ID: <CA+nfvwSZnWiVMVA2NL4+9Au9RTACROWNuHDJdx_miQFidkP_8A@mail.gmail.com>

On Wed, Jun 4, 2014 at 11:08 AM, Sanne Grinovero <sanne at infinispan.org>
wrote:

> On 4 June 2014 08:08, Tomas Sykora <tsykora at redhat.com> wrote:
> > Hello all,
> > I'd like to know what is our policy in a following matter:
> >
> > I've wrote a new test which is failing. (local branch)
> >
> > 1) Do we want to integrate also failing test into our test-suite? To see
> the test failing regularly until the issue is fixed? I suppose no.
> > 2) The other and clearly better "solution" is to push failing test into
> my own remote branch, create JIRA, let others to try out the issue from my
> remote branch and wait for fix, then, integrate (already passing) test into
> upstream.
> >
> > Is here any possible place for 1) as well? Or we strictly follow 2)?
>
> We strictly follow 2, as otherwise it gets very hard to tell if any
> change is introducing regressions.
> Any "fix" we make is surely well intentioned, but wathever you do, you
> want to make sure the project is evolving in a better direction.
>
> > The only reason which I can see for a policy 1) is that the test would
> by failing regularly and wouldn't be easily overlooked.
>
> Issues are tracked on the issue tracker -> JIRA.
> Traditionally faling tests have been attached as patch files on the
> issue, pointing to a branch is much nicer of course..
>

OTOH personal branches will be removed at some point, but attached files
remain in JIRA.
So I'd keep a patch or a full test class attached in JIRA, and only add a
branch reference as a convenience.


>
> >
> > Thank you for your thoughts!
> > Tomas
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140604/2683e9b0/attachment-0001.html 

From emmanuel at hibernate.org  Wed Jun  4 08:16:05 2014
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Wed, 4 Jun 2014 14:16:05 +0200
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <538F04AA.2010803@redhat.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
	<538F04AA.2010803@redhat.com>
Message-ID: <20140604121605.GE738@hibernate.org>

Suntory Hibiki.
I've got to denounce this beer loving blood bath.

On Wed 2014-06-04 13:36, Tristan Tarrant wrote:
> On 04/06/2014 11:32, Mircea Markus wrote:
> > A bit late in the game, but let the beer flow!
> > I'll start: guinness
> >
> >
> > Cheers,
> Aventinus, one of my favourites:
> 
> http://www.beeradvocate.com/beer/profile/72/224/
> 
> Tristan
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From ben.cotton at ALUMNI.RUTGERS.EDU  Wed Jun  4 08:21:19 2014
From: ben.cotton at ALUMNI.RUTGERS.EDU (cotton-ben)
Date: Wed, 4 Jun 2014 05:21:19 -0700 (PDT)
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <538F04AA.2010803@redhat.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
	<538F04AA.2010803@redhat.com>
Message-ID: <0cab01cf7fef$697bb520$3c731f60$@alumni.rutgers.edu>

+1 Aventinus

 

From: Tristan Tarrant-2 [via Infinispan Developer List]
[mailto:ml-node+s980875n4029301h48 at n3.nabble.com] 
Sent: Wednesday, June 4, 2014 7:37 AM
To: cotton-ben
Subject: Re: [infinispan-dev] release name for Infinispan 7.0

 

On 04/06/2014 11:32, Mircea Markus wrote: 
> A bit late in the game, but let the beer flow! 
> I'll start: guinness 
> 
> 
> Cheers, 
Aventinus, one of my favourites: 

http://www.beeradvocate.com/beer/profile/72/224/

Tristan 

_______________________________________________ 
infinispan-dev mailing list 
[hidden email] 
https://lists.jboss.org/mailman/listinfo/infinispan-dev



  _____  

If you reply to this email, your message will be added to the discussion
below:

http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-release
-name-for-Infinispan-7-0-tp4029297p4029301.html 

To start a new topic under Infinispan Developer List, email
ml-node+s980875n2085493h0 at n3.nabble.com 
To unsubscribe from Infinispan Developer List, click here
<http://infinispan-developer-list.980875.n3.nabble.com/template/NamlServlet.
jtp?macro=unsubscribe_by_code&node=2085493&code=YmVuLmNvdHRvbkBBTFVNTkkuUlVU
R0VSUy5FRFV8MjA4NTQ5M3wtMTU2ODA0NTc1OA==> .
 
<http://infinispan-developer-list.980875.n3.nabble.com/template/NamlServlet.
jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.nam
l.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.
view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aem
ail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3A
email.naml> NAML 





--
View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-release-name-for-Infinispan-7-0-tp4029297p4029304.html
Sent from the Infinispan Developer List mailing list archive at Nabble.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140604/f4edd596/attachment.html 

From ygokirmak at gmail.com  Wed Jun  4 08:30:52 2014
From: ygokirmak at gmail.com (yavuz gokirmak)
Date: Wed, 4 Jun 2014 15:30:52 +0300
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <0cab01cf7fef$697bb520$3c731f60$@alumni.rutgers.edu>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
	<538F04AA.2010803@redhat.com>
	<0cab01cf7fef$697bb520$3c731f60$@alumni.rutgers.edu>
Message-ID: <CAGNSiSYH6vpt70khDCzF72qKBVhLrRiBUwFJcCQchA3jhBWPHA@mail.gmail.com>

"owl" for wisdom :)


On 4 June 2014 15:21, cotton-ben <ben.cotton at alumni.rutgers.edu> wrote:

> +1 Aventinus
>
>
>
> *From:* Tristan Tarrant-2 [via Infinispan Developer List] [mailto:[hidden
> email] <http://user/SendEmail.jtp?type=node&node=4029304&i=0>]
> *Sent:* Wednesday, June 4, 2014 7:37 AM
> *To:* cotton-ben
> *Subject:* Re: [infinispan-dev] release name for Infinispan 7.0
>
>
>
> On 04/06/2014 11:32, Mircea Markus wrote:
> > A bit late in the game, but let the beer flow!
> > I'll start: guinness
> >
> >
> > Cheers,
> Aventinus, one of my favourites:
>
> http://www.beeradvocate.com/beer/profile/72/224/
>
> Tristan
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email] <http://user/SendEmail.jtp?type=node&node=4029301&i=0>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> ------------------------------
>
> *If you reply to this email, your message will be added to the discussion
> below:*
>
>
> http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-release-name-for-Infinispan-7-0-tp4029297p4029301.html
>
> To start a new topic under Infinispan Developer List, email [hidden email]
> <http://user/SendEmail.jtp?type=node&node=4029304&i=1>
> To unsubscribe from Infinispan Developer List, click here.
> NAML
> <http://infinispan-developer-list.980875.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
> ------------------------------
> View this message in context: RE: [infinispan-dev] release name for
> Infinispan 7.0
> <http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-release-name-for-Infinispan-7-0-tp4029297p4029304.html>
> Sent from the Infinispan Developer List mailing list archive
> <http://infinispan-developer-list.980875.n3.nabble.com/> at Nabble.com.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140604/9bccf5f7/attachment.html 

From vblagoje at redhat.com  Wed Jun  4 09:39:03 2014
From: vblagoje at redhat.com (Vladimir Blagojevic)
Date: Wed, 04 Jun 2014 09:39:03 -0400
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
Message-ID: <538F2177.40101@redhat.com>

On 2014-06-04, 5:32 AM, Mircea Markus wrote:
> A bit late in the game, but let the beer flow!
> I'll start: guinness
>
>
> Cheers,
Let's name it after some beer outside of Europe, N.A. Maybe something 
from S.A. In Brazil they have a beer "Antarctica".

Vladimir

From mmarkus at redhat.com  Wed Jun  4 13:22:43 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Wed, 4 Jun 2014 18:22:43 +0100
Subject: [infinispan-dev] Policy: PR with failing test as a reproducer
In-Reply-To: <CA+nfvwSZnWiVMVA2NL4+9Au9RTACROWNuHDJdx_miQFidkP_8A@mail.gmail.com>
References: <1594367917.15986589.1401865186732.JavaMail.zimbra@redhat.com>
	<1470145088.15989650.1401865698264.JavaMail.zimbra@redhat.com>
	<CAFm4XO0uUUwRUWiEXZ0v9OL0zpraWMaYYW=O+hqRiTfhKBQDWg@mail.gmail.com>
	<CA+nfvwSZnWiVMVA2NL4+9Au9RTACROWNuHDJdx_miQFidkP_8A@mail.gmail.com>
Message-ID: <A6CB964B-B10F-4B42-BEA8-C05B903D0A0C@redhat.com>


On Jun 4, 2014, at 13:04, Dan Berindei <dan.berindei at gmail.com> wrote:

> On Wed, Jun 4, 2014 at 11:08 AM, Sanne Grinovero <sanne at infinispan.org> wrote:
> On 4 June 2014 08:08, Tomas Sykora <tsykora at redhat.com> wrote:
> > Hello all,
> > I'd like to know what is our policy in a following matter:
> >
> > I've wrote a new test which is failing. (local branch)
> >
> > 1) Do we want to integrate also failing test into our test-suite? To see the test failing regularly until the issue is fixed? I suppose no.
> > 2) The other and clearly better "solution" is to push failing test into my own remote branch, create JIRA, let others to try out the issue from my remote branch and wait for fix, then, integrate (already passing) test into upstream.
> >
> > Is here any possible place for 1) as well? Or we strictly follow 2)?
> 
> We strictly follow 2, as otherwise it gets very hard to tell if any
> change is introducing regressions.
> Any "fix" we make is surely well intentioned, but wathever you do, you
> want to make sure the project is evolving in a better direction.
> 
> > The only reason which I can see for a policy 1) is that the test would by failing regularly and wouldn't be easily overlooked.
> 
> Issues are tracked on the issue tracker -> JIRA.
> Traditionally faling tests have been attached as patch files on the
> issue, pointing to a branch is much nicer of course..
> 
> OTOH personal branches will be removed at some point, but attached files remain in JIRA.
> So I'd keep a patch or a full test class attached in JIRA, and only add a branch reference as a convenience.

+1 for the patch files, more safe that way.

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From sanne at infinispan.org  Wed Jun  4 18:24:11 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 4 Jun 2014 23:24:11 +0100
Subject: [infinispan-dev] Externalizer IDs have reserved ranges
Message-ID: <CAFm4XO3JCtTDBSXdPLBkTKRicWUTZ6hZw0XMhoEOA7qR8VQsFw@mail.gmail.com>

All,
please remember that you can't "just take" any id when creating a new
Externalizer, especially in a module which didn't have any:

http://infinispan.org/docs/7.0.x/user_guide/user_guide.html#_preassigned_externalizer_id_ranges

Cheers,
Sanne

From manik at infinispan.org  Wed Jun  4 19:52:56 2014
From: manik at infinispan.org (Manik Surtani)
Date: Wed, 4 Jun 2014 16:52:56 -0700
Subject: [infinispan-dev] "Unknown Yet"
In-Reply-To: <538ECB54.40102@redhat.com>
References: <CAKUx19rfm4L3AinTBGOdQ=J139k7Fw+HTSq18Tqk23wEmtkNQA@mail.gmail.com>
	<538ECB54.40102@redhat.com>
Message-ID: <CAKUx19qOxVFjAZ-VBzL1DSWNqK1UspwmUs8eE7mDzFoB4znUOQ@mail.gmail.com>

Come on, guys!  :-)

I nominate and vote for Old Rasputin.

http://www.northcoastbrewing.com/beer-rasputin.htm

- Manik




On 4 June 2014 00:31, Radim Vansa <rvansa at redhat.com> wrote:

>  Has there been any poll? I can't see/remember any. Or call for name
> suggestions?
>
> Radim
>
>
> On 06/04/2014 01:02 AM, Manik Surtani wrote:
>
> Really?  Do we not like beer anymore?  :-)
>
>
> _______________________________________________
> infinispan-dev mailing listinfinispan-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> --
> Radim Vansa <rvansa at redhat.com> <rvansa at redhat.com>
> JBoss DataGrid QA
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140604/a15e44f6/attachment-0001.html 

From ales.justin at gmail.com  Thu Jun  5 03:50:44 2014
From: ales.justin at gmail.com (Ales Justin)
Date: Thu, 5 Jun 2014 09:50:44 +0200
Subject: [infinispan-dev] "Unknown Yet"
In-Reply-To: <CAKUx19qOxVFjAZ-VBzL1DSWNqK1UspwmUs8eE7mDzFoB4znUOQ@mail.gmail.com>
References: <CAKUx19rfm4L3AinTBGOdQ=J139k7Fw+HTSq18Tqk23wEmtkNQA@mail.gmail.com>
	<538ECB54.40102@redhat.com>
	<CAKUx19qOxVFjAZ-VBzL1DSWNqK1UspwmUs8eE7mDzFoB4znUOQ@mail.gmail.com>
Message-ID: <CD7329FB-5B76-4A52-8F03-96965B54F887@gmail.com>

Bevog:
* http://www.bevog.at/

-Ales

On 05 Jun 2014, at 01:52, Manik Surtani <manik at infinispan.org> wrote:

> Come on, guys!  :-)
> 
> I nominate and vote for Old Rasputin.
> 
> http://www.northcoastbrewing.com/beer-rasputin.htm
> 
> - Manik
> 
> 
> 
> 
> On 4 June 2014 00:31, Radim Vansa <rvansa at redhat.com> wrote:
> Has there been any poll? I can't see/remember any. Or call for name suggestions?
> 
> Radim
> 
> 
> On 06/04/2014 01:02 AM, Manik Surtani wrote:
>> Really?  Do we not like beer anymore?  :-)
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> -- 
> Radim Vansa <rvansa at redhat.com>
> JBoss DataGrid QA
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140605/2de1ca70/attachment.html 

From sanne at infinispan.org  Thu Jun  5 06:53:52 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 5 Jun 2014 11:53:52 +0100
Subject: [infinispan-dev] Performance validation of Remote & Embedded Query
	functionality
Message-ID: <CAFm4XO3MoJPBDueWG6mBM_pA-h929rDYGtJJ5suCH22Xkyt=cw@mail.gmail.com>

Hi Radim, all,
I'm in a face to face meeting with Adrian and Gustavo, to make plans
for the next steps of Query development.
One thing which is essential is of course having some idea of its
scalability and general performance characteristics, both to identify
the low hanging fruits which might be in the code, to be able to give
some guidance to users on expectations, and also to know which
configuration options are better for each use case: I have some
expectations but these haven't been validated on the new-gen Query
functionality.

I was assuming that we would have to develop it: we need it to be able
to get working on our laptops as a first step to use to identify the
most obvious mistakes, as well as make it possible to validate in the
QA lab on more realistic hardware configurations, when the most
obvious issues will have been resolved.
Martin suggested that you have this task too as one of the next goals,
so let's develop it together?

We couldn't think of a cool example to use as a model, but roughly
this is what we aim to cover, and the data we aim to collect;
suggestions are very welcome:

## Benchmark 1: Remote Queries (over Hot Rod)
Should perform a (weighted) mixture of the following Read and Write operations:
 (R) Ranges (with and without pagination)
 (R) Exact
 (R) All the above, combined (with and without pagination)
 (W) Insert an entry
 (W) Delete an entry
 (W) Update an entry

 Configuration options
- data sizes: let's aim at having a consistent data set of at least 4GB.
- 1 node / 4 nodes / 8 nodes
        - REPL/DIST for the Data storing Cache
- variable ratio of results out of the index (Query retrieves just 5
entries out of a million vs. half a million)
- control ratio of operations; eg. : no writes / mostly writes / just
Range queries
        - for write operations: make sure to trigger some Merge events
- SYNC / ASYNC indexing backend and control IndexWriting tuning
- NRT / Non-NRT backends (Infinispan IndexManager only available as non-NRT)
- FSDirectory / InfinispanDirectory
          -- Infinispan Directory: Stored in REPL / DIST independently
from the Data Cache
                                 : With/Without CacheStore
- Have an option to run "Index-Less" (the tablescan approach)
- Have an option to validate that the queries are returning the expected results

Track:
 - response time: all samples, build distribution of outliers, output
histograms.
 - break down response time of different phases: be able to generate
an histogram of a specific phase only.
 - Count the number of RPCs generated by a specific operation
 - Count the number of CacheStore writes/reads being triggered
 - number of parallel requests it can handle

Data:
It could be random generated but in that case let's have it use a
fixed seed and make sure it generates the same data set at each run,
probably depending just on the target size.
We should also test for distribution of properties of the searched
fields, since we want to be able to predict results to validate them
(or find a different way to validate).
Having a random generator makes preparation faster and allows us to
generate a specific data size, but in alternative we could download
some known public data set; assertions on validity of queries would be
much simpler.

I would like to set specific goals to be reached for each metric, but
let's see the initial results first. We should then also narrow down
the configuration option combinations that we actually want to run in
a set of defined profiles to match common use cases, but let's have
the code ready to run any combination.

## Benchmark 2: Embedded Queries

Same tests as Remote Queries (using the same API, so no full-text).
We might want to develop this one first for simplicity, but results
for the Remote Query functionality are more urgent.

## Benchmark 3: CapeDwarf & Objectify

Help the CapeDwarf team by validating embedded queries; it's useful
for us to have a benchmark running a more complex application. I'm not
too familiar with RadarGun, do you think this could be created as a
RadarGun job, so to have the benchmark run regularly and simplify
setup ?

## Benchmark 4: Hibernate OGM

Another great use case for a more complex test ;-)
The remote support for OGM still needs some coding though, but we
could start looking at the embedded mode.

Priorities?
Some of these are totally independent, but we don't have many hands to
work on it.

I'm going to move this to a wiki, unless I get some "revolutionary" suggestions.

Cheers,
Sanne

From mmarkus at redhat.com  Thu Jun  5 07:54:41 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Thu, 5 Jun 2014 12:54:41 +0100
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <CAGNSiSYH6vpt70khDCzF72qKBVhLrRiBUwFJcCQchA3jhBWPHA@mail.gmail.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
	<538F04AA.2010803@redhat.com>
	<0cab01cf7fef$697bb520$3c731f60$@alumni.rutgers.edu>
	<CAGNSiSYH6vpt70khDCzF72qKBVhLrRiBUwFJcCQchA3jhBWPHA@mail.gmail.com>
Message-ID: <B6B07045-E29E-4D66-A5EA-1B285B9AD282@redhat.com>

Sorry, needs to be a beer name :-)
On Jun 4, 2014, at 13:30, yavuz gokirmak <ygokirmak at gmail.com> wrote:

> "owl" for wisdom :)
> 
> 
> On 4 June 2014 15:21, cotton-ben <ben.cotton at alumni.rutgers.edu> wrote:
> +1 Aventinus
> 
>  
> 
> From: Tristan Tarrant-2 [via Infinispan Developer List] [mailto:[hidden email]] 
> Sent: Wednesday, June 4, 2014 7:37 AM
> To: cotton-ben
> Subject: Re: [infinispan-dev] release name for Infinispan 7.0
> 
>  
> 
> 
> On 04/06/2014 11:32, Mircea Markus wrote: 
> > A bit late in the game, but let the beer flow! 
> > I'll start: guinness 
> > 
> > 
> > Cheers, 
> Aventinus, one of my favourites: 
> 
> http://www.beeradvocate.com/beer/profile/72/224/
> 
> Tristan 
> 
> _______________________________________________ 
> infinispan-dev mailing list 
> [hidden email] 
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> If you reply to this email, your message will be added to the discussion below:
> 
> http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-release-name-for-Infinispan-7-0-tp4029297p4029301.html
> 
> To start a new topic under Infinispan Developer List, email [hidden email] 
> To unsubscribe from Infinispan Developer List, click here.
> NAML
> 
> 
> View this message in context: RE: [infinispan-dev] release name for Infinispan 7.0
> Sent from the Infinispan Developer List mailing list archive at Nabble.com.
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From mmarkus at redhat.com  Thu Jun  5 08:11:33 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Thu, 5 Jun 2014 13:11:33 +0100
Subject: [infinispan-dev] please vote for the Infinispan 7.0 release name!
Message-ID: <330B0FD1-A88F-402C-9D2D-E33A47D78CEC@redhat.com>

https://docs.google.com/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewform


Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From rvansa at redhat.com  Thu Jun  5 09:54:47 2014
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 05 Jun 2014 15:54:47 +0200
Subject: [infinispan-dev] Performance validation of Remote & Embedded
	Query functionality
In-Reply-To: <CAFm4XO3MoJPBDueWG6mBM_pA-h929rDYGtJJ5suCH22Xkyt=cw@mail.gmail.com>
References: <CAFm4XO3MoJPBDueWG6mBM_pA-h929rDYGtJJ5suCH22Xkyt=cw@mail.gmail.com>
Message-ID: <539076A7.6040502@redhat.com>

Hi Sanne,

yes, I do plan to do (Remote|Embedded) Query performance testing, and 
also compare the results with some competing solutions.

I definitely plan to use RadarGun for that as so far the development of 
tests in it proved to be really effective, I have all the infrastructure 
set up to run in distributed environment etc... There is some support 
for embedded query performance testing, but this is really basic and 
I'll probably have to rewrite most of it in order to widen the capabilities.

As RadarGun aims to be generic framework (although Infinispan features 
are naturally covered more thoroughly), I'll have to build some basic 
API common to remote, embedded and other querying frameworks, and 
translate that into Infinispan APIs. Or use just the query string that 
will differ for Infinispan and other solutions. Of course, I could find 
myself tailoring that a bit more for Infinispan :)

 >> break down response time of different phases: be able to generate an 
histogram of a specific phase only.

What do you mean by the phases?

 >> Count the number of RPCs generated by a specific operation

Huh, Infinispan does not report the RPC stats - although I agree that 
would be cool! There's another thread regarding MapReduce where we 
discuss that there should be more collectors of perf data scattered 
around Infinispan. Created [2].

 >> Count the number of CacheStore writes/reads being triggered

Same as above. Created [1]

Regarding data: Specifying size is not enough. Could you draft several 
objects, with some interesting annotations describing the various 
querying settings?
I'd probably start with generated data - the deployment is simpler, and 
we need to generate them somehow anyway. Fixing seeds is simple.

Ad CapeDwarf & Hibernate OGM:
I don't have any experience with CapeDwarf, but think about RadarGun 
this way: it has two main functions:
1) orchestrates the test: starting nodes with some configuration, 
running some test stages, gathering and presenting results and so on
2) abstracts the implementations of any functionality by providing 
unified (maybe simplified) interface [3] - this is important only if we 
want to compare against competitors using the same test

The test itself yet has to be written.
I can't tell whether CapeDwarf guys will be willing to use RadarGun - 
probably not, people are usually reluctant to learn any new tool, 
especially one that has no GUI. I'll focus on the first benchmarks, and 
let's see how that will end up.
PS: For Hibernate OGM, I can easily imagine RadarGun wrapping JPA 
EntityManager and routing to Hibernate OGM or generally any JPA 
implementation, so that could work.

Radim

[1] https://issues.jboss.org/browse/ISPN-4352
[2] https://issues.jboss.org/browse/ISPN-4353
[3] http://xkcd.com/927/

On 06/05/2014 12:53 PM, Sanne Grinovero wrote:
> Hi Radim, all,
> I'm in a face to face meeting with Adrian and Gustavo, to make plans
> for the next steps of Query development.
> One thing which is essential is of course having some idea of its
> scalability and general performance characteristics, both to identify
> the low hanging fruits which might be in the code, to be able to give
> some guidance to users on expectations, and also to know which
> configuration options are better for each use case: I have some
> expectations but these haven't been validated on the new-gen Query
> functionality.
>
> I was assuming that we would have to develop it: we need it to be able
> to get working on our laptops as a first step to use to identify the
> most obvious mistakes, as well as make it possible to validate in the
> QA lab on more realistic hardware configurations, when the most
> obvious issues will have been resolved.
> Martin suggested that you have this task too as one of the next goals,
> so let's develop it together?
>
> We couldn't think of a cool example to use as a model, but roughly
> this is what we aim to cover, and the data we aim to collect;
> suggestions are very welcome:
>
> ## Benchmark 1: Remote Queries (over Hot Rod)
> Should perform a (weighted) mixture of the following Read and Write operations:
>   (R) Ranges (with and without pagination)
>   (R) Exact
>   (R) All the above, combined (with and without pagination)
>   (W) Insert an entry
>   (W) Delete an entry
>   (W) Update an entry
>
>   Configuration options
> - data sizes: let's aim at having a consistent data set of at least 4GB.
> - 1 node / 4 nodes / 8 nodes
>          - REPL/DIST for the Data storing Cache
> - variable ratio of results out of the index (Query retrieves just 5
> entries out of a million vs. half a million)
> - control ratio of operations; eg. : no writes / mostly writes / just
> Range queries
>          - for write operations: make sure to trigger some Merge events
> - SYNC / ASYNC indexing backend and control IndexWriting tuning
> - NRT / Non-NRT backends (Infinispan IndexManager only available as non-NRT)
> - FSDirectory / InfinispanDirectory
>            -- Infinispan Directory: Stored in REPL / DIST independently
> from the Data Cache
>                                   : With/Without CacheStore
> - Have an option to run "Index-Less" (the tablescan approach)
> - Have an option to validate that the queries are returning the expected results
>
> Track:
>   - response time: all samples, build distribution of outliers, output
> histograms.
>   - break down response time of different phases: be able to generate
> an histogram of a specific phase only.
>   - Count the number of RPCs generated by a specific operation
>   - Count the number of CacheStore writes/reads being triggered
>   - number of parallel requests it can handle
>
> Data:
> It could be random generated but in that case let's have it use a
> fixed seed and make sure it generates the same data set at each run,
> probably depending just on the target size.
> We should also test for distribution of properties of the searched
> fields, since we want to be able to predict results to validate them
> (or find a different way to validate).
> Having a random generator makes preparation faster and allows us to
> generate a specific data size, but in alternative we could download
> some known public data set; assertions on validity of queries would be
> much simpler.
>
> I would like to set specific goals to be reached for each metric, but
> let's see the initial results first. We should then also narrow down
> the configuration option combinations that we actually want to run in
> a set of defined profiles to match common use cases, but let's have
> the code ready to run any combination.
>
> ## Benchmark 2: Embedded Queries
>
> Same tests as Remote Queries (using the same API, so no full-text).
> We might want to develop this one first for simplicity, but results
> for the Remote Query functionality are more urgent.
>
> ## Benchmark 3: CapeDwarf & Objectify
>
> Help the CapeDwarf team by validating embedded queries; it's useful
> for us to have a benchmark running a more complex application. I'm not
> too familiar with RadarGun, do you think this could be created as a
> RadarGun job, so to have the benchmark run regularly and simplify
> setup ?
>
> ## Benchmark 4: Hibernate OGM
>
> Another great use case for a more complex test ;-)
> The remote support for OGM still needs some coding though, but we
> could start looking at the embedded mode.
>
> Priorities?
> Some of these are totally independent, but we don't have many hands to
> work on it.
>
> I'm going to move this to a wiki, unless I get some "revolutionary" suggestions.
>
> Cheers,
> Sanne


-- 
Radim Vansa <rvansa at redhat.com>
JBoss DataGrid QA


From manik at infinispan.org  Thu Jun  5 16:45:43 2014
From: manik at infinispan.org (Manik Surtani)
Date: Thu, 5 Jun 2014 13:45:43 -0700
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <20140604121605.GE738@hibernate.org>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
	<538F04AA.2010803@redhat.com> <20140604121605.GE738@hibernate.org>
Message-ID: <CAKUx19pTOz=FJt8JOdBRqUQ+0dk-fU-x5ozZN8V-ujYBA7uN=Q@mail.gmail.com>

Beer names: for project codenames.
Whiskey: strictly for drinking.  Good choice with the Hibiki, BTW.


On 4 June 2014 05:16, Emmanuel Bernard <emmanuel at hibernate.org> wrote:

> Suntory Hibiki.
> I've got to denounce this beer loving blood bath.
>
> On Wed 2014-06-04 13:36, Tristan Tarrant wrote:
> > On 04/06/2014 11:32, Mircea Markus wrote:
> > > A bit late in the game, but let the beer flow!
> > > I'll start: guinness
> > >
> > >
> > > Cheers,
> > Aventinus, one of my favourites:
> >
> > http://www.beeradvocate.com/beer/profile/72/224/
> >
> > Tristan
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140605/e7381f7c/attachment-0001.html 

From hugo.guerrero.olivares at gmail.com  Thu Jun  5 17:35:24 2014
From: hugo.guerrero.olivares at gmail.com (Hugo Guerrero)
Date: Thu, 5 Jun 2014 16:35:24 -0500
Subject: [infinispan-dev] Name infinispan
Message-ID: <5AB496A6-045A-4A0C-B664-DF0780C6EDE9@gmail.com>

Sanctum de Calavera Beer
http://en.m.wikipedia.org/wiki/Beer_in_Mexico
Enviado desde mi iPhone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140605/04eb6290/attachment.html 

From sanne at infinispan.org  Thu Jun  5 19:58:15 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 6 Jun 2014 00:58:15 +0100
Subject: [infinispan-dev] Cutting down on dependencies
Message-ID: <CAFm4XO2aVyrUdvAhUUCutkw_xZHcbnD5wfgaFYqbxtin8y9kOQ@mail.gmail.com>

I'm starting from the most obvious place: looking for unused dependencies.

Some dependencies are declared in the parent pom.xml, but it turns out
nobody actually needed them :-)
I'm trying to remove them all, one by one.. a slow background process
as I want the full testsuite to complete for each removal.

It would be easier if no dependency whatsoever was added to the parent
pom; for example we also have a situation in which we would like to
test:

 # if anything works without any transactionmananager available on classpath
 # that Log4J is really optional (will it blow up at runtime if it's not there?)
 # I need a specific module to _not_ have TestNG on classpath because
of conflicts

None of these are possible as these dependencies (Log4j, Narayana,
TestNG) are mandated globally.

I'm working to gradually remove all global dependencies from the
parent pom: modules which need them should declare them explicitly.
Please support me by never ever adding any global dependency to the parent?

Essentially I think no jar belongs there, but we could make an
exception for things like net.jcip:jcip-annotations.

Cheers,
Sanne

From mmarkus at redhat.com  Fri Jun  6 07:23:42 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Fri, 6 Jun 2014 12:23:42 +0100
Subject: [infinispan-dev] please vote for the Infinispan 7.0 release
	name!
In-Reply-To: <330B0FD1-A88F-402C-9D2D-E33A47D78CEC@redhat.com>
References: <330B0FD1-A88F-402C-9D2D-E33A47D78CEC@redhat.com>
Message-ID: <73DAB30A-67F2-4360-9C80-E2300CF3D675@redhat.com>

Guiness it is!

https://docs.google.com/a/infinispan.org/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewanalytics

On Jun 5, 2014, at 13:11, Mircea Markus <mmarkus at redhat.com> wrote:

> https://docs.google.com/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewform
> 
> 
> Cheers,
> -- 
> Mircea Markus
> Infinispan lead (www.infinispan.org)
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From ttarrant at redhat.com  Fri Jun  6 07:30:00 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 06 Jun 2014 13:30:00 +0200
Subject: [infinispan-dev] please vote for the Infinispan 7.0 release
	name!
In-Reply-To: <73DAB30A-67F2-4360-9C80-E2300CF3D675@redhat.com>
References: <330B0FD1-A88F-402C-9D2D-E33A47D78CEC@redhat.com>
	<73DAB30A-67F2-4360-9C80-E2300CF3D675@redhat.com>
Message-ID: <5391A638.8050006@redhat.com>

You rigged it !!!!!

Tristan

On 06/06/2014 13:23, Mircea Markus wrote:
> Guiness it is!
>
> https://docs.google.com/a/infinispan.org/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewanalytics
>
> On Jun 5, 2014, at 13:11, Mircea Markus <mmarkus at redhat.com> wrote:
>
>> https://docs.google.com/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewform
>>
>>
>> Cheers,
>> -- 
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> Cheers,


From ttarrant at redhat.com  Fri Jun  6 07:35:31 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 06 Jun 2014 13:35:31 +0200
Subject: [infinispan-dev] Cutting down on dependencies
In-Reply-To: <CAFm4XO2aVyrUdvAhUUCutkw_xZHcbnD5wfgaFYqbxtin8y9kOQ@mail.gmail.com>
References: <CAFm4XO2aVyrUdvAhUUCutkw_xZHcbnD5wfgaFYqbxtin8y9kOQ@mail.gmail.com>
Message-ID: <5391A783.4000100@redhat.com>

While we're spring-cleaning our poms...

I would also like one or more -Pquick-* profiles which skip 
compiling/testing some parts of the codebase, e.g. quick-embedded, 
quick-server.

Tristan

On 06/06/2014 01:58, Sanne Grinovero wrote:
> I'm starting from the most obvious place: looking for unused dependencies.
>
> Some dependencies are declared in the parent pom.xml, but it turns out
> nobody actually needed them :-)
> I'm trying to remove them all, one by one.. a slow background process
> as I want the full testsuite to complete for each removal.
>
> It would be easier if no dependency whatsoever was added to the parent
> pom; for example we also have a situation in which we would like to
> test:
>
>   # if anything works without any transactionmananager available on classpath
>   # that Log4J is really optional (will it blow up at runtime if it's not there?)
>   # I need a specific module to _not_ have TestNG on classpath because
> of conflicts
>
> None of these are possible as these dependencies (Log4j, Narayana,
> TestNG) are mandated globally.
>
> I'm working to gradually remove all global dependencies from the
> parent pom: modules which need them should declare them explicitly.
> Please support me by never ever adding any global dependency to the parent?
>
> Essentially I think no jar belongs there, but we could make an
> exception for things like net.jcip:jcip-annotations.
>
> Cheers,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>


From pedro at infinispan.org  Fri Jun  6 08:38:19 2014
From: pedro at infinispan.org (Pedro Ruivo)
Date: Fri, 06 Jun 2014 13:38:19 +0100
Subject: [infinispan-dev] OGM, Hot Rod and Grouping API
In-Reply-To: <CANoTJYOMfM+EAAOuS7VunwGgTRfECTpHR9CWzY6Z8oQNR5_mEQ@mail.gmail.com>
References: <CANoTJYOMfM+EAAOuS7VunwGgTRfECTpHR9CWzY6Z8oQNR5_mEQ@mail.gmail.com>
Message-ID: <5391B63B.8000008@infinispan.org>

Hi Davide,

what kind of transactional semantics are expected? when you invoke 
getGroup(), do you expect the keys to be attached to the transaction (in 
the read-set). same question for removeGroup()

Cheers,
Pedro

On 05/28/2014 11:52 AM, Davide D'Alto wrote:
> Hi all,
> some time ago we talked on the mailing list about the integration
> between Hibernate OGM and Hot Rod.
>
> To achieve this we would need to include the grouping API in the Hot Rod
> protocol and to add a couple of methods in the grouping API:
>
> - to get the keys in a group
> - to remove the keys in a group
>
> Mircea created an experimental stub where the method " <G, KG> Set<KG>
> getGroupKeys(G group) " is added to the Cache interface.
> I've rebased the branch to the latest changes (I might have introduce
> some errors): https://github.com/DavideD/infinispan/compare/ISPN-3981
>
> I should have implemented the methods but I haven't had the time to work
> on these features.
>
> There are also two issues to keep track of this:
>
> https://issues.jboss.org/browse/ISPN-3732
> https://issues.jboss.org/browse/ISPN-3981
>
> As far as I know, the API for Infinispan 7 is going to be freezed soon,
> I was wondering if this changes have been taken into account and,
> if not, is it possible to include them?
>
> Thanks,
> Davide
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

From emmanuel at hibernate.org  Fri Jun  6 08:51:24 2014
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Fri, 6 Jun 2014 14:51:24 +0200
Subject: [infinispan-dev] OGM, Hot Rod and Grouping API
In-Reply-To: <5391B63B.8000008@infinispan.org>
References: <CANoTJYOMfM+EAAOuS7VunwGgTRfECTpHR9CWzY6Z8oQNR5_mEQ@mail.gmail.com>
	<5391B63B.8000008@infinispan.org>
Message-ID: <4186B7E9-C052-40E4-BB66-F3EDA1146308@hibernate.org>

We expect the same semantic as a RDBMS in READ_COMMITTED. getGroup() would be equivalent to a select * from ASSOC_TABLE where owner_id = 2. Phantom reads would be acceptable.

Does that answer your question? Because I?m not sure what you mean by the key being attached to the Tx.

On 06 Jun 2014, at 14:38, Pedro Ruivo <pedro at infinispan.org> wrote:

> Hi Davide,
> 
> what kind of transactional semantics are expected? when you invoke 
> getGroup(), do you expect the keys to be attached to the transaction (in 
> the read-set). same question for removeGroup()
> 
> Cheers,
> Pedro
> 
> On 05/28/2014 11:52 AM, Davide D'Alto wrote:
>> Hi all,
>> some time ago we talked on the mailing list about the integration
>> between Hibernate OGM and Hot Rod.
>> 
>> To achieve this we would need to include the grouping API in the Hot Rod
>> protocol and to add a couple of methods in the grouping API:
>> 
>> - to get the keys in a group
>> - to remove the keys in a group
>> 
>> Mircea created an experimental stub where the method " <G, KG> Set<KG>
>> getGroupKeys(G group) " is added to the Cache interface.
>> I've rebased the branch to the latest changes (I might have introduce
>> some errors): https://github.com/DavideD/infinispan/compare/ISPN-3981
>> 
>> I should have implemented the methods but I haven't had the time to work
>> on these features.
>> 
>> There are also two issues to keep track of this:
>> 
>> https://issues.jboss.org/browse/ISPN-3732
>> https://issues.jboss.org/browse/ISPN-3981
>> 
>> As far as I know, the API for Infinispan 7 is going to be freezed soon,
>> I was wondering if this changes have been taken into account and,
>> if not, is it possible to include them?
>> 
>> Thanks,
>> Davide
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From pedro at infinispan.org  Fri Jun  6 09:01:12 2014
From: pedro at infinispan.org (Pedro Ruivo)
Date: Fri, 06 Jun 2014 14:01:12 +0100
Subject: [infinispan-dev] OGM, Hot Rod and Grouping API
In-Reply-To: <4186B7E9-C052-40E4-BB66-F3EDA1146308@hibernate.org>
References: <CANoTJYOMfM+EAAOuS7VunwGgTRfECTpHR9CWzY6Z8oQNR5_mEQ@mail.gmail.com>	<5391B63B.8000008@infinispan.org>
	<4186B7E9-C052-40E4-BB66-F3EDA1146308@hibernate.org>
Message-ID: <5391BB98.4000902@infinispan.org>



On 06/06/2014 01:51 PM, Emmanuel Bernard wrote:
> We expect the same semantic as a RDBMS in READ_COMMITTED. getGroup() would be equivalent to a select * from ASSOC_TABLE where owner_id = 2. Phantom reads would be acceptable.
>
> Does that answer your question? Because I?m not sure what you mean by the key being attached to the Tx.
>
yes, thanks

> On 06 Jun 2014, at 14:38, Pedro Ruivo <pedro at infinispan.org> wrote:
>
>> Hi Davide,
>>
>> what kind of transactional semantics are expected? when you invoke
>> getGroup(), do you expect the keys to be attached to the transaction (in
>> the read-set). same question for removeGroup()
>>
>> Cheers,
>> Pedro
>>
>> On 05/28/2014 11:52 AM, Davide D'Alto wrote:
>>> Hi all,
>>> some time ago we talked on the mailing list about the integration
>>> between Hibernate OGM and Hot Rod.
>>>
>>> To achieve this we would need to include the grouping API in the Hot Rod
>>> protocol and to add a couple of methods in the grouping API:
>>>
>>> - to get the keys in a group
>>> - to remove the keys in a group
>>>
>>> Mircea created an experimental stub where the method " <G, KG> Set<KG>
>>> getGroupKeys(G group) " is added to the Cache interface.
>>> I've rebased the branch to the latest changes (I might have introduce
>>> some errors): https://github.com/DavideD/infinispan/compare/ISPN-3981
>>>
>>> I should have implemented the methods but I haven't had the time to work
>>> on these features.
>>>
>>> There are also two issues to keep track of this:
>>>
>>> https://issues.jboss.org/browse/ISPN-3732
>>> https://issues.jboss.org/browse/ISPN-3981
>>>
>>> As far as I know, the API for Infinispan 7 is going to be freezed soon,
>>> I was wondering if this changes have been taken into account and,
>>> if not, is it possible to include them?
>>>
>>> Thanks,
>>> Davide
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

From mudokonman at gmail.com  Fri Jun  6 09:04:17 2014
From: mudokonman at gmail.com (William Burns)
Date: Fri, 6 Jun 2014 09:04:17 -0400
Subject: [infinispan-dev] OGM, Hot Rod and Grouping API
In-Reply-To: <4186B7E9-C052-40E4-BB66-F3EDA1146308@hibernate.org>
References: <CANoTJYOMfM+EAAOuS7VunwGgTRfECTpHR9CWzY6Z8oQNR5_mEQ@mail.gmail.com>
	<5391B63B.8000008@infinispan.org>
	<4186B7E9-C052-40E4-BB66-F3EDA1146308@hibernate.org>
Message-ID: <CA+YCuUMqVL2VV73Z7zEbkx4-qw6CLWxVF5x8ES46S2qJYxUL1A@mail.gmail.com>

On Fri, Jun 6, 2014 at 8:51 AM, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
> We expect the same semantic as a RDBMS in READ_COMMITTED. getGroup() would be equivalent to a select * from ASSOC_TABLE where owner_id = 2. Phantom reads would be acceptable.

Yeah we can't currently stop any phantom reads.

>
> Does that answer your question? Because I?m not sure what you mean by the key being attached to the Tx.

The difference is say you have a tx1 on thread 1 then on the same
thread you ask for the group, if it is in the transaction controls
whether or not that pending update is seen.

Example:

   tx1 starts
   // Note key1 is in group 1
   cache.put(key1, someValue);

   Set<K> keys = cache.getGroup("group1");
   // The keys would contain key1 if taking part of transaction.

>
> On 06 Jun 2014, at 14:38, Pedro Ruivo <pedro at infinispan.org> wrote:
>
>> Hi Davide,
>>
>> what kind of transactional semantics are expected? when you invoke
>> getGroup(), do you expect the keys to be attached to the transaction (in
>> the read-set). same question for removeGroup()

But the reason I brought up transactions is I was concerned about
removeGroup being used in a non tx way since it would be very prone to
lock timeouts since you could be calling it from the same thread that
has a transaction that holds a lock on one of those keys.

>>
>> Cheers,
>> Pedro
>>
>> On 05/28/2014 11:52 AM, Davide D'Alto wrote:
>>> Hi all,
>>> some time ago we talked on the mailing list about the integration
>>> between Hibernate OGM and Hot Rod.
>>>
>>> To achieve this we would need to include the grouping API in the Hot Rod
>>> protocol and to add a couple of methods in the grouping API:
>>>
>>> - to get the keys in a group
>>> - to remove the keys in a group
>>>
>>> Mircea created an experimental stub where the method " <G, KG> Set<KG>
>>> getGroupKeys(G group) " is added to the Cache interface.
>>> I've rebased the branch to the latest changes (I might have introduce
>>> some errors): https://github.com/DavideD/infinispan/compare/ISPN-3981
>>>
>>> I should have implemented the methods but I haven't had the time to work
>>> on these features.
>>>
>>> There are also two issues to keep track of this:
>>>
>>> https://issues.jboss.org/browse/ISPN-3732
>>> https://issues.jboss.org/browse/ISPN-3981
>>>
>>> As far as I know, the API for Infinispan 7 is going to be freezed soon,
>>> I was wondering if this changes have been taken into account and,
>>> if not, is it possible to include them?
>>>
>>> Thanks,
>>> Davide
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From emmanuel at hibernate.org  Fri Jun  6 09:29:20 2014
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Fri, 6 Jun 2014 15:29:20 +0200
Subject: [infinispan-dev] OGM, Hot Rod and Grouping API
In-Reply-To: <CA+YCuUMqVL2VV73Z7zEbkx4-qw6CLWxVF5x8ES46S2qJYxUL1A@mail.gmail.com>
References: <CANoTJYOMfM+EAAOuS7VunwGgTRfECTpHR9CWzY6Z8oQNR5_mEQ@mail.gmail.com>
	<5391B63B.8000008@infinispan.org>
	<4186B7E9-C052-40E4-BB66-F3EDA1146308@hibernate.org>
	<CA+YCuUMqVL2VV73Z7zEbkx4-qw6CLWxVF5x8ES46S2qJYxUL1A@mail.gmail.com>
Message-ID: <E736C27C-7E7E-43D7-B814-F9FD8630FF5E@hibernate.org>


On 06 Jun 2014, at 15:04, William Burns <mudokonman at gmail.com> wrote:

>> Does that answer your question? Because I?m not sure what you mean by the key being attached to the Tx.
> 
> The difference is say you have a tx1 on thread 1 then on the same
> thread you ask for the group, if it is in the transaction controls
> whether or not that pending update is seen.
> 
> Example:
> 
>   tx1 starts
>   // Note key1 is in group 1
>   cache.put(key1, someValue);
> 
>   Set<K> keys = cache.getGroup("group1");
>   // The keys would contain key1 if taking part of transaction.

Yes that would be necessary.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140606/9129a9f7/attachment.html 

From sanne at infinispan.org  Fri Jun  6 09:31:09 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 6 Jun 2014 14:31:09 +0100
Subject: [infinispan-dev] Test failures
Message-ID: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>

I'm having several failures, these are blocking our progress on Query.

Should I disable them all?

Sample output of three different runs:

Failed tests:
  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
locals st...
  StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
? Runtime
Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0

Failed tests:
  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
locals st...
  StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
? Runtime
  L1StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPut:84->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
? Runtime
Tests run: 5430, Failures: 3, Errors: 0, Skipped: 0

Failed tests:
  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
locals st...
  CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered:611->testIterationBeganAndSegmentNotComplete:510
expected [11] but found [6]
Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0


From emmanuel at hibernate.org  Fri Jun  6 09:33:46 2014
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Fri, 6 Jun 2014 15:33:46 +0200
Subject: [infinispan-dev] OGM, Hot Rod and Grouping API
In-Reply-To: <E736C27C-7E7E-43D7-B814-F9FD8630FF5E@hibernate.org>
References: <CANoTJYOMfM+EAAOuS7VunwGgTRfECTpHR9CWzY6Z8oQNR5_mEQ@mail.gmail.com>
	<5391B63B.8000008@infinispan.org>
	<4186B7E9-C052-40E4-BB66-F3EDA1146308@hibernate.org>
	<CA+YCuUMqVL2VV73Z7zEbkx4-qw6CLWxVF5x8ES46S2qJYxUL1A@mail.gmail.com>
	<E736C27C-7E7E-43D7-B814-F9FD8630FF5E@hibernate.org>
Message-ID: <DF7B48F3-191B-47F7-AFB7-F24459FDD083@hibernate.org>


On 06 Jun 2014, at 15:29, Emmanuel Bernard <emmanuel at hibernate.org> wrote:

> 
> On 06 Jun 2014, at 15:04, William Burns <mudokonman at gmail.com> wrote:
> 
>>> Does that answer your question? Because I?m not sure what you mean by the key being attached to the Tx.
>> 
>> The difference is say you have a tx1 on thread 1 then on the same
>> thread you ask for the group, if it is in the transaction controls
>> whether or not that pending update is seen.
>> 
>> Example:
>> 
>>   tx1 starts
>>   // Note key1 is in group 1
>>   cache.put(key1, someValue);
>> 
>>   Set<K> keys = cache.getGroup("group1");
>>   // The keys would contain key1 if taking part of transaction.
> 
> Yes that would be necessary.

That is provided by repeatable read BTW I think.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140606/b2015dde/attachment.html 

From mudokonman at gmail.com  Fri Jun  6 09:50:46 2014
From: mudokonman at gmail.com (William Burns)
Date: Fri, 6 Jun 2014 09:50:46 -0400
Subject: [infinispan-dev] OGM, Hot Rod and Grouping API
In-Reply-To: <DF7B48F3-191B-47F7-AFB7-F24459FDD083@hibernate.org>
References: <CANoTJYOMfM+EAAOuS7VunwGgTRfECTpHR9CWzY6Z8oQNR5_mEQ@mail.gmail.com>
	<5391B63B.8000008@infinispan.org>
	<4186B7E9-C052-40E4-BB66-F3EDA1146308@hibernate.org>
	<CA+YCuUMqVL2VV73Z7zEbkx4-qw6CLWxVF5x8ES46S2qJYxUL1A@mail.gmail.com>
	<E736C27C-7E7E-43D7-B814-F9FD8630FF5E@hibernate.org>
	<DF7B48F3-191B-47F7-AFB7-F24459FDD083@hibernate.org>
Message-ID: <CA+YCuUPE8ueK6Rp_bVUGbdpFjsiZ15c=w4KiwtSeOVN0LsZibw@mail.gmail.com>

On Fri, Jun 6, 2014 at 9:33 AM, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>
> On 06 Jun 2014, at 15:29, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>
>
> On 06 Jun 2014, at 15:04, William Burns <mudokonman at gmail.com> wrote:
>
> Does that answer your question? Because I?m not sure what you mean by the
> key being attached to the Tx.
>
>
> The difference is say you have a tx1 on thread 1 then on the same
> thread you ask for the group, if it is in the transaction controls
> whether or not that pending update is seen.
>
> Example:
>
>   tx1 starts
>   // Note key1 is in group 1
>   cache.put(key1, someValue);
>
>   Set<K> keys = cache.getGroup("group1");
>   // The keys would contain key1 if taking part of transaction.
>
>
> Yes that would be necessary.
>
>
> That is provided by repeatable read BTW I think.

It is provided for both REPEATABLE_READ and READ_COMMITTED.

I think there is a disconnect with what we mean when we say part of
the transaction.  What we mean when we say part of the transaction is
we literally would suspend any current transaction on that thread,
start a new one if needed and then do the operation, commit/rollback
the new transaction and resume the original one.  In this case any
pending updates in the original transaction are not viewable to the
second.


>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From jeff.ramsdale at gmail.com  Fri Jun  6 12:21:47 2014
From: jeff.ramsdale at gmail.com (Jeff Ramsdale)
Date: Fri, 6 Jun 2014 09:21:47 -0700
Subject: [infinispan-dev] release name for Infinispan 7.0
In-Reply-To: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
References: <3647469A-929F-4F83-B37B-89F5FD4A0C0B@redhat.com>
Message-ID: <CACf_hY03Hs3LVVXpLa5z_YadYDtY79GWpkK0=WnQw6LrEb9SHA@mail.gmail.com>

A few Seattle beers:
* Kilt Lifter
* Dragonstooth
* Naughty Nellie
* El Jefe

I'll place my vote for Dragonstooth...

-j


On Wed, Jun 4, 2014 at 2:32 AM, Mircea Markus <mmarkus at redhat.com> wrote:

> A bit late in the game, but let the beer flow!
> I'll start: guinness
>
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140606/fb696b81/attachment.html 

From dan.berindei at gmail.com  Mon Jun  9 05:19:52 2014
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 9 Jun 2014 12:19:52 +0300
Subject: [infinispan-dev] Test failures
In-Reply-To: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>
References: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>
Message-ID: <CA+nfvwRWfZTkxcAeGs6VDT7=3TCSKUy5nfAGX9ePGRoHypwRGA@mail.gmail.com>

On Fri, Jun 6, 2014 at 4:31 PM, Sanne Grinovero <sanne at infinispan.org>
wrote:

> I'm having several failures, these are blocking our progress on Query.
>
> Should I disable them all?
>

You could disable them, but I'm not quite sure how that would help the
query module... surely you don't need to run the core tests every time you
modify something in query?


>
> Sample output of three different runs:
>
> Failed tests:
>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> locals st...
>

I couldn't reproduce this failure on my machine, but I modified
ThreadLocalLeakTest to ignore that particular thread-local:
https://github.com/infinispan/infinispan/pull/2614
You're the only one that reported seeing it, so please test the PR on your
machine and integrate it.



>
> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> ? Runtime
> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>

I've created https://issues.jboss.org/browse/ISPN-4368 for Will to look
into, I'm not sure we need the "placeholder" key that is causing the random
failures.


>
> Failed tests:
>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> locals st...
>
> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> ? Runtime
>
> L1StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPut:84->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> ? Runtime
>

The L1StateTransferOverwriteTest failure seems to have the same cause as
StateTransferOverwriteTest failure.


> Tests run: 5430, Failures: 3, Errors: 0, Skipped: 0
>
> Failed tests:
>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> locals st...
>
> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered:611->testIterationBeganAndSegmentNotComplete:510
> expected [11] but found [6]
>

I've seen this a couple times on my machine as well, I've created
https://issues.jboss.org/browse/ISPN-4370


> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0



I'd let Will investigate ISPN-4368 and ISPN-4370 for a bit before disabling
those tests - he may be able to issue a PR for them quite quickly.

Still, just because I don't like disabling tests it doesn't mean I don't
appreciate your stability reports - keep 'em coming!


Cheers
Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140609/798db890/attachment-0001.html 

From sanne at infinispan.org  Mon Jun  9 05:43:48 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Mon, 9 Jun 2014 10:43:48 +0100
Subject: [infinispan-dev] Test failures
In-Reply-To: <CA+nfvwRWfZTkxcAeGs6VDT7=3TCSKUy5nfAGX9ePGRoHypwRGA@mail.gmail.com>
References: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>
	<CA+nfvwRWfZTkxcAeGs6VDT7=3TCSKUy5nfAGX9ePGRoHypwRGA@mail.gmail.com>
Message-ID: <CAFm4XO265nEkRB=nV8RLcqS5WJHcrBAH+SKmAwGwOQe_Y88AWA@mail.gmail.com>

Hi Dan!

The reason is that I'm making substantial API changes in the Query
module, and I've lost count on how many other modules and integration
tests depend on it: I need to run all the testsuite to evaluate where
I'm heading.. I don't need it just as last touches but continually, to
be able to make good choices while work is in progress.

Not only I'm changing API but also substantial changes in the
dependency tree.. without a working testsuite I can't make progress.
I'm working around it by deleting core in a temporary commit... :-/
(and even so the suite takes more than an hour ??!)

I'll test your PRs ASAP, thanks a lot.

Cheers,
Sanne


On 9 June 2014 10:19, Dan Berindei <dan.berindei at gmail.com> wrote:
>
>
>
> On Fri, Jun 6, 2014 at 4:31 PM, Sanne Grinovero <sanne at infinispan.org>
> wrote:
>>
>> I'm having several failures, these are blocking our progress on Query.
>>
>> Should I disable them all?
>
>
> You could disable them, but I'm not quite sure how that would help the query
> module... surely you don't need to run the core tests every time you modify
> something in query?
>
>>
>>
>> Sample output of three different runs:
>>
>> Failed tests:
>>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>> locals st...
>
>
> I couldn't reproduce this failure on my machine, but I modified
> ThreadLocalLeakTest to ignore that particular thread-local:
> https://github.com/infinispan/infinispan/pull/2614
> You're the only one that reported seeing it, so please test the PR on your
> machine and integrate it.
>
>
>>
>>
>> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> ? Runtime
>> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>
>
> I've created https://issues.jboss.org/browse/ISPN-4368 for Will to look
> into, I'm not sure we need the "placeholder" key that is causing the random
> failures.
>
>>
>>
>> Failed tests:
>>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>> locals st...
>>
>> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> ? Runtime
>>
>> L1StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPut:84->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> ? Runtime
>
>
> The L1StateTransferOverwriteTest failure seems to have the same cause as
> StateTransferOverwriteTest failure.
>
>>
>> Tests run: 5430, Failures: 3, Errors: 0, Skipped: 0
>>
>> Failed tests:
>>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>> locals st...
>>
>> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered:611->testIterationBeganAndSegmentNotComplete:510
>> expected [11] but found [6]
>
>
> I've seen this a couple times on my machine as well, I've created
> https://issues.jboss.org/browse/ISPN-4370
>
>>
>> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>
>
>
> I'd let Will investigate ISPN-4368 and ISPN-4370 for a bit before disabling
> those tests - he may be able to issue a PR for them quite quickly.
>
> Still, just because I don't like disabling tests it doesn't mean I don't
> appreciate your stability reports - keep 'em coming!
>
>
> Cheers
> Dan
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From rory.odonnell at oracle.com  Mon Jun  9 08:24:40 2014
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Mon, 09 Jun 2014 05:24:40 -0700
Subject: [infinispan-dev] Early Access builds for JDK 9 b16,
 JDK 8u20 b17 are available on java.net
Message-ID: <5395A788.6040708@oracle.com>

Hi Galder,

Early Access builds for JDK 9 b16 <https://jdk9.java.net/download/>  and 
JDK 8u20 b17 <https://jdk8.java.net/download.html>  are available on 
java.net.

As we enter the later phases of development for JDK 8u20 , please log 
any show
stoppers as soon as possible.

Rgds, Rory

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140609/b30e7ba1/attachment.html 

From dan.berindei at gmail.com  Mon Jun  9 09:42:00 2014
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 9 Jun 2014 16:42:00 +0300
Subject: [infinispan-dev] Test failures
In-Reply-To: <CAFm4XO265nEkRB=nV8RLcqS5WJHcrBAH+SKmAwGwOQe_Y88AWA@mail.gmail.com>
References: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>
	<CA+nfvwRWfZTkxcAeGs6VDT7=3TCSKUy5nfAGX9ePGRoHypwRGA@mail.gmail.com>
	<CAFm4XO265nEkRB=nV8RLcqS5WJHcrBAH+SKmAwGwOQe_Y88AWA@mail.gmail.com>
Message-ID: <CA+nfvwQKdzpWhukFMdTw7bfsB0DkZg-AGPiQQfQMAjXDRJbRBQ@mail.gmail.com>

On Mon, Jun 9, 2014 at 12:43 PM, Sanne Grinovero <sanne at infinispan.org>
wrote:

> Hi Dan!
>
> The reason is that I'm making substantial API changes in the Query
> module, and I've lost count on how many other modules and integration
> tests depend on it: I need to run all the testsuite to evaluate where
> I'm heading.. I don't need it just as last touches but continually, to
> be able to make good choices while work is in progress.
>

Wouldn't "mvn install -DskipTests" be enough for testing dependencies?
You could then use "mvn surefire:test -pl query,lucene-directory" to run
just the tests you're interested in.
I have a script that does just that - even for core, I don't want to think
about whether I changed something in commons or not, but I almost never
want to run the commons tests :)


>
> Not only I'm changing API but also substantial changes in the
> dependency tree.. without a working testsuite I can't make progress.
> I'm working around it by deleting core in a temporary commit... :-/
> (and even so the suite takes more than an hour ??!)
>
>
1 hour just for the core, or for everything? I used to get about 1h for
everything, if just the core takes that long to fail it's definitely a
problem. There was also a bit of a slowdown after the JGroups 3.5.0.Beta7
upgrade (ISPN-4355), but that should be fixed now.

I was also thinking of moving the failsafe plugin to the "extras" profile,
so that we can avoid running the server integration tests in dev builds.
But disabling the extras profile also disables the bundling for OSGi, so
perhaps a separate profile would be better.

Dan



> I'll test your PRs ASAP, thanks a lot.
>
> Cheers,
> Sanne
>
>
> On 9 June 2014 10:19, Dan Berindei <dan.berindei at gmail.com> wrote:
> >
> >
> >
> > On Fri, Jun 6, 2014 at 4:31 PM, Sanne Grinovero <sanne at infinispan.org>
> > wrote:
> >>
> >> I'm having several failures, these are blocking our progress on Query.
> >>
> >> Should I disable them all?
> >
> >
> > You could disable them, but I'm not quite sure how that would help the
> query
> > module... surely you don't need to run the core tests every time you
> modify
> > something in query?
> >
> >>
> >>
> >> Sample output of three different runs:
> >>
> >> Failed tests:
> >>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> >> locals st...
> >
> >
> > I couldn't reproduce this failure on my machine, but I modified
> > ThreadLocalLeakTest to ignore that particular thread-local:
> > https://github.com/infinispan/infinispan/pull/2614
> > You're the only one that reported seeing it, so please test the PR on
> your
> > machine and integrate it.
> >
> >
> >>
> >>
> >>
> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> >> ? Runtime
> >> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
> >
> >
> > I've created https://issues.jboss.org/browse/ISPN-4368 for Will to look
> > into, I'm not sure we need the "placeholder" key that is causing the
> random
> > failures.
> >
> >>
> >>
> >> Failed tests:
> >>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> >> locals st...
> >>
> >>
> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> >> ? Runtime
> >>
> >>
> L1StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPut:84->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> >> ? Runtime
> >
> >
> > The L1StateTransferOverwriteTest failure seems to have the same cause as
> > StateTransferOverwriteTest failure.
> >
> >>
> >> Tests run: 5430, Failures: 3, Errors: 0, Skipped: 0
> >>
> >> Failed tests:
> >>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> >> locals st...
> >>
> >>
> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered:611->testIterationBeganAndSegmentNotComplete:510
> >> expected [11] but found [6]
> >
> >
> > I've seen this a couple times on my machine as well, I've created
> > https://issues.jboss.org/browse/ISPN-4370
> >
> >>
> >> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
> >
> >
> >
> > I'd let Will investigate ISPN-4368 and ISPN-4370 for a bit before
> disabling
> > those tests - he may be able to issue a PR for them quite quickly.
> >
> > Still, just because I don't like disabling tests it doesn't mean I don't
> > appreciate your stability reports - keep 'em coming!
> >
> >
> > Cheers
> > Dan
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140609/88cbf317/attachment.html 

From mmarkus at redhat.com  Mon Jun  9 10:02:47 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Mon, 9 Jun 2014 15:02:47 +0100
Subject: [infinispan-dev] weekly meeting moved to Tue at 14:00 GMT (same
	time)
Message-ID: <FCE3C97D-3351-4EA1-AD97-F9E2087F2F38@redhat.com>

Because of many people being PTO and bank holidays.

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)





From manik at infinispan.org  Mon Jun  9 11:33:09 2014
From: manik at infinispan.org (Manik Surtani)
Date: Mon, 9 Jun 2014 16:33:09 +0100
Subject: [infinispan-dev] please vote for the Infinispan 7.0 release
	name!
In-Reply-To: <5391A638.8050006@redhat.com>
References: <330B0FD1-A88F-402C-9D2D-E33A47D78CEC@redhat.com>
	<73DAB30A-67F2-4360-9C80-E2300CF3D675@redhat.com>
	<5391A638.8050006@redhat.com>
Message-ID: <CAKUx19oSxQD1eMKsQYYwCS+4wLNTTke-Q_c58GZADVxtvyOFwA@mail.gmail.com>

Foul play!  :-)


On 6 June 2014 12:30, Tristan Tarrant <ttarrant at redhat.com> wrote:

> You rigged it !!!!!
>
> Tristan
>
> On 06/06/2014 13:23, Mircea Markus wrote:
> > Guiness it is!
> >
> >
> https://docs.google.com/a/infinispan.org/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewanalytics
> >
> > On Jun 5, 2014, at 13:11, Mircea Markus <mmarkus at redhat.com> wrote:
> >
> >>
> https://docs.google.com/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewform
> >>
> >>
> >> Cheers,
> >> --
> >> Mircea Markus
> >> Infinispan lead (www.infinispan.org)
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > Cheers,
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140609/2f4b14da/attachment-0001.html 

From sanne at infinispan.org  Mon Jun  9 17:06:05 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Mon, 9 Jun 2014 22:06:05 +0100
Subject: [infinispan-dev] Test failures
In-Reply-To: <CA+nfvwQKdzpWhukFMdTw7bfsB0DkZg-AGPiQQfQMAjXDRJbRBQ@mail.gmail.com>
References: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>
	<CA+nfvwRWfZTkxcAeGs6VDT7=3TCSKUy5nfAGX9ePGRoHypwRGA@mail.gmail.com>
	<CAFm4XO265nEkRB=nV8RLcqS5WJHcrBAH+SKmAwGwOQe_Y88AWA@mail.gmail.com>
	<CA+nfvwQKdzpWhukFMdTw7bfsB0DkZg-AGPiQQfQMAjXDRJbRBQ@mail.gmail.com>
Message-ID: <CAFm4XO0fdee6r4f+QT_M5rr-dD6g=j2rxMQGLCOuxyALsjRZEw@mail.gmail.com>

On 9 June 2014 14:42, Dan Berindei <dan.berindei at gmail.com> wrote:
>
>
> On Mon, Jun 9, 2014 at 12:43 PM, Sanne Grinovero <sanne at infinispan.org>
> wrote:
>>
>> Hi Dan!
>>
>> The reason is that I'm making substantial API changes in the Query
>> module, and I've lost count on how many other modules and integration
>> tests depend on it: I need to run all the testsuite to evaluate where
>> I'm heading.. I don't need it just as last touches but continually, to
>> be able to make good choices while work is in progress.
>
>
> Wouldn't "mvn install -DskipTests" be enough for testing dependencies?
> You could then use "mvn surefire:test -pl query,lucene-directory" to run
> just the tests you're interested in.
> I have a script that does just that - even for core, I don't want to think
> about whether I changed something in commons or not, but I almost never want
> to run the commons tests :)

I need to run all modules, and AFAIK there is no way to run the tests
on all modules except one (unless you list all of them explicitly..
good luck with that).

What I did to go ahead in this ball of mud is to patch the core
pom.xml to add "skipTests" option to the surefire configuration, at
least I could test my stuff.


>> Not only I'm changing API but also substantial changes in the
>> dependency tree.. without a working testsuite I can't make progress.
>> I'm working around it by deleting core in a temporary commit... :-/
>> (and even so the suite takes more than an hour ??!)
>>
>
> 1 hour just for the core, or for everything? I used to get about 1h for
> everything, if just the core takes that long to fail it's definitely a
> problem. There was also a bit of a slowdown after the JGroups 3.5.0.Beta7
> upgrade (ISPN-4355), but that should be fixed now.

It's 1h and 20 minutes now to test all modules. This took 12 minutes a
couple of months ago. My opinion is that even 12 minutes is too slow,
I would consider it acceptable if we where running some soak tests but
as far as I know we're just load testing the testing framework.

Sanne

>
> I was also thinking of moving the failsafe plugin to the "extras" profile,
> so that we can avoid running the server integration tests in dev builds. But
> disabling the extras profile also disables the bundling for OSGi, so perhaps
> a separate profile would be better.
>
> Dan
>
>
>>
>> I'll test your PRs ASAP, thanks a lot.
>>
>> Cheers,
>> Sanne
>>
>>
>> On 9 June 2014 10:19, Dan Berindei <dan.berindei at gmail.com> wrote:
>> >
>> >
>> >
>> > On Fri, Jun 6, 2014 at 4:31 PM, Sanne Grinovero <sanne at infinispan.org>
>> > wrote:
>> >>
>> >> I'm having several failures, these are blocking our progress on Query.
>> >>
>> >> Should I disable them all?
>> >
>> >
>> > You could disable them, but I'm not quite sure how that would help the
>> > query
>> > module... surely you don't need to run the core tests every time you
>> > modify
>> > something in query?
>> >
>> >>
>> >>
>> >> Sample output of three different runs:
>> >>
>> >> Failed tests:
>> >>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>> >> locals st...
>> >
>> >
>> > I couldn't reproduce this failure on my machine, but I modified
>> > ThreadLocalLeakTest to ignore that particular thread-local:
>> > https://github.com/infinispan/infinispan/pull/2614
>> > You're the only one that reported seeing it, so please test the PR on
>> > your
>> > machine and integrate it.
>> >
>> >
>> >>
>> >>
>> >>
>> >> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> >> ? Runtime
>> >> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>> >
>> >
>> > I've created https://issues.jboss.org/browse/ISPN-4368 for Will to look
>> > into, I'm not sure we need the "placeholder" key that is causing the
>> > random
>> > failures.
>> >
>> >>
>> >>
>> >> Failed tests:
>> >>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>> >> locals st...
>> >>
>> >>
>> >> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> >> ? Runtime
>> >>
>> >>
>> >> L1StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPut:84->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> >> ? Runtime
>> >
>> >
>> > The L1StateTransferOverwriteTest failure seems to have the same cause as
>> > StateTransferOverwriteTest failure.
>> >
>> >>
>> >> Tests run: 5430, Failures: 3, Errors: 0, Skipped: 0
>> >>
>> >> Failed tests:
>> >>   ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>> >> locals st...
>> >>
>> >>
>> >> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered:611->testIterationBeganAndSegmentNotComplete:510
>> >> expected [11] but found [6]
>> >
>> >
>> > I've seen this a couple times on my machine as well, I've created
>> > https://issues.jboss.org/browse/ISPN-4370
>> >
>> >>
>> >> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>> >
>> >
>> >
>> > I'd let Will investigate ISPN-4368 and ISPN-4370 for a bit before
>> > disabling
>> > those tests - he may be able to issue a PR for them quite quickly.
>> >
>> > Still, just because I don't like disabling tests it doesn't mean I don't
>> > appreciate your stability reports - keep 'em coming!
>> >
>> >
>> > Cheers
>> > Dan
>> >
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From sanne at infinispan.org  Tue Jun 10 04:31:14 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 10 Jun 2014 09:31:14 +0100
Subject: [infinispan-dev] Leaking ThreadLocal
Message-ID: <CAFm4XO31_NrfA18ZmJYiZs0xFce+9E81LC-HO8ZdE7J8OQgHsQ@mail.gmail.com>

Hi all,
as spotted by the testsuite - which was rightfully complaining - the
CHMv8 which we backported from the JDK has a static ThreadLocal that
it uses internally.

Our copy is having the same, and so Dan enhanced the "ThreadLocal
detection" utility in the testsuite to "allow" specific threadlocal
instances; nice as at least infinispan-core now has no more failures.

Next the bigger question: this might be ok in the JDK - which
obviously doesn't get redeployed - but is it ok in our stuff?
I guess the answer is no.. would you all agree that the CHM instances
should share an instance whose lifecycle has to be managed by the
CacheManager ?

Cheers,
Sanne

From mmarkus at redhat.com  Tue Jun 10 07:04:02 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Tue, 10 Jun 2014 12:04:02 +0100
Subject: [infinispan-dev] please vote for the Infinispan 7.0 release
	name!
In-Reply-To: <CAKUx19oSxQD1eMKsQYYwCS+4wLNTTke-Q_c58GZADVxtvyOFwA@mail.gmail.com>
References: <330B0FD1-A88F-402C-9D2D-E33A47D78CEC@redhat.com>
	<73DAB30A-67F2-4360-9C80-E2300CF3D675@redhat.com>
	<5391A638.8050006@redhat.com>
	<CAKUx19oSxQD1eMKsQYYwCS+4wLNTTke-Q_c58GZADVxtvyOFwA@mail.gmail.com>
Message-ID: <9857B19D-8B83-4BB1-B04F-DCAE203D72C8@redhat.com>

I don't like Guinness THTA much! :-)

On Jun 9, 2014, at 16:33, Manik Surtani <manik at infinispan.org> wrote:
> Foul play!  :-)
> 
> 
> On 6 June 2014 12:30, Tristan Tarrant <ttarrant at redhat.com> wrote:
> You rigged it !!!!!
> 
> Tristan
> 
> On 06/06/2014 13:23, Mircea Markus wrote:
> > Guiness it is!
> >
> > https://docs.google.com/a/infinispan.org/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewanalytics
> >
> > On Jun 5, 2014, at 13:11, Mircea Markus <mmarkus at redhat.com> wrote:
> >
> >> https://docs.google.com/forms/d/1cA8qPO1nVx_G9Nz3pI_40F8HKsmEPyGSUczjbWKZgnM/viewform
> >>
> >>
> >> Cheers,
> >> --
> >> Mircea Markus
> >> Infinispan lead (www.infinispan.org)
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > Cheers,
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From galder at redhat.com  Mon Jun  9 13:39:58 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Mon, 9 Jun 2014 18:39:58 +0100
Subject: [infinispan-dev] Uber Jars
In-Reply-To: <538DA3D0.2020701@redhat.com>
References: <538D7EF2.5060103@redhat.com>	<1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>
	<CAFm4XO32xSMyoY12GY7oSscim4T=3RL6vr+OwQWx8CNh9-MMKQ@mail.gmail.com>
	<538DA3D0.2020701@redhat.com>
Message-ID: <629A3E29-E4AD-4149-B213-987ABAFDB022@redhat.com>

Sounds good to me. I do like the fact that the artifacts have a -all ending, it kinda signals that it?s an Uber jar. This might be handy for users who just look at the list of artifacts in the repository. Alternatively, we could have a group id that signals it, e.g. org.infinispan.all or org.infinispan.uber or something different ?

In terms of the actual uber jars, I?m happy with the separation of embedded/query/remote.

Cheers,

On 03 Jun 2014, at 11:30, Tristan Tarrant <ttarrant at redhat.com> wrote:

> On 03/06/2014 11:35, Sanne Grinovero wrote:
>> On 3 June 2014 09:57, Mircea Markus <mmarkus at redhat.com> wrote:
>>> On Jun 3, 2014, at 8:53, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>> 
>>>> Dear all,
>>>> 
>>>> on Thursday I issued a PR [1] to introduce Uber Jars, i.e. single jars
>>>> which wrap our multitude of jars and some of the transitive
>>>> dependencies, but it was (rightly) pointed out that we should have a
>>>> little discussion here first.
>> thanks!
>> 
>>>> Firstly, I'm using the maven shade plugin which repackages multiple jars
>>>> into one with:
>>>> - automatic transitive resolution
>>>> - the ability to include/exclude certain jars
>>>> - move classes if necessary to other packages to avoid conflicts
>>>> - rewrite the POM with the new dependencies.
>>>> 
>>>> Here is my global strategy:
>>>> - define a set of uber-jars (see below)
>>>> - include all non-optional dependencies in each uber-jar except for the
>>>> specification Jars (e.g. javax.transaction and javax.persistence)
>>>> - rename some internal-only dependencies to avoid conflicts
>>>> - uber jars MUST NOT inherit from infinispan-parent (too much cruft in
>>>> there) but only from infinispan-bom.
>> What is the definition of an "internal-only" dependency?
>> I really like the intention but renaming seems quite dangerous.
>> 
>> For example, JGroups is quite internal but doing so it means the user
>> could not create a custom Protocol and define it in his configuration
>> file.
>> Also I suspect several frameworks rely on reflection to "wire-up" at
>> boot time (like JGroups does with Protocol class names), so if we're
>> in the business of relocating classes, this should be followed by
>> integration tests.
> The only dependency which currently "fits" my definitions is 
> jboss-logging. It is only used internally and its APIs are not 
> "re-exported".
> My PR also includes a simple integration test for the "embedded" case. I 
> will obviously integrate it with tests with the others once we agree on 
> the structure.
>>> Just for my own understanding, what is the rationale behind this?
> Users who don't use a dependency management system (i.e. Maven, Ivy, 
> etc) can write an application just by adding a "single" jar.
>>> 
>>>> The Uber Jars
>>>> - infinispan-embedded-all (infinispan-commons, infinispan-core, jgroups,
>>>> jboss-marshalling-osgi, jboss-logging, infinispan-cachestore-jdbc,
>>>> infinispan-cachestore-jpa, infinispan-cachestore-leveldb)
>> I wouldn't call a package "-all" if it's missing some things. For
>> example, without Query functionality this "All" is actually a very
>> limited subset ;-)
>> 
>> WDYT "infinispan-embedded"
> Ok.
>>>> - infinispan-remote-all (infinispan-commons, infinispan-client-hotrod,
>>>> commons-pool, jboss-marshalling-osgi, jboss-logging,
>>>> infinispan-remote-query-client, infinispan-query-dsl,
>>>> infinispan-protostream, protobuf-java)
>> Same objection on the "-all" naming.
>> I'd suggest "infinispan-client".
> Ok
>>>> - infinispan-query-all (infinispan-query, infinispan-query-dsl,
>>>> hibernate-hql-parser, antlr, stringtemplate, hibernate-hql-lucene,
>>>> hibernate-search-engine, infinispan-lucene-v4,
>>>> hibernate-search-analyzers, lucene*, solr*, avro, jackson-core,
>>>> jackson-mapper, paranamer, apache-compress, infinispan-lucene-directory,
>>>> hibernate-search-infinispan, hibernate-commons-annotations). This
>>>> package will depend on infinispan-embedded-all and we should only make a
>>>> Lucene v4 version).
>> If you make only a Lucene V4 version that's not going to work with
>> master as it's today, my patches to upgade the Query engine to use
>> Lucene V4 are not integrated yet. (And if it's meant for a Lucene4
>> target, then you don't need Solr)
> I added solr since that is what my aging neuron remembered. Shade 
> actually pulls in the dependency tree automatically so I don't have to 
> think about these things (and the Infinispan Query dependency hierarchy 
> is hairier than a sea otter).
>> More importantly: is the user meant to use these as Either/OR or as
>> additive packages? From the dependencies you listed it looks like
>> additive packages, but I think the name suggests that using
>> "infinispan-query-all" you have all what's needed.
> 
> infinispan-embedded-query would still depend on infinispan-embedded, so 
> the user would require both jars. For the maven user, just depending on 
> infinispan-embedded-query will do the trick, since infinispan-embedded 
> would be a transitive.
> 
>> Much simpler this way, so why not making this the default packaging?
>> 
> The "uber" jars are what we should advertise in our quickstarts, docs, 
> etc. Obviously the single underlying jars would still be deployed to maven.
> 
> Tristan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From isavin at redhat.com  Tue Jun 10 11:13:14 2014
From: isavin at redhat.com (Ion Savin)
Date: Tue, 10 Jun 2014 18:13:14 +0300
Subject: [infinispan-dev] Uber Jars
In-Reply-To: <629A3E29-E4AD-4149-B213-987ABAFDB022@redhat.com>
References: <538D7EF2.5060103@redhat.com>	<1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>	<CAFm4XO32xSMyoY12GY7oSscim4T=3RL6vr+OwQWx8CNh9-MMKQ@mail.gmail.com>	<538DA3D0.2020701@redhat.com>
	<629A3E29-E4AD-4149-B213-987ABAFDB022@redhat.com>
Message-ID: <5397208A.9010209@redhat.com>

On 06/09/2014 08:39 PM, Galder Zamarre?o wrote:
> Sounds good to me. I do like the fact that the artifacts have a -all ending, it kinda signals that it?s an Uber jar.

Noticed on central that some projects use the -uber ending. Might be an 
option if -all is misleading:

http://search.maven.org/#search|ga|1|uber.jar

Regards,
  Ion Savin

From galder at redhat.com  Tue Jun 10 11:23:01 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Tue, 10 Jun 2014 17:23:01 +0200
Subject: [infinispan-dev] Test failures
In-Reply-To: <CAFm4XO0fdee6r4f+QT_M5rr-dD6g=j2rxMQGLCOuxyALsjRZEw@mail.gmail.com>
References: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>
	<CA+nfvwRWfZTkxcAeGs6VDT7=3TCSKUy5nfAGX9ePGRoHypwRGA@mail.gmail.com>
	<CAFm4XO265nEkRB=nV8RLcqS5WJHcrBAH+SKmAwGwOQe_Y88AWA@mail.gmail.com>
	<CA+nfvwQKdzpWhukFMdTw7bfsB0DkZg-AGPiQQfQMAjXDRJbRBQ@mail.gmail.com>
	<CAFm4XO0fdee6r4f+QT_M5rr-dD6g=j2rxMQGLCOuxyALsjRZEw@mail.gmail.com>
Message-ID: <002B61E4-8A6F-49E7-B9A8-36AED6091F67@redhat.com>

Just recently I read this: http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

Food for thought, and maybe we should consider doing for next major. Maybe if we?re moving to JUnit we could look at this in detail too?

Cheers,

On 09 Jun 2014, at 23:06, Sanne Grinovero <sanne at infinispan.org> wrote:

> On 9 June 2014 14:42, Dan Berindei <dan.berindei at gmail.com> wrote:
>> 
>> 
>> On Mon, Jun 9, 2014 at 12:43 PM, Sanne Grinovero <sanne at infinispan.org>
>> wrote:
>>> 
>>> Hi Dan!
>>> 
>>> The reason is that I'm making substantial API changes in the Query
>>> module, and I've lost count on how many other modules and integration
>>> tests depend on it: I need to run all the testsuite to evaluate where
>>> I'm heading.. I don't need it just as last touches but continually, to
>>> be able to make good choices while work is in progress.
>> 
>> 
>> Wouldn't "mvn install -DskipTests" be enough for testing dependencies?
>> You could then use "mvn surefire:test -pl query,lucene-directory" to run
>> just the tests you're interested in.
>> I have a script that does just that - even for core, I don't want to think
>> about whether I changed something in commons or not, but I almost never want
>> to run the commons tests :)
> 
> I need to run all modules, and AFAIK there is no way to run the tests
> on all modules except one (unless you list all of them explicitly..
> good luck with that).
> 
> What I did to go ahead in this ball of mud is to patch the core
> pom.xml to add "skipTests" option to the surefire configuration, at
> least I could test my stuff.
> 
> 
>>> Not only I'm changing API but also substantial changes in the
>>> dependency tree.. without a working testsuite I can't make progress.
>>> I'm working around it by deleting core in a temporary commit... :-/
>>> (and even so the suite takes more than an hour ??!)
>>> 
>> 
>> 1 hour just for the core, or for everything? I used to get about 1h for
>> everything, if just the core takes that long to fail it's definitely a
>> problem. There was also a bit of a slowdown after the JGroups 3.5.0.Beta7
>> upgrade (ISPN-4355), but that should be fixed now.
> 
> It's 1h and 20 minutes now to test all modules. This took 12 minutes a
> couple of months ago. My opinion is that even 12 minutes is too slow,
> I would consider it acceptable if we where running some soak tests but
> as far as I know we're just load testing the testing framework.
> 
> Sanne
> 
>> 
>> I was also thinking of moving the failsafe plugin to the "extras" profile,
>> so that we can avoid running the server integration tests in dev builds. But
>> disabling the extras profile also disables the bundling for OSGi, so perhaps
>> a separate profile would be better.
>> 
>> Dan
>> 
>> 
>>> 
>>> I'll test your PRs ASAP, thanks a lot.
>>> 
>>> Cheers,
>>> Sanne
>>> 
>>> 
>>> On 9 June 2014 10:19, Dan Berindei <dan.berindei at gmail.com> wrote:
>>>> 
>>>> 
>>>> 
>>>> On Fri, Jun 6, 2014 at 4:31 PM, Sanne Grinovero <sanne at infinispan.org>
>>>> wrote:
>>>>> 
>>>>> I'm having several failures, these are blocking our progress on Query.
>>>>> 
>>>>> Should I disable them all?
>>>> 
>>>> 
>>>> You could disable them, but I'm not quite sure how that would help the
>>>> query
>>>> module... surely you don't need to run the core tests every time you
>>>> modify
>>>> something in query?
>>>> 
>>>>> 
>>>>> 
>>>>> Sample output of three different runs:
>>>>> 
>>>>> Failed tests:
>>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>>>>> locals st...
>>>> 
>>>> 
>>>> I couldn't reproduce this failure on my machine, but I modified
>>>> ThreadLocalLeakTest to ignore that particular thread-local:
>>>> https://github.com/infinispan/infinispan/pull/2614
>>>> You're the only one that reported seeing it, so please test the PR on
>>>> your
>>>> machine and integrate it.
>>>> 
>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>>>>> ? Runtime
>>>>> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>>>> 
>>>> 
>>>> I've created https://issues.jboss.org/browse/ISPN-4368 for Will to look
>>>> into, I'm not sure we need the "placeholder" key that is causing the
>>>> random
>>>> failures.
>>>> 
>>>>> 
>>>>> 
>>>>> Failed tests:
>>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>>>>> locals st...
>>>>> 
>>>>> 
>>>>> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>>>>> ? Runtime
>>>>> 
>>>>> 
>>>>> L1StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPut:84->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>>>>> ? Runtime
>>>> 
>>>> 
>>>> The L1StateTransferOverwriteTest failure seems to have the same cause as
>>>> StateTransferOverwriteTest failure.
>>>> 
>>>>> 
>>>>> Tests run: 5430, Failures: 3, Errors: 0, Skipped: 0
>>>>> 
>>>>> Failed tests:
>>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
>>>>> locals st...
>>>>> 
>>>>> 
>>>>> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered:611->testIterationBeganAndSegmentNotComplete:510
>>>>> expected [11] but found [6]
>>>> 
>>>> 
>>>> I've seen this a couple times on my machine as well, I've created
>>>> https://issues.jboss.org/browse/ISPN-4370
>>>> 
>>>>> 
>>>>> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>>>> 
>>>> 
>>>> 
>>>> I'd let Will investigate ISPN-4368 and ISPN-4370 for a bit before
>>>> disabling
>>>> those tests - he may be able to issue a PR for them quite quickly.
>>>> 
>>>> Still, just because I don't like disabling tests it doesn't mean I don't
>>>> appreciate your stability reports - keep 'em coming!
>>>> 
>>>> 
>>>> Cheers
>>>> Dan
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From dan.berindei at gmail.com  Wed Jun 11 03:03:51 2014
From: dan.berindei at gmail.com (Dan Berindei)
Date: Wed, 11 Jun 2014 10:03:51 +0300
Subject: [infinispan-dev] Test failures
In-Reply-To: <002B61E4-8A6F-49E7-B9A8-36AED6091F67@redhat.com>
References: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>
	<CA+nfvwRWfZTkxcAeGs6VDT7=3TCSKUy5nfAGX9ePGRoHypwRGA@mail.gmail.com>
	<CAFm4XO265nEkRB=nV8RLcqS5WJHcrBAH+SKmAwGwOQe_Y88AWA@mail.gmail.com>
	<CA+nfvwQKdzpWhukFMdTw7bfsB0DkZg-AGPiQQfQMAjXDRJbRBQ@mail.gmail.com>
	<CAFm4XO0fdee6r4f+QT_M5rr-dD6g=j2rxMQGLCOuxyALsjRZEw@mail.gmail.com>
	<002B61E4-8A6F-49E7-B9A8-36AED6091F67@redhat.com>
Message-ID: <CA+nfvwSJR6z_iB3vdDhLfmDiko8GgW7-DOu6pgcbf9nZ1EUXkQ@mail.gmail.com>

I'll skip straight to the conclusions in the paper:

> ? Keep regression tests around for up to a year ? but most of
those will be system-level tests rather than unit tests.

I would say most of our tests qualify as regression tests, and they are
system-level (they start a full cache manager, with a JGroups channel and
everything).
But how would we know that they haven't failed in a year? I mean we can
find which tests never failed in CI, but we can't find which tests never
failed on any dev's machine (thus helping that dev find a problem in his
code).

> ? Keep unit tests that test key algorithms for which there is a
broad, formal, independent oracle of correctness, and for
which there is ascribable business value.
> ? Except for the preceding case, if X has business value and you
can text X with either a system test or a unit test, use a system
test ? context is everything.

We have about 100 tests in the "unit" group, some of which should really be
in the "functional" group (e.g. BaseStoreTest).
It shouldn't take long to walk over those and delete some that may be
tested better by system tests, but it won't have a visible effect on the
test suite run time either.

> ? Design a test with more care than you design the code.

I admit that when reviewing PRs I pay less attention to the tests than to
the production code. I will try to change that :)

> ? Turn most unit tests into assertions.

I imagine some kinds of tests can indeed be replaced with assertions in
production code, but again we don't have so many unit tests.

> ? Throw away tests that haven?t failed in a year.
> ? Testing can?t replace good development: a high test failure
rate suggests you should shorten development intervals,
perhaps radically, and make sure your architecture and design
regimens have teeth
> ? If you find that individual functions being tested are trivial,
double-check the way you incentivize developers?
performance. Rewarding coverage or other meaningless
metrics can lead to rapid architecture decay.
> ? Be humble about what tests can achieve. Tests don?t improve
quality: developers do.




On Tue, Jun 10, 2014 at 6:23 PM, Galder Zamarre?o <galder at redhat.com> wrote:

> Just recently I read this:
> http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf
>
> Food for thought, and maybe we should consider doing for next major. Maybe
> if we?re moving to JUnit we could look at this in detail too?
>
> Cheers,
>
> On 09 Jun 2014, at 23:06, Sanne Grinovero <sanne at infinispan.org> wrote:
>
> > On 9 June 2014 14:42, Dan Berindei <dan.berindei at gmail.com> wrote:
> >>
> >>
> >> On Mon, Jun 9, 2014 at 12:43 PM, Sanne Grinovero <sanne at infinispan.org>
> >> wrote:
> >>>
> >>> Hi Dan!
> >>>
> >>> The reason is that I'm making substantial API changes in the Query
> >>> module, and I've lost count on how many other modules and integration
> >>> tests depend on it: I need to run all the testsuite to evaluate where
> >>> I'm heading.. I don't need it just as last touches but continually, to
> >>> be able to make good choices while work is in progress.
> >>
> >>
> >> Wouldn't "mvn install -DskipTests" be enough for testing dependencies?
> >> You could then use "mvn surefire:test -pl query,lucene-directory" to run
> >> just the tests you're interested in.
> >> I have a script that does just that - even for core, I don't want to
> think
> >> about whether I changed something in commons or not, but I almost never
> want
> >> to run the commons tests :)
> >
> > I need to run all modules, and AFAIK there is no way to run the tests
> > on all modules except one (unless you list all of them explicitly..
> > good luck with that).
> >
> > What I did to go ahead in this ball of mud is to patch the core
> > pom.xml to add "skipTests" option to the surefire configuration, at
> > least I could test my stuff.
> >
> >
> >>> Not only I'm changing API but also substantial changes in the
> >>> dependency tree.. without a working testsuite I can't make progress.
> >>> I'm working around it by deleting core in a temporary commit... :-/
> >>> (and even so the suite takes more than an hour ??!)
> >>>
> >>
> >> 1 hour just for the core, or for everything? I used to get about 1h for
> >> everything, if just the core takes that long to fail it's definitely a
> >> problem. There was also a bit of a slowdown after the JGroups
> 3.5.0.Beta7
> >> upgrade (ISPN-4355), but that should be fixed now.
> >
> > It's 1h and 20 minutes now to test all modules. This took 12 minutes a
> > couple of months ago. My opinion is that even 12 minutes is too slow,
> > I would consider it acceptable if we where running some soak tests but
> > as far as I know we're just load testing the testing framework.
> >
> > Sanne
> >
> >>
> >> I was also thinking of moving the failsafe plugin to the "extras"
> profile,
> >> so that we can avoid running the server integration tests in dev
> builds. But
> >> disabling the extras profile also disables the bundling for OSGi, so
> perhaps
> >> a separate profile would be better.
> >>
> >> Dan
> >>
> >>
> >>>
> >>> I'll test your PRs ASAP, thanks a lot.
> >>>
> >>> Cheers,
> >>> Sanne
> >>>
> >>>
> >>> On 9 June 2014 10:19, Dan Berindei <dan.berindei at gmail.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Jun 6, 2014 at 4:31 PM, Sanne Grinovero <sanne at infinispan.org
> >
> >>>> wrote:
> >>>>>
> >>>>> I'm having several failures, these are blocking our progress on
> Query.
> >>>>>
> >>>>> Should I disable them all?
> >>>>
> >>>>
> >>>> You could disable them, but I'm not quite sure how that would help the
> >>>> query
> >>>> module... surely you don't need to run the core tests every time you
> >>>> modify
> >>>> something in query?
> >>>>
> >>>>>
> >>>>>
> >>>>> Sample output of three different runs:
> >>>>>
> >>>>> Failed tests:
> >>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> >>>>> locals st...
> >>>>
> >>>>
> >>>> I couldn't reproduce this failure on my machine, but I modified
> >>>> ThreadLocalLeakTest to ignore that particular thread-local:
> >>>> https://github.com/infinispan/infinispan/pull/2614
> >>>> You're the only one that reported seeing it, so please test the PR on
> >>>> your
> >>>> machine and integrate it.
> >>>>
> >>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> >>>>> ? Runtime
> >>>>> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
> >>>>
> >>>>
> >>>> I've created https://issues.jboss.org/browse/ISPN-4368 for Will to
> look
> >>>> into, I'm not sure we need the "placeholder" key that is causing the
> >>>> random
> >>>> failures.
> >>>>
> >>>>>
> >>>>>
> >>>>> Failed tests:
> >>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> >>>>> locals st...
> >>>>>
> >>>>>
> >>>>>
> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> >>>>> ? Runtime
> >>>>>
> >>>>>
> >>>>>
> L1StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPut:84->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
> >>>>> ? Runtime
> >>>>
> >>>>
> >>>> The L1StateTransferOverwriteTest failure seems to have the same cause
> as
> >>>> StateTransferOverwriteTest failure.
> >>>>
> >>>>>
> >>>>> Tests run: 5430, Failures: 3, Errors: 0, Skipped: 0
> >>>>>
> >>>>> Failed tests:
> >>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
> >>>>> locals st...
> >>>>>
> >>>>>
> >>>>>
> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered:611->testIterationBeganAndSegmentNotComplete:510
> >>>>> expected [11] but found [6]
> >>>>
> >>>>
> >>>> I've seen this a couple times on my machine as well, I've created
> >>>> https://issues.jboss.org/browse/ISPN-4370
> >>>>
> >>>>>
> >>>>> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
> >>>>
> >>>>
> >>>>
> >>>> I'd let Will investigate ISPN-4368 and ISPN-4370 for a bit before
> >>>> disabling
> >>>> those tests - he may be able to issue a PR for them quite quickly.
> >>>>
> >>>> Still, just because I don't like disabling tests it doesn't mean I
> don't
> >>>> appreciate your stability reports - keep 'em coming!
> >>>>
> >>>>
> >>>> Cheers
> >>>> Dan
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140611/7b72339e/attachment.html 

From ttarrant at redhat.com  Wed Jun 11 03:40:13 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Wed, 11 Jun 2014 09:40:13 +0200
Subject: [infinispan-dev] Uber Jars
In-Reply-To: <5397208A.9010209@redhat.com>
References: <538D7EF2.5060103@redhat.com>	<1A27A249-11E4-46A6-9833-FC3D0A633778@redhat.com>	<CAFm4XO32xSMyoY12GY7oSscim4T=3RL6vr+OwQWx8CNh9-MMKQ@mail.gmail.com>	<538DA3D0.2020701@redhat.com>	<629A3E29-E4AD-4149-B213-987ABAFDB022@redhat.com>
	<5397208A.9010209@redhat.com>
Message-ID: <539807DD.3070407@redhat.com>

infinispan-embedded-obese

Tristan

On 10/06/2014 17:13, Ion Savin wrote:
> On 06/09/2014 08:39 PM, Galder Zamarre?o wrote:
>> Sounds good to me. I do like the fact that the artifacts have a -all ending, it kinda signals that it?s an Uber jar.
> Noticed on central that some projects use the -uber ending. Might be an
> option if -all is misleading:
>
> http://search.maven.org/#search|ga|1|uber.jar
>
> Regards,
>    Ion Savin
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>


From gustavonalle at gmail.com  Wed Jun 11 04:52:03 2014
From: gustavonalle at gmail.com (Gustavo Fernandes)
Date: Wed, 11 Jun 2014 09:52:03 +0100
Subject: [infinispan-dev] Query integration tests performance and JGroups
Message-ID: <1402476723.14336.27.camel@localhost.localdomain>

Hi,

While investigating some CI failures in query (timeouts), I've narrowed
down the issue to the Jgroups protocol stack being used. 
Running a 'mvn clean install' in the query/ module takes about 6min
(when timeout does not happen). If I run instead: 

mvn -Dtest.protocol.jgroups=udp clean install

Time goes down to around 50s. Recent changes in core's jgoups-tcp.xml
for the tests were the removal of the loopback=true and the modification
of the bundler_type, but they don't seem to affect the outcome.

FYI, taking a single test and stripping down from it everything but the
cluster formation and data population (5 objects) leads to the cpu
hotspot below, and it takes almost 1 minute  

I'd be happy to change the query tests to udp, but first would like to
hear your thoughts about this issue

Gustavo


+----------------------------------------------------------------------------------+------------------+--------------------+
|                                       Name                                       |    Time (ms)     |  Invocation Count  |
+----------------------------------------------------------------------------------+------------------+--------------------+
|  +---java.net.SocketInputStream.read(byte[], int, int, int)                      |  101,742  100 %  |             4,564  |
|    |                                                                             |                  |                    |
|    +---java.net.SocketInputStream.read(byte[], int, int)                         |                  |                    |
|      |                                                                           |                  |                    |
|      +---java.io.BufferedInputStream.fill()                                      |                  |                    |
|        |                                                                         |                  |                    |
|        +---java.io.BufferedInputStream.read()                                    |                  |                    |
|          |                                                                       |                  |                    |
|          +---java.io.DataInputStream.readInt()                                   |                  |                    |
|            |                                                                     |                  |                    |
|            +---org.jgroups.blocks.TCPConnectionMap$TCPConnection$Receiver.run()  |                  |                    |
|              |                                                                   |                  |                    |
|              +---java.lang.Thread.run()                                          |                  |                    |
+----------------------------------------------------------------------------------+------------------+--------------------+

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140611/905c230d/attachment-0001.html 

From bban at redhat.com  Wed Jun 11 05:13:54 2014
From: bban at redhat.com (Bela Ban)
Date: Wed, 11 Jun 2014 11:13:54 +0200
Subject: [infinispan-dev] Query integration tests performance and JGroups
In-Reply-To: <1402476723.14336.27.camel@localhost.localdomain>
References: <1402476723.14336.27.camel@localhost.localdomain>
Message-ID: <53981DD2.2060500@redhat.com>

What's that test doing ? Can this be reproduced in JGroups ?

Can you perhaps instrument the JGroups src code in Receiver.run() to 
print out the number of bytes read and print the Throwable to stderr ?

Somehow this smells like a tight loop, but I wonder what would cause 
this... A closed socket would terminate the loop and reading an EOF 
would also terminate it...

On 11/06/14 10:52, Gustavo Fernandes wrote:
> Hi,
>
> While investigating some CI failures in query (timeouts), I've narrowed
> down the issue to the Jgroups protocol stack being used.
> Running a 'mvn clean install' in the query/ module takes about 6min
> (when timeout does not happen). If I run instead:
>
> mvn -Dtest.protocol.jgroups=udp clean install
>
> Time goes down to around 50s. Recent changes in core's jgoups-tcp.xml
> for the tests were the removal of the loopback=true and the modification
> of the bundler_type, but they don't seem to affect the outcome.
>
> FYI, taking a single test and stripping down from it everything but the
> cluster formation and data population (5 objects) leads to the cpu
> hotspot below, and it takes almost 1 minute
>
> I'd be happy to change the query tests to udp, but first would like to
> hear your thoughts about this issue
>
> Gustavo
>
> +----------------------------------------------------------------------------------+------------------+--------------------+
> |                                       Name                                       |    Time (ms)     |  Invocation Count  |
> +----------------------------------------------------------------------------------+------------------+--------------------+
> |  +---java.net.SocketInputStream.read(byte[], int, int, int)                      |  101,742  100 %  |             4,564  |
> |    |                                                                             |                  |                    |
> |    +---java.net.SocketInputStream.read(byte[], int, int)                         |                  |                    |
> |      |                                                                           |                  |                    |
> |      +---java.io.BufferedInputStream.fill()                                      |                  |                    |
> |        |                                                                         |                  |                    |
> |        +---java.io.BufferedInputStream.read()                                    |                  |                    |
> |          |                                                                       |                  |                    |
> |          +---java.io.DataInputStream.readInt()                                   |                  |                    |
> |            |                                                                     |                  |                    |
> |            +---org.jgroups.blocks.TCPConnectionMap$TCPConnection$Receiver.run()  |                  |                    |
> |              |                                                                   |                  |                    |
> |              +---java.lang.Thread.run()                                          |                  |                    |
> +----------------------------------------------------------------------------------+------------------+--------------------+
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)

From galder at redhat.com  Wed Jun 11 06:38:26 2014
From: galder at redhat.com (=?iso-8859-1?Q?Galder_Zamarre=F1o?=)
Date: Wed, 11 Jun 2014 12:38:26 +0200
Subject: [infinispan-dev] Leaking ThreadLocal
In-Reply-To: <CAFm4XO31_NrfA18ZmJYiZs0xFce+9E81LC-HO8ZdE7J8OQgHsQ@mail.gmail.com>
References: <CAFm4XO31_NrfA18ZmJYiZs0xFce+9E81LC-HO8ZdE7J8OQgHsQ@mail.gmail.com>
Message-ID: <7BE52DA8-AF6B-4C83-A783-AFFEAD8240CB@redhat.com>


On 10 Jun 2014, at 10:31, Sanne Grinovero <sanne at infinispan.org> wrote:

> Hi all,
> as spotted by the testsuite - which was rightfully complaining - the
> CHMv8 which we backported from the JDK has a static ThreadLocal that
> it uses internally.
> 
> Our copy is having the same, and so Dan enhanced the "ThreadLocal
> detection" utility in the testsuite to "allow" specific threadlocal
> instances; nice as at least infinispan-core now has no more failures.
> 
> Next the bigger question: this might be ok in the JDK - which
> obviously doesn't get redeployed - but is it ok in our stuff?
> I guess the answer is no.. would you all agree that the CHM instances
> should share an instance whose lifecycle has to be managed by the
> CacheManager ?

Either that or modify CHMv8 code to use a ConcurrentWeakKeyHashMap<Thread, CounterHashCode> instance variable.

We could also email concurrency-interest at cs.oswego.edu to get any other suggestions from Doug et al?

Cheers,

> 
> Cheers,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From sanne at infinispan.org  Wed Jun 11 11:46:56 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 11 Jun 2014 16:46:56 +0100
Subject: [infinispan-dev] Leaking ThreadLocal
In-Reply-To: <7BE52DA8-AF6B-4C83-A783-AFFEAD8240CB@redhat.com>
References: <CAFm4XO31_NrfA18ZmJYiZs0xFce+9E81LC-HO8ZdE7J8OQgHsQ@mail.gmail.com>
	<7BE52DA8-AF6B-4C83-A783-AFFEAD8240CB@redhat.com>
Message-ID: <CAFm4XO0Pqxx-O7C+aAQObJO-fPsp-KH0wJ3NjjYbrF_FSfTFbw@mail.gmail.com>

On 11 June 2014 11:38, Galder Zamarre?o <galder at redhat.com> wrote:
>
> On 10 Jun 2014, at 10:31, Sanne Grinovero <sanne at infinispan.org> wrote:
>
>> Hi all,
>> as spotted by the testsuite - which was rightfully complaining - the
>> CHMv8 which we backported from the JDK has a static ThreadLocal that
>> it uses internally.
>>
>> Our copy is having the same, and so Dan enhanced the "ThreadLocal
>> detection" utility in the testsuite to "allow" specific threadlocal
>> instances; nice as at least infinispan-core now has no more failures.
>>
>> Next the bigger question: this might be ok in the JDK - which
>> obviously doesn't get redeployed - but is it ok in our stuff?
>> I guess the answer is no.. would you all agree that the CHM instances
>> should share an instance whose lifecycle has to be managed by the
>> CacheManager ?
>
> Either that or modify CHMv8 code to use a ConcurrentWeakKeyHashMap<Thread, CounterHashCode> instance variable.

Right, but then we'd need to validate impact on performance, and Weak
References are a nightmare for the JVM to organize.

> We could also email concurrency-interest at cs.oswego.edu to get any other suggestions from Doug et al?

Worst case why not :) But I suspect we can handle this, and they might
not be interested as it's not a bug in the JDK.

Created: https://issues.jboss.org/browse/ISPN-4390

Sanne

>
> Cheers,
>
>>
>> Cheers,
>> Sanne
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From sanne at infinispan.org  Wed Jun 11 12:02:03 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 11 Jun 2014 17:02:03 +0100
Subject: [infinispan-dev] More commits, or squashing?
Message-ID: <CAFm4XO08TvV6p-TnadJ2deB9Vek0r3KVdp7aO_YF+rE9zhv36Q@mail.gmail.com>

A discussion which could interest more of you:
https://github.com/infinispan/infinispan/pull/2613#issuecomment-45760469

There is no golden rule, but I believe it's highly desirable to give
the sequence a logical flow which makes sense.
In this specific situation, it also helped myself as I've been making
slow progress fragmented over some months on it.

For example, this single commit is a single sed script, whose source
is included in the commit:
https://github.com/Sanne/infinispan/commit/c4b22038761d38da21480d87a37abc8a0b3c1a30

Guess what I need to do when rebasing?

Also if you keep the flow "logical" and separate file rename/move from
changes, git will help you massively on resolving conflicts on any
change made concurrently on the same files. For example the WildFly 81
PR would have been much less painful, and I think would have been
integrated way earlier as it wouldn't look as scary..

Finally, I want to win the commits competitions of course ;-)
https://github.com/Sanne

Cheers,
Sanne

From dereed at redhat.com  Wed Jun 11 12:27:41 2014
From: dereed at redhat.com (Dennis Reed)
Date: Wed, 11 Jun 2014 11:27:41 -0500
Subject: [infinispan-dev] Query integration tests performance and JGroups
In-Reply-To: <1402476723.14336.27.camel@localhost.localdomain>
References: <1402476723.14336.27.camel@localhost.localdomain>
Message-ID: <5398837D.3050707@redhat.com>

Can you double check that you're interpreting the profiler data correctly
(specifically with respect to where threads are spending time versus 
where they are using CPU)?

The spot you pointed out *should* show up as a place where threads spend 
lots of time,
as these threads just sit waiting in the read calls for the vast 
majority of their life.

But it should *not* be a CPU hotspot -- these threads should be idle 
during that time.

-Dennis

On 06/11/2014 03:52 AM, Gustavo Fernandes wrote:
> Hi,
>
> While investigating some CI failures in query (timeouts), I've 
> narrowed down the issue to the Jgroups protocol stack being used.
> Running a 'mvn clean install' in the query/ module takes about 6min 
> (when timeout does not happen). If I run instead:
>
> mvn -Dtest.protocol.jgroups=udp clean install
>
> Time goes down to around 50s. Recent changes in core's jgoups-tcp.xml 
> for the tests were the removal of the loopback=true and the 
> modification of the bundler_type, but they don't seem to affect the 
> outcome.
>
> FYI, taking a single test and stripping down from it everything but 
> the cluster formation and data population (5 objects) leads to the cpu 
> hotspot below, and it takes almost 1 minute
>
> I'd be happy to change the query tests to udp, but first would like to 
> hear your thoughts about this issue
>
> Gustavo
>
> +----------------------------------------------------------------------------------+------------------+--------------------+
> |                                       Name                                       |    Time (ms)     |  Invocation Count  |
> +----------------------------------------------------------------------------------+------------------+--------------------+
> |  +---java.net.SocketInputStream.read(byte[], int, int, int)                      |  101,742  100 %  |             4,564  |
> |    |                                                                             |                  |                    |
> |    +---java.net.SocketInputStream.read(byte[], int, int)                         |                  |                    |
> |      |                                                                           |                  |                    |
> |      +---java.io.BufferedInputStream.fill()                                      |                  |                    |
> |        |                                                                         |                  |                    |
> |        +---java.io.BufferedInputStream.read()                                    |                  |                    |
> |          |                                                                       |                  |                    |
> |          +---java.io.DataInputStream.readInt()                                   |                  |                    |
> |            |                                                                     |                  |                    |
> |            +---org.jgroups.blocks.TCPConnectionMap$TCPConnection$Receiver.run()  |                  |                    |
> |              |                                                                   |                  |                    |
> |              +---java.lang.Thread.run()                                          |                  |                    |
> +----------------------------------------------------------------------------------+------------------+--------------------+
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140611/16eada77/attachment.html 

From anistor at redhat.com  Wed Jun 11 12:45:08 2014
From: anistor at redhat.com (Adrian Nistor)
Date: Wed, 11 Jun 2014 19:45:08 +0300
Subject: [infinispan-dev] More commits, or squashing?
In-Reply-To: <CAFm4XO08TvV6p-TnadJ2deB9Vek0r3KVdp7aO_YF+rE9zhv36Q@mail.gmail.com>
References: <CAFm4XO08TvV6p-TnadJ2deB9Vek0r3KVdp7aO_YF+rE9zhv36Q@mail.gmail.com>
Message-ID: <53988794.6000604@redhat.com>

I think the journey is at least as important as the destination and 
documenting each major step along the way by leaving a clear mark in 
version history is a good thing. +1 for splitting commits whenever it 
makes sense.

Adrian

On 06/11/2014 07:02 PM, Sanne Grinovero wrote:
> A discussion which could interest more of you:
> https://github.com/infinispan/infinispan/pull/2613#issuecomment-45760469
>
> There is no golden rule, but I believe it's highly desirable to give
> the sequence a logical flow which makes sense.
> In this specific situation, it also helped myself as I've been making
> slow progress fragmented over some months on it.
>
> For example, this single commit is a single sed script, whose source
> is included in the commit:
> https://github.com/Sanne/infinispan/commit/c4b22038761d38da21480d87a37abc8a0b3c1a30
>
> Guess what I need to do when rebasing?
>
> Also if you keep the flow "logical" and separate file rename/move from
> changes, git will help you massively on resolving conflicts on any
> change made concurrently on the same files. For example the WildFly 81
> PR would have been much less painful, and I think would have been
> integrated way earlier as it wouldn't look as scary..
>
> Finally, I want to win the commits competitions of course ;-)
> https://github.com/Sanne
>
> Cheers,
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From mmarkus at redhat.com  Wed Jun 11 12:48:47 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Wed, 11 Jun 2014 17:48:47 +0100
Subject: [infinispan-dev] More commits, or squashing?
In-Reply-To: <53988794.6000604@redhat.com>
References: <CAFm4XO08TvV6p-TnadJ2deB9Vek0r3KVdp7aO_YF+rE9zhv36Q@mail.gmail.com>
	<53988794.6000604@redhat.com>
Message-ID: <3C0487E9-0AF4-4BAB-A5A3-F0D8C87269CF@redhat.com>


On Jun 11, 2014, at 17:45, Adrian Nistor <anistor at redhat.com> wrote:

> I think the journey is at least as important as the destination and 
> documenting each major step along the way by leaving a clear mark in 
> version history is a good thing. +1 for splitting commits whenever it 
> makes sense.

+1, greatly helps with review and history.

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)





From gustavonalle at gmail.com  Wed Jun 11 13:03:54 2014
From: gustavonalle at gmail.com (Gustavo Fernandes)
Date: Wed, 11 Jun 2014 18:03:54 +0100
Subject: [infinispan-dev] Query integration tests performance and JGroups
In-Reply-To: <5398837D.3050707@redhat.com>
References: <1402476723.14336.27.camel@localhost.localdomain>
	<5398837D.3050707@redhat.com>
Message-ID: <1402506234.14336.45.camel@localhost.localdomain>

Hi, sorry for the confusion. The profiler was configured to measure
"wall time" for all the methods rather than CPU time. During the test, 
CPU usage was pretty low (< 10%).

Btw, Ion Savin suggested trying to change the tcp_nodelay setting to
"true": it made a big difference, tests went down from 6min to 1min in
my environment, only about 10% slower than UDP

Gustavo

On Wed, 2014-06-11 at 11:27 -0500, Dennis Reed wrote:
> Can you double check that you're interpreting the profiler data
> correctly
> (specifically with respect to where threads are spending time versus
> where they are using CPU)?
> 
> The spot you pointed out *should* show up as a place where threads
> spend lots of time,
> as these threads just sit waiting in the read calls for the vast
> majority of their life.
> 
> But it should *not* be a CPU hotspot -- these threads should be idle
> during that time.
> 
> -Dennis
> 
> On 06/11/2014 03:52 AM, Gustavo Fernandes wrote:
> 
> > Hi,
> > 
> > While investigating some CI failures in query (timeouts), I've
> > narrowed down the issue to the Jgroups protocol stack being used. 
> > Running a 'mvn clean install' in the query/ module takes about 6min
> > (when timeout does not happen). If I run instead: 
> > 
> > mvn -Dtest.protocol.jgroups=udp clean install
> > 
> > Time goes down to around 50s. Recent changes in core's
> > jgoups-tcp.xml for the tests were the removal of the loopback=true
> > and the modification of the bundler_type, but they don't seem to
> > affect the outcome.
> > 
> > FYI, taking a single test and stripping down from it everything but
> > the cluster formation and data population (5 objects) leads to the
> > cpu hotspot below, and it takes almost 1 minute  
> > 
> > I'd be happy to change the query tests to udp, but first would like
> > to hear your thoughts about this issue
> > 
> > Gustavo
> > 
> > +----------------------------------------------------------------------------------+------------------+--------------------+
> > |                                       Name                                       |    Time (ms)     |  Invocation Count  |
> > +----------------------------------------------------------------------------------+------------------+--------------------+
> > |  +---java.net.SocketInputStream.read(byte[], int, int, int)                      |  101,742  100 %  |             4,564  |
> > |    |                                                                             |                  |                    |
> > |    +---java.net.SocketInputStream.read(byte[], int, int)                         |                  |                    |
> > |      |                                                                           |                  |                    |
> > |      +---java.io.BufferedInputStream.fill()                                      |                  |                    |
> > |        |                                                                         |                  |                    |
> > |        +---java.io.BufferedInputStream.read()                                    |                  |                    |
> > |          |                                                                       |                  |                    |
> > |          +---java.io.DataInputStream.readInt()                                   |                  |                    |
> > |            |                                                                     |                  |                    |
> > |            +---org.jgroups.blocks.TCPConnectionMap$TCPConnection$Receiver.run()  |                  |                    |
> > |              |                                                                   |                  |                    |
> > |              +---java.lang.Thread.run()                                          |                  |                    |
> > +----------------------------------------------------------------------------------+------------------+--------------------+
> > 
> > 
> > 
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From vblagoje at redhat.com  Wed Jun 11 14:48:47 2014
From: vblagoje at redhat.com (Vladimir Blagojevic)
Date: Wed, 11 Jun 2014 14:48:47 -0400
Subject: [infinispan-dev] More commits, or squashing?
In-Reply-To: <3C0487E9-0AF4-4BAB-A5A3-F0D8C87269CF@redhat.com>
References: <CAFm4XO08TvV6p-TnadJ2deB9Vek0r3KVdp7aO_YF+rE9zhv36Q@mail.gmail.com>	<53988794.6000604@redhat.com>
	<3C0487E9-0AF4-4BAB-A5A3-F0D8C87269CF@redhat.com>
Message-ID: <5398A48F.3030904@redhat.com>

On 2014-06-11, 12:48 PM, Mircea Markus wrote:
> On Jun 11, 2014, at 17:45, Adrian Nistor <anistor at redhat.com> wrote:
>
>> I think the journey is at least as important as the destination and
>> documenting each major step along the way by leaving a clear mark in
>> version history is a good thing. +1 for splitting commits whenever it
>> makes sense.
> +1, greatly helps with review and history.
>
> Cheers,
Well said. I agree as well. I don't see anything bad in splitting a 
major change in a few logical commits.


From ttarrant at redhat.com  Thu Jun 12 07:58:34 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Thu, 12 Jun 2014 13:58:34 +0200
Subject: [infinispan-dev] "Lightweight" security
Message-ID: <539995EA.80402@redhat.com>

Hi all,

This morning I have issued a rather large PR [1] which contains a new 
way to handle security. Because of the implications, I thought it best 
to explain the rationale behind it here.

When a cache has been configured with authorization, retrieving such a 
cache returns an instance of SecureCache. SecureCache is a simple 
wrapper around a Cache which checks whether the "current user" has 
enough permissions to perform an operation.
For us the "current user" is a Subject associated with the 
AccessControlContext, which basically means Subject.doAs(subject, ...).
Occasionally, however, we have internal code which needs to do things 
with caches with "elevated privileges". As an example, the distexec code 
needs to check the cache configuration and obtain the RpcManager, 
operations which the current Subject might not have.
In my initial implementation, this was done using the 
AccessController.doPrivileged(...) method which requires the presence of 
a SecurityManager to be able to grant the privileges to trusted code.
Unfortunately a SecurityManager is a bit overkill (usability-wise and 
performance-wise) for our use-case, since its purpose is mainly to avoid 
running untrusted code (think an applet).
But the typical "authorization"-enabled Infinispan application is 
running trusted code, and it just needs a way to identify one Subject 
from another when performing cache operations.

So, I present to you the new org.infinispan.security.Security class 
which uses a much simpler approach: it identifies "privileged" code 
simply by its package name, using the caller stack in a way similar to 
what logging frameworks do. While this approach is "insecure" in the 
usual meaning of the word, since it can be easily subverted, it covers 
the common use-case in a much lighter and faster way.
Obviously using a SecurityManager, if so inclined, is still supported 
and will be used to validate privileges if present.

Tristan

[1] https://github.com/infinispan/infinispan/pull/2629

From mgencur at redhat.com  Thu Jun 12 08:30:34 2014
From: mgencur at redhat.com (Martin Gencur)
Date: Thu, 12 Jun 2014 14:30:34 +0200
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
Message-ID: <53999D6A.9080300@redhat.com>

Hi,
let me mention an issue that several people faced in the past, 
independently of each other:

A user app uses a custom JGroups configuration file. However, they 
choose the same name as the files which we bundle inside 
infinispan-core.jar.
Result? People are wondering why their custom configuration does not 
take effect.
Reason? Infinispan uses the default jgroups file bundled in infinispan-core
Who faced the issue? (I suppose it's just a small subset:)) Me, Radim, 
Alan, Wolf Fink

I believe a lot of users run into this issue.

We were considering a possible solution and this one seems like it could 
work (use both 1) and 2)):
1) rename the config files in the distribution e.g. this way:
jgroups-ec2.xml -> default-jgroups-ec2.xml
jgroups-udp.xml -> default-jgroups-udp.xml
jgroups-tcp.xml -> default-jgroups-tcp.xml

Any other suggestions? internal-jgroups-udp.xml ? 
dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml 
? (joke)
(simply something that users would automatically like to change once 
they use it in their app)

2) Throw a warning whenever a user wants to use a custom jgroups 
configuration file that has the same name as one of the above


WDYT?

Thanks!
Martin


From ttarrant at redhat.com  Thu Jun 12 08:37:22 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Thu, 12 Jun 2014 14:37:22 +0200
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <53999D6A.9080300@redhat.com>
References: <53999D6A.9080300@redhat.com>
Message-ID: <53999F02.3020008@redhat.com>

I think the "internal" jgroups files should be "moved" to a separate 
directory within the core jar, to be searched after the "root". So the 
user can still provide a jgroups-udp.xml and it won't conflict.

Tristan

On 12/06/14 14:30, Martin Gencur wrote:
> Hi,
> let me mention an issue that several people faced in the past,
> independently of each other:
>
> A user app uses a custom JGroups configuration file. However, they
> choose the same name as the files which we bundle inside
> infinispan-core.jar.
> Result? People are wondering why their custom configuration does not
> take effect.
> Reason? Infinispan uses the default jgroups file bundled in infinispan-core
> Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
> Alan, Wolf Fink
>
> I believe a lot of users run into this issue.
>
> We were considering a possible solution and this one seems like it could
> work (use both 1) and 2)):
> 1) rename the config files in the distribution e.g. this way:
> jgroups-ec2.xml -> default-jgroups-ec2.xml
> jgroups-udp.xml -> default-jgroups-udp.xml
> jgroups-tcp.xml -> default-jgroups-tcp.xml
>
> Any other suggestions? internal-jgroups-udp.xml ?
> dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
> ? (joke)
> (simply something that users would automatically like to change once
> they use it in their app)
>
> 2) Throw a warning whenever a user wants to use a custom jgroups
> configuration file that has the same name as one of the above
>
>
> WDYT?
>
> Thanks!
> Martin
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>


From afield at redhat.com  Thu Jun 12 08:44:04 2014
From: afield at redhat.com (Alan Field)
Date: Thu, 12 Jun 2014 08:44:04 -0400 (EDT)
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <53999F02.3020008@redhat.com>
References: <53999D6A.9080300@redhat.com> <53999F02.3020008@redhat.com>
Message-ID: <386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>

Tristan,

So the server and library configuration parsers will handle something like this?

    <jgroups>
        <stack-file name="jgroupsStack" path="config/jgroups-udp.xml"/>
    </jgroups>

If this is true, then I agree that this is a good solution as well.

Thanks,
Alan

----- Original Message -----
> From: "Tristan Tarrant" <ttarrant at redhat.com>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Thursday, June 12, 2014 2:37:22 PM
> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
> 
> I think the "internal" jgroups files should be "moved" to a separate
> directory within the core jar, to be searched after the "root". So the
> user can still provide a jgroups-udp.xml and it won't conflict.
> 
> Tristan
> 
> On 12/06/14 14:30, Martin Gencur wrote:
> > Hi,
> > let me mention an issue that several people faced in the past,
> > independently of each other:
> >
> > A user app uses a custom JGroups configuration file. However, they
> > choose the same name as the files which we bundle inside
> > infinispan-core.jar.
> > Result? People are wondering why their custom configuration does not
> > take effect.
> > Reason? Infinispan uses the default jgroups file bundled in infinispan-core
> > Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
> > Alan, Wolf Fink
> >
> > I believe a lot of users run into this issue.
> >
> > We were considering a possible solution and this one seems like it could
> > work (use both 1) and 2)):
> > 1) rename the config files in the distribution e.g. this way:
> > jgroups-ec2.xml -> default-jgroups-ec2.xml
> > jgroups-udp.xml -> default-jgroups-udp.xml
> > jgroups-tcp.xml -> default-jgroups-tcp.xml
> >
> > Any other suggestions? internal-jgroups-udp.xml ?
> > dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
> > ? (joke)
> > (simply something that users would automatically like to change once
> > they use it in their app)
> >
> > 2) Throw a warning whenever a user wants to use a custom jgroups
> > configuration file that has the same name as one of the above
> >
> >
> > WDYT?
> >
> > Thanks!
> > Martin
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From ttarrant at redhat.com  Thu Jun 12 09:32:56 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Thu, 12 Jun 2014 15:32:56 +0200
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
References: <53999D6A.9080300@redhat.com> <53999F02.3020008@redhat.com>
	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
Message-ID: <5399AC08.8080406@redhat.com>

Actually my idea is:

I specify I want jgroups-udp.xml
so first of all I look at the classpath root: if I find the file there I 
use it, otherwise I see if it is in (proposed) 
META-INF/_internal/jgroups-udp.xml

Tristan

On 12/06/14 14:44, Alan Field wrote:
> Tristan,
>
> So the server and library configuration parsers will handle something like this?
>
>      <jgroups>
>          <stack-file name="jgroupsStack" path="config/jgroups-udp.xml"/>
>      </jgroups>
>
> If this is true, then I agree that this is a good solution as well.
>
> Thanks,
> Alan
>
> ----- Original Message -----
>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>> Sent: Thursday, June 12, 2014 2:37:22 PM
>> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
>>
>> I think the "internal" jgroups files should be "moved" to a separate
>> directory within the core jar, to be searched after the "root". So the
>> user can still provide a jgroups-udp.xml and it won't conflict.
>>
>> Tristan
>>
>> On 12/06/14 14:30, Martin Gencur wrote:
>>> Hi,
>>> let me mention an issue that several people faced in the past,
>>> independently of each other:
>>>
>>> A user app uses a custom JGroups configuration file. However, they
>>> choose the same name as the files which we bundle inside
>>> infinispan-core.jar.
>>> Result? People are wondering why their custom configuration does not
>>> take effect.
>>> Reason? Infinispan uses the default jgroups file bundled in infinispan-core
>>> Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
>>> Alan, Wolf Fink
>>>
>>> I believe a lot of users run into this issue.
>>>
>>> We were considering a possible solution and this one seems like it could
>>> work (use both 1) and 2)):
>>> 1) rename the config files in the distribution e.g. this way:
>>> jgroups-ec2.xml -> default-jgroups-ec2.xml
>>> jgroups-udp.xml -> default-jgroups-udp.xml
>>> jgroups-tcp.xml -> default-jgroups-tcp.xml
>>>
>>> Any other suggestions? internal-jgroups-udp.xml ?
>>> dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
>>> ? (joke)
>>> (simply something that users would automatically like to change once
>>> they use it in their app)
>>>
>>> 2) Throw a warning whenever a user wants to use a custom jgroups
>>> configuration file that has the same name as one of the above
>>>
>>>
>>> WDYT?
>>>
>>> Thanks!
>>> Martin
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>


From wfink at redhat.com  Thu Jun 12 09:41:01 2014
From: wfink at redhat.com (Wolf-Dieter Fink)
Date: Thu, 12 Jun 2014 15:41:01 +0200
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <5399AC08.8080406@redhat.com>
References: <53999D6A.9080300@redhat.com>
	<53999F02.3020008@redhat.com>	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
	<5399AC08.8080406@redhat.com>
Message-ID: <5399ADED.2080907@redhat.com>

+1 for a solution like this, I think nobody will use such path for the 
application configuration.

Wolf

On 12/06/14 15:32, Tristan Tarrant wrote:
> Actually my idea is:
>
> I specify I want jgroups-udp.xml
> so first of all I look at the classpath root: if I find the file there I
> use it, otherwise I see if it is in (proposed)
> META-INF/_internal/jgroups-udp.xml
>
> Tristan
>
> On 12/06/14 14:44, Alan Field wrote:
>> Tristan,
>>
>> So the server and library configuration parsers will handle something like this?
>>
>>       <jgroups>
>>           <stack-file name="jgroupsStack" path="config/jgroups-udp.xml"/>
>>       </jgroups>
>>
>> If this is true, then I agree that this is a good solution as well.
>>
>> Thanks,
>> Alan
>>
>> ----- Original Message -----
>>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>>> Sent: Thursday, June 12, 2014 2:37:22 PM
>>> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
>>>
>>> I think the "internal" jgroups files should be "moved" to a separate
>>> directory within the core jar, to be searched after the "root". So the
>>> user can still provide a jgroups-udp.xml and it won't conflict.
>>>
>>> Tristan
>>>
>>> On 12/06/14 14:30, Martin Gencur wrote:
>>>> Hi,
>>>> let me mention an issue that several people faced in the past,
>>>> independently of each other:
>>>>
>>>> A user app uses a custom JGroups configuration file. However, they
>>>> choose the same name as the files which we bundle inside
>>>> infinispan-core.jar.
>>>> Result? People are wondering why their custom configuration does not
>>>> take effect.
>>>> Reason? Infinispan uses the default jgroups file bundled in infinispan-core
>>>> Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
>>>> Alan, Wolf Fink
>>>>
>>>> I believe a lot of users run into this issue.
>>>>
>>>> We were considering a possible solution and this one seems like it could
>>>> work (use both 1) and 2)):
>>>> 1) rename the config files in the distribution e.g. this way:
>>>> jgroups-ec2.xml -> default-jgroups-ec2.xml
>>>> jgroups-udp.xml -> default-jgroups-udp.xml
>>>> jgroups-tcp.xml -> default-jgroups-tcp.xml
>>>>
>>>> Any other suggestions? internal-jgroups-udp.xml ?
>>>> dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
>>>> ? (joke)
>>>> (simply something that users would automatically like to change once
>>>> they use it in their app)
>>>>
>>>> 2) Throw a warning whenever a user wants to use a custom jgroups
>>>> configuration file that has the same name as one of the above
>>>>
>>>>
>>>> WDYT?
>>>>
>>>> Thanks!
>>>> Martin
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From bban at redhat.com  Thu Jun 12 09:46:11 2014
From: bban at redhat.com (Bela Ban)
Date: Thu, 12 Jun 2014 15:46:11 +0200
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <5399AC08.8080406@redhat.com>
References: <53999D6A.9080300@redhat.com>
	<53999F02.3020008@redhat.com>	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
	<5399AC08.8080406@redhat.com>
Message-ID: <5399AF23.7040103@redhat.com>

Does Infinispan actually also support other location formats, such as 
the following ?

- http://www.host.com/configs/jgroups-udp.xml // URLs
- /home/bela/jgroups-udp.xml

On 12/06/14 15:32, Tristan Tarrant wrote:
> Actually my idea is:
>
> I specify I want jgroups-udp.xml
> so first of all I look at the classpath root: if I find the file there I
> use it, otherwise I see if it is in (proposed)
> META-INF/_internal/jgroups-udp.xml
>
> Tristan
>
> On 12/06/14 14:44, Alan Field wrote:
>> Tristan,
>>
>> So the server and library configuration parsers will handle something like this?
>>
>>       <jgroups>
>>           <stack-file name="jgroupsStack" path="config/jgroups-udp.xml"/>
>>       </jgroups>
>>
>> If this is true, then I agree that this is a good solution as well.
>>
>> Thanks,
>> Alan
>>
>> ----- Original Message -----
>>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>>> Sent: Thursday, June 12, 2014 2:37:22 PM
>>> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
>>>
>>> I think the "internal" jgroups files should be "moved" to a separate
>>> directory within the core jar, to be searched after the "root". So the
>>> user can still provide a jgroups-udp.xml and it won't conflict.
>>>
>>> Tristan
>>>
>>> On 12/06/14 14:30, Martin Gencur wrote:
>>>> Hi,
>>>> let me mention an issue that several people faced in the past,
>>>> independently of each other:
>>>>
>>>> A user app uses a custom JGroups configuration file. However, they
>>>> choose the same name as the files which we bundle inside
>>>> infinispan-core.jar.
>>>> Result? People are wondering why their custom configuration does not
>>>> take effect.
>>>> Reason? Infinispan uses the default jgroups file bundled in infinispan-core
>>>> Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
>>>> Alan, Wolf Fink
>>>>
>>>> I believe a lot of users run into this issue.
>>>>
>>>> We were considering a possible solution and this one seems like it could
>>>> work (use both 1) and 2)):
>>>> 1) rename the config files in the distribution e.g. this way:
>>>> jgroups-ec2.xml -> default-jgroups-ec2.xml
>>>> jgroups-udp.xml -> default-jgroups-udp.xml
>>>> jgroups-tcp.xml -> default-jgroups-tcp.xml
>>>>
>>>> Any other suggestions? internal-jgroups-udp.xml ?
>>>> dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
>>>> ? (joke)
>>>> (simply something that users would automatically like to change once
>>>> they use it in their app)
>>>>
>>>> 2) Throw a warning whenever a user wants to use a custom jgroups
>>>> configuration file that has the same name as one of the above
>>>>
>>>>
>>>> WDYT?
>>>>
>>>> Thanks!
>>>> Martin
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)

From galder at redhat.com  Thu Jun 12 09:54:17 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Thu, 12 Jun 2014 15:54:17 +0200
Subject: [infinispan-dev] Reliability of return values
In-Reply-To: <B999DCB7-3F63-42D8-97F2-7E053E187EA7@redhat.com>
References: <53707669.7070701@redhat.com>
	<CA+nfvwT66bci7eY2hqBcAgtba7tO7ov_dBt7t5ypk026qEwvAQ@mail.gmail.com>
	<CAFm4XO0-d_sQPgNRJxDnVb56ADvu00UEn9qj=fKD88+fj5BRwQ@mail.gmail.com>
	<5370A873.8070509@redhat.com>
	<CA+nfvwQDkzbZ7fuoLX80xtx0KgspGZVWoqD+vo=W5iFRSCcKEg@mail.gmail.com>
	<53723CD4.9060300@redhat.com>
	<CA+nfvwTKCSf1fw=SDJGswzZu29Lcdv0j6jdpFEwLgECj0aFpdg@mail.gmail.com>
	<B999DCB7-3F63-42D8-97F2-7E053E187EA7@redhat.com>
Message-ID: <2A1770A0-9884-461D-B288-14B2F18A96E8@redhat.com>

Hi all,

I?m working on the implementation of this, and the solution noted in the JIRA does not work for situations where you have to return a previous value that might have been overriden due to partial operation application. Example (assuming 1 owner only):

1. remote client does a put(?key?, 1) in Node-A
2. remote client does a replace(?key?, 2) on Node-A but the operation fails to replicate and gets partially applieed in Node-A only.
3. remote client, seeing the replace failed, retries the replace(?key?, 2) in Node-B. replace underneath finds that the previous value of ?key? is 2 (since it got partially applied in Node-A), so it succeeds but the previous value returned is 2, which is wrong. The previous value should have been 1, but this value is gone?

In my view, there is two ways to solve this issue:

1. Make Hot Rod caches transactional. By doign so, operations are not partially applied. They?re done fully cluster wide or they?re rolled back. I?ve verified this and the test passes once you make the cache transactional. The downside is of course performance. IOW, anyone using conditional operations, or relying on previous values, would need transactional caches. This should work well with the retry mechanism being implemented for ISPN-2956, which is still needed. A big question here is whether Hot Rod caches should be transactional by default or viceversa. If they?re transactional, our performance will go down for sure but we won?t have this type of issues (with retry in place). If you?re not transactional, you are faster but you?re exposed to these edge cases, and you need to consider them when deploying your app, something people might miss, although we could provide WARN messages when conditional operations or Flag.FORCE_RETURN_VALUE is used with non-transactional caches.

2. Get rid of returning previous value in the Hot Rod protocol for modifying operations. For conditional operations, returning true/false is at least enough to see if the condition was applied. So, replaceIfUnmodified/replace/remove(conditional), they would only return true/false. This would be complicated due to reliance on Map/ConcurrentMap APIs. Maybe something to consider for when we stop relying on JDK APIs.

I also considered applying corrective actions but that?s very messy and prone to concurrency issues, so I quickly discarded that.

Any other options? Thoughts on the options above?

Cheers,

On 26 May 2014, at 18:11, Galder Zamarre?o <galder at redhat.com> wrote:

> Hi all,
> 
> I?ve been looking into ISPN-2956 last week and I think we have a solution for it which requires a protocol change [1]
> 
> Since we?re in the middle of the Hot Rod 2.0 development, this is a good opportunity to implement it.
> 
> Cheers,
> 
> [1] https://issues.jboss.org/browse/ISPN-2956?focusedCommentId=12970541&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12970541
> 
> On 14 May 2014, at 09:36, Dan Berindei <dan.berindei at gmail.com> wrote:
> 
>> 
>> 
>> 
>> On Tue, May 13, 2014 at 6:40 PM, Radim Vansa <rvansa at redhat.com> wrote:
>> On 05/13/2014 03:58 PM, Dan Berindei wrote:
>>> 
>>> 
>>> On Mon, May 12, 2014 at 1:54 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>> @Dan: It's absolutely correct to do the further writes in order to make
>>> the cache consistent, I am not arguing against that. You've fixed the
>>> outcome (state of cache) well. My point was that we should let the user
>>> know that the value he gets is not 100% correct when we already know
>>> that - and given the API, the only option to do that seems to me as
>>> throwing an exception.
>>> 
>>> The problem, as I see it, is that users also expect methods that throw an exception to *not* modify the cache.
>>> So we would break some of the users' expectations anyway.
>> 
>> When the response from primary owner does not arrive soon, we throw timeout exception and the cache is modified anyway, isn't it?
>> If we throw ~ReturnValueUnreliableException, the user has at least some chance to react. Currently, for code requiring 100% reliable value, you can't do anything but ignore the return value, even for CAS operations.
>> 
>> 
>> Yes, but we don't expect the user to handle a TimeoutException in any meaningful way. Instead, we expect the user to choose his hardware and configuration to avoid timeouts, if he cares about consistency. How could you handle an exception that tells you "I may have written the value you asked me to in the cache, or maybe not. Either way, you will never know what the previous value was. Muahahaha!" in an application that cares about consistency?
>> 
>> But the proposed ReturnValueUnreliableException can't be avoided by the user, it has to be handled every time the cluster membership changes. So it would be more like WriteSkewException than TimeoutException. And when we throw a WriteSkewException, we don't write anything to the cache. 
>> 
>> Remember, most users do not care about the previous value at all - that's the reason why JCache and our HotRod client don't return the previous value by default. Those that do care about the previous value, use the conditional write operations, and those already work (well, except for the scenario below). So you would force everyone to handle an exception that they don't care about.
>> 
>> It would make sense to throw an exception if we didn't return the previous value by default, and the user requested the return value explicitly. But we do return the value by default, so I don't think it would be a good idea for us.
>> 
>>> 
>>> 
>>> @Sanne: I was not suggesting that for now - sure, value versioning is (I
>>> hope) on the roadmap. But that's more complicated, I though just about
>>> making an adjustment to the current implementation.
>>> 
>>> 
>>> Actually, just keeping a history of values would not fix the the return value in all cases.
>>> 
>>> When retrying a put on the new primary owner, the primary owner would still have to compare our value with the latest value, and return the previous value if they are equal. So we could have something like this:
>>> 
>>> A is the originator, B is the primary owner, k = v0
>>> A -> B: put(k, v1)
>>> B dies before writing v, C is now primary owner
>>> D -> C: put(k, v1) // another put operation from D, with the same value
>>> C -> D: null
>>> A -> C: retry_put(k, v1)
>>> C -> A: v0 // C assumes A is overwriting its own value, so it's returning the previous one
>>> 
>>> To fix that, we'd need a unique version generated by the originator - kind of like a transaction id ;)
>> 
>> Is it such a problem to associate unique ID with each write? History implementation seems to me like the more complicated part.
>> 
>> I also think maintaining a version history would be quite complicated, and it also would make it harder for users to estimate their cache's memory usage. That's why I was trying to show that it's not a panacea.
>> 
>> 
>> 
>>> And to fix the HotRod use case, the HotRod client would have to be the one generating the version.
>> 
>> I agree.
>> 
>> Radim
>> 
>> 
>>> 
>>> Cheers
>>> Dan
>>> 
>>> 
>>> 
>>> Radim
>>> 
>>> On 05/12/2014 12:02 PM, Sanne Grinovero wrote:
>>>> I don't think we are in a position to decide what is a reasonable
>>>> compromise; we can do better.
>>>> For example - as Radim suggested - it might seem reasonable to have
>>>> the older value around for a little while. We'll need a little bit of
>>>> history of values and tombstones anyway for many other reasons.
>>>> 
>>>> 
>>>> Sanne
>>>> 
>>>> On 12 May 2014 09:37, Dan Berindei <dan.berindei at gmail.com> wrote:
>>>>> Radim, I would contend that the first and foremost guarantee that put()
>>>>> makes is to leave the cache in a consistent state. So we can't just throw an
>>>>> exception and give up, leaving k=v on one owner and k=null on another.
>>>>> 
>>>>> Secondly, put(k, v) being atomic means that it either succeeds, it writes
>>>>> k=v in the cache, and it returns the previous value, or it doesn't succeed,
>>>>> and it doesn't write k=v in the cache. Returning the wrong previous value is
>>>>> bad, but leaving k=v in the cache is just as bad, even if the all the owners
>>>>> have the same value.
>>>>> 
>>>>> And last, we can't have one node seeing k=null, then k=v, then k=null again,
>>>>> when the only write we did on the cache was a put(k, v). So trying to undo
>>>>> the write would not help.
>>>>> 
>>>>> In the end, we have to make a compromise, and I think returning the wrong
>>>>> value in some of the cases is a reasonable compromise. Of course, we should
>>>>> document that :)
>>>>> 
>>>>> I also believe ISPN-2956 could be fixed so that HotRod behaves just like
>>>>> embedded mode after the ISPN-3422 fix, by adding a RETRY flag to the HotRod
>>>>> protocol and to the cache itself.
>>>>> 
>>>>> Incidentally, transactional caches have a similar problem when the
>>>>> originator leaves the cluster: ISPN-3421 [1]
>>>>> And we can't handle transactional caches any better than non-transactional
>>>>> caches until we expose transactions to the HotRod client.
>>>>> 
>>>>> [1] https://issues.jboss.org/browse/ISPN-2956
>>>>> 
>>>>> Cheers
>>>>> Dan
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On Mon, May 12, 2014 at 10:21 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> recently I've stumbled upon one already expected behaviour (one instance
>>>>>> is [1]), but which did not got much attention.
>>>>>> 
>>>>>> In non-tx cache, when the primary owner fails after the request has been
>>>>>> replicated to backup owner, the request is retried in the new topology.
>>>>>> Then, the operation is executed on the new primary (the previous
>>>>>> backup). The outcome has been already fixed in [2], but the return value
>>>>>> may be wrong. For example, when we do a put, the return value for the
>>>>>> second attempt will be the currently inserted value (although the entry
>>>>>> was just created). Same situation may happen for other operations.
>>>>>> 
>>>>>> Currently, it's not possible to return the correct value (because it has
>>>>>> already been overwritten and we don't keep a history of values), but
>>>>>> shouldn't we rather throw an exception if we were not able to fulfil the
>>>>>> API contract?
>>>>>> 
>>>>>> Radim
>>>>>> 
>>>>>> [1] https://issues.jboss.org/browse/ISPN-2956
>>>>>> [2] https://issues.jboss.org/browse/ISPN-3422
>>>>>> 
>>>>>> --
>>>>>> Radim Vansa <rvansa at redhat.com>
>>>>>> JBoss DataGrid QA
>>>>>> 
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> 
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss DataGrid QA
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> 
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> -- 
>> Radim Vansa 
>> <rvansa at redhat.com>
>> 
>> JBoss DataGrid QA
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From sanne at infinispan.org  Thu Jun 12 09:57:12 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 12 Jun 2014 14:57:12 +0100
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <5399AF23.7040103@redhat.com>
References: <53999D6A.9080300@redhat.com> <53999F02.3020008@redhat.com>
	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
	<5399AC08.8080406@redhat.com> <5399AF23.7040103@redhat.com>
Message-ID: <CAFm4XO2nfTEnFmG6vHzbOc3R8Kdh34wYhGZEV_B4=6iTgwU0JQ@mail.gmail.com>

+1 for Tristan's approach to look in the modules in a specific (and
correct) order: Infinispan core jar is last.

But you don't need to rename it.

BTW this relates to the thread I opened a while back pointing out that
the configuration parser needs to expose on the API the classloader I
want it to load configuration files from, which is not necessarily the
same as the module where the Parser needs to load its own extensions
from.


On 12 June 2014 14:46, Bela Ban <bban at redhat.com> wrote:
> Does Infinispan actually also support other location formats, such as
> the following ?
>
> - http://www.host.com/configs/jgroups-udp.xml // URLs
> - /home/bela/jgroups-udp.xml
>
> On 12/06/14 15:32, Tristan Tarrant wrote:
>> Actually my idea is:
>>
>> I specify I want jgroups-udp.xml
>> so first of all I look at the classpath root: if I find the file there I
>> use it, otherwise I see if it is in (proposed)
>> META-INF/_internal/jgroups-udp.xml
>>
>> Tristan
>>
>> On 12/06/14 14:44, Alan Field wrote:
>>> Tristan,
>>>
>>> So the server and library configuration parsers will handle something like this?
>>>
>>>       <jgroups>
>>>           <stack-file name="jgroupsStack" path="config/jgroups-udp.xml"/>
>>>       </jgroups>
>>>
>>> If this is true, then I agree that this is a good solution as well.
>>>
>>> Thanks,
>>> Alan
>>>
>>> ----- Original Message -----
>>>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>>>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>>>> Sent: Thursday, June 12, 2014 2:37:22 PM
>>>> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
>>>>
>>>> I think the "internal" jgroups files should be "moved" to a separate
>>>> directory within the core jar, to be searched after the "root". So the
>>>> user can still provide a jgroups-udp.xml and it won't conflict.
>>>>
>>>> Tristan
>>>>
>>>> On 12/06/14 14:30, Martin Gencur wrote:
>>>>> Hi,
>>>>> let me mention an issue that several people faced in the past,
>>>>> independently of each other:
>>>>>
>>>>> A user app uses a custom JGroups configuration file. However, they
>>>>> choose the same name as the files which we bundle inside
>>>>> infinispan-core.jar.
>>>>> Result? People are wondering why their custom configuration does not
>>>>> take effect.
>>>>> Reason? Infinispan uses the default jgroups file bundled in infinispan-core
>>>>> Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
>>>>> Alan, Wolf Fink
>>>>>
>>>>> I believe a lot of users run into this issue.
>>>>>
>>>>> We were considering a possible solution and this one seems like it could
>>>>> work (use both 1) and 2)):
>>>>> 1) rename the config files in the distribution e.g. this way:
>>>>> jgroups-ec2.xml -> default-jgroups-ec2.xml
>>>>> jgroups-udp.xml -> default-jgroups-udp.xml
>>>>> jgroups-tcp.xml -> default-jgroups-tcp.xml
>>>>>
>>>>> Any other suggestions? internal-jgroups-udp.xml ?
>>>>> dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
>>>>> ? (joke)
>>>>> (simply something that users would automatically like to change once
>>>>> they use it in their app)
>>>>>
>>>>> 2) Throw a warning whenever a user wants to use a custom jgroups
>>>>> configuration file that has the same name as one of the above
>>>>>
>>>>>
>>>>> WDYT?
>>>>>
>>>>> Thanks!
>>>>> Martin
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From dereed at redhat.com  Thu Jun 12 12:46:54 2014
From: dereed at redhat.com (Dennis Reed)
Date: Thu, 12 Jun 2014 11:46:54 -0500
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
References: <53999D6A.9080300@redhat.com> <53999F02.3020008@redhat.com>
	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
Message-ID: <5399D97E.3000003@redhat.com>

+1 to changing the name/directory.
-100 to changing the order of where it's looked for instead.

All resource lookups should use the normal rules for finding resources.
Don't change standard behavior without a *very* good reason.

Doing anything special (like META-INF/_internal/jgroups-udp.xml) is 
completely non-intuitive
and will cause support issues down the road.
Using config/jgroups-udp.xml is standard, and would be immediately 
understood by anyone.

-Dennis

On 06/12/2014 07:44 AM, Alan Field wrote:
> Tristan,
>
> So the server and library configuration parsers will handle something like this?
>
>      <jgroups>
>          <stack-file name="jgroupsStack" path="config/jgroups-udp.xml"/>
>      </jgroups>
>
> If this is true, then I agree that this is a good solution as well.
>
> Thanks,
> Alan
>
> ----- Original Message -----
>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>> Sent: Thursday, June 12, 2014 2:37:22 PM
>> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
>>
>> I think the "internal" jgroups files should be "moved" to a separate
>> directory within the core jar, to be searched after the "root". So the
>> user can still provide a jgroups-udp.xml and it won't conflict.
>>
>> Tristan
>>
>> On 12/06/14 14:30, Martin Gencur wrote:
>>> Hi,
>>> let me mention an issue that several people faced in the past,
>>> independently of each other:
>>>
>>> A user app uses a custom JGroups configuration file. However, they
>>> choose the same name as the files which we bundle inside
>>> infinispan-core.jar.
>>> Result? People are wondering why their custom configuration does not
>>> take effect.
>>> Reason? Infinispan uses the default jgroups file bundled in infinispan-core
>>> Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
>>> Alan, Wolf Fink
>>>
>>> I believe a lot of users run into this issue.
>>>
>>> We were considering a possible solution and this one seems like it could
>>> work (use both 1) and 2)):
>>> 1) rename the config files in the distribution e.g. this way:
>>> jgroups-ec2.xml -> default-jgroups-ec2.xml
>>> jgroups-udp.xml -> default-jgroups-udp.xml
>>> jgroups-tcp.xml -> default-jgroups-tcp.xml
>>>
>>> Any other suggestions? internal-jgroups-udp.xml ?
>>> dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
>>> ? (joke)
>>> (simply something that users would automatically like to change once
>>> they use it in their app)
>>>
>>> 2) Throw a warning whenever a user wants to use a custom jgroups
>>> configuration file that has the same name as one of the above
>>>
>>>
>>> WDYT?
>>>
>>> Thanks!
>>> Martin
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From ttarrant at redhat.com  Thu Jun 12 14:14:26 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Thu, 12 Jun 2014 20:14:26 +0200
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <5399D97E.3000003@redhat.com>
References: <53999D6A.9080300@redhat.com>
	<53999F02.3020008@redhat.com>	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
	<5399D97E.3000003@redhat.com>
Message-ID: <5399EE02.3030503@redhat.com>

On 12/06/14 18:46, Dennis Reed wrote:
> +1 to changing the name/directory.
> -100 to changing the order of where it's looked for instead.
>
> All resource lookups should use the normal rules for finding resources.
> Don't change standard behavior without a *very* good reason.
>
> Doing anything special (like META-INF/_internal/jgroups-udp.xml) is
> completely non-intuitive
> and will cause support issues down the road.
> Using config/jgroups-udp.xml is standard, and would be immediately
> understood by anyone.
>
Users don't even need to know that META-INF/_internal/blah actually 
exists. It is just an internal detail when using the "default" (i.e. 
just enable clustering without explicitly specifying a configuration file).

Tristan

From sanne at infinispan.org  Thu Jun 12 19:44:08 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 13 Jun 2014 00:44:08 +0100
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <5399D97E.3000003@redhat.com>
References: <53999D6A.9080300@redhat.com> <53999F02.3020008@redhat.com>
	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
	<5399D97E.3000003@redhat.com>
Message-ID: <CAFm4XO1Egej_tSeeCCw=rNDDJ2or94CEDFDdntL=YO-haUgD+w@mail.gmail.com>

On 12 June 2014 17:46, Dennis Reed <dereed at redhat.com> wrote:
> +1 to changing the name/directory.
> -100 to changing the order of where it's looked for instead.
>
> All resource lookups should use the normal rules for finding resources.
> Don't change standard behavior without a *very* good reason.

There isn't a "standard behaviour", the problem is exactly that it's
not defined.
By actually specifying an order we can decide that a user
configuration file will take priority over our own file, at least
assuming a modular classloader is being used.
So indeed it's not a solution for flat classloaders, but it's at least
correct in that case.

This is what we do in other frameworks: the order in which you look
for resources needs to be well defined, but it's not in Infinispan.

>
> Doing anything special (like META-INF/_internal/jgroups-udp.xml) is
> completely non-intuitive
> and will cause support issues down the road.
> Using config/jgroups-udp.xml is standard, and would be immediately
> understood by anyone.

I would agree with you about this being a common expectation, but it's
not rock-solid; we still can't state for sure that the user won't use
"config" directory too..

The only foolproof solution is to store some signature of the
jgroups-udp.xml hardcoded in a class, or simply hardcode the whole
file as a constant.
We could use the maven inject plugin to maintain the xml file as
normal source code, and "seal" some constant at build time.. but I
don't think it's worth it as by defining the lookup order the problem
is solved in JBoss or WildFly as we would know that the
infinispan-core.jar classloader is different than the user one, unless
people bundle the Infinispan jars in their app.

Considering JGroups already logs the full configuration file that it's
applying, maybe it could log some hash signature as well? This way you
could just compare the signature to make sure it's using some "well
known" configuration file.

Sanne


>
> -Dennis
>
> On 06/12/2014 07:44 AM, Alan Field wrote:
>> Tristan,
>>
>> So the server and library configuration parsers will handle something like this?
>>
>>      <jgroups>
>>          <stack-file name="jgroupsStack" path="config/jgroups-udp.xml"/>
>>      </jgroups>
>>
>> If this is true, then I agree that this is a good solution as well.
>>
>> Thanks,
>> Alan
>>
>> ----- Original Message -----
>>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>>> Sent: Thursday, June 12, 2014 2:37:22 PM
>>> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
>>>
>>> I think the "internal" jgroups files should be "moved" to a separate
>>> directory within the core jar, to be searched after the "root". So the
>>> user can still provide a jgroups-udp.xml and it won't conflict.
>>>
>>> Tristan
>>>
>>> On 12/06/14 14:30, Martin Gencur wrote:
>>>> Hi,
>>>> let me mention an issue that several people faced in the past,
>>>> independently of each other:
>>>>
>>>> A user app uses a custom JGroups configuration file. However, they
>>>> choose the same name as the files which we bundle inside
>>>> infinispan-core.jar.
>>>> Result? People are wondering why their custom configuration does not
>>>> take effect.
>>>> Reason? Infinispan uses the default jgroups file bundled in infinispan-core
>>>> Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
>>>> Alan, Wolf Fink
>>>>
>>>> I believe a lot of users run into this issue.
>>>>
>>>> We were considering a possible solution and this one seems like it could
>>>> work (use both 1) and 2)):
>>>> 1) rename the config files in the distribution e.g. this way:
>>>> jgroups-ec2.xml -> default-jgroups-ec2.xml
>>>> jgroups-udp.xml -> default-jgroups-udp.xml
>>>> jgroups-tcp.xml -> default-jgroups-tcp.xml
>>>>
>>>> Any other suggestions? internal-jgroups-udp.xml ?
>>>> dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
>>>> ? (joke)
>>>> (simply something that users would automatically like to change once
>>>> they use it in their app)
>>>>
>>>> 2) Throw a warning whenever a user wants to use a custom jgroups
>>>> configuration file that has the same name as one of the above
>>>>
>>>>
>>>> WDYT?
>>>>
>>>> Thanks!
>>>> Martin
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From mgencur at redhat.com  Fri Jun 13 01:36:10 2014
From: mgencur at redhat.com (Martin Gencur)
Date: Fri, 13 Jun 2014 07:36:10 +0200
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <5399EE02.3030503@redhat.com>
References: <53999D6A.9080300@redhat.com>	<53999F02.3020008@redhat.com>	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>	<5399D97E.3000003@redhat.com>
	<5399EE02.3030503@redhat.com>
Message-ID: <539A8DCA.5020202@redhat.com>

On 12.6.2014 20:14, Tristan Tarrant wrote:
> On 12/06/14 18:46, Dennis Reed wrote:
>> +1 to changing the name/directory.
>> -100 to changing the order of where it's looked for instead.
>>
>> All resource lookups should use the normal rules for finding resources.
>> Don't change standard behavior without a *very* good reason.
>>
>> Doing anything special (like META-INF/_internal/jgroups-udp.xml) is
>> completely non-intuitive
>> and will cause support issues down the road.
>> Using config/jgroups-udp.xml is standard, and would be immediately
>> understood by anyone.
>>
> Users don't even need to know that META-INF/_internal/blah actually
> exists. It is just an internal detail when using the "default" (i.e.
> just enable clustering without explicitly specifying a configuration file).

My understanding was that users just take an example config. file (i.e. 
jgroups-udp.xml), copy it into their application and modify. That's how 
users get the same name for their configuration file as the default. So 
in this case, they might find it again, even in 
META-INF/_internal/jgroups.udp.xml :)

Martin


>
> Tristan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From afield at redhat.com  Fri Jun 13 03:09:47 2014
From: afield at redhat.com (Alan Field)
Date: Fri, 13 Jun 2014 03:09:47 -0400 (EDT)
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <539A8DCA.5020202@redhat.com>
References: <53999D6A.9080300@redhat.com> <53999F02.3020008@redhat.com>
	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>
	<5399D97E.3000003@redhat.com> <5399EE02.3030503@redhat.com>
	<539A8DCA.5020202@redhat.com>
Message-ID: <2017472995.20619817.1402643387052.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "Martin Gencur" <mgencur at redhat.com>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Friday, June 13, 2014 7:36:10 AM
> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
> 
> On 12.6.2014 20:14, Tristan Tarrant wrote:
> > On 12/06/14 18:46, Dennis Reed wrote:
> >> +1 to changing the name/directory.
> >> -100 to changing the order of where it's looked for instead.
> >>
> >> All resource lookups should use the normal rules for finding resources.
> >> Don't change standard behavior without a *very* good reason.
> >>
> >> Doing anything special (like META-INF/_internal/jgroups-udp.xml) is
> >> completely non-intuitive
> >> and will cause support issues down the road.
> >> Using config/jgroups-udp.xml is standard, and would be immediately
> >> understood by anyone.
> >>
> > Users don't even need to know that META-INF/_internal/blah actually
> > exists. It is just an internal detail when using the "default" (i.e.
> > just enable clustering without explicitly specifying a configuration file).
> 
> My understanding was that users just take an example config. file (i.e.
> jgroups-udp.xml), copy it into their application and modify. That's how
> users get the same name for their configuration file as the default. So
> in this case, they might find it again, even in
> META-INF/_internal/jgroups.udp.xml :)

I agree that defining the order that configuration files are loaded is important and should be defined. I would prefer a more readable path like "META-INF/example_configurations/jgroups/udp.xml". This also gives us a good location to provide example cache configuration files as well. I also think that a message should be logged with the path to the configuration file being used:

2014-06-12 06:45:02,871 [thread-name] INFO  [package-name] Using JGroups configuration file 'jar:META-INF/example_configurations/jgroups/udp.xml'
2014-06-12 06:45:02,871 [thread-name] INFO  [package-name] Using JGroups configuration file 'file:/app_home/config/jgroups-udp.xml'

This would make it more obvious to the user which configuration file is in use.

Thanks,
Alan


> 
> Martin
> 
> 
> >
> > Tristan
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From ttarrant at redhat.com  Fri Jun 13 04:08:54 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 13 Jun 2014 10:08:54 +0200
Subject: [infinispan-dev] Issue with JGroups config files in ispn-core
In-Reply-To: <CAFm4XO1Egej_tSeeCCw=rNDDJ2or94CEDFDdntL=YO-haUgD+w@mail.gmail.com>
References: <53999D6A.9080300@redhat.com>
	<53999F02.3020008@redhat.com>	<386547548.20027436.1402577043992.JavaMail.zimbra@redhat.com>	<5399D97E.3000003@redhat.com>
	<CAFm4XO1Egej_tSeeCCw=rNDDJ2or94CEDFDdntL=YO-haUgD+w@mail.gmail.com>
Message-ID: <539AB196.5070108@redhat.com>

Let me add:

specifying a configuration file just by name is NOT enough. We also need 
to allow users to specify an InputStream which IMHO is the "only correct 
way" :)

Tristan

On 13/06/14 01:44, Sanne Grinovero wrote:
> On 12 June 2014 17:46, Dennis Reed <dereed at redhat.com> wrote:
>> +1 to changing the name/directory.
>> -100 to changing the order of where it's looked for instead.
>>
>> All resource lookups should use the normal rules for finding resources.
>> Don't change standard behavior without a *very* good reason.
> There isn't a "standard behaviour", the problem is exactly that it's
> not defined.
> By actually specifying an order we can decide that a user
> configuration file will take priority over our own file, at least
> assuming a modular classloader is being used.
> So indeed it's not a solution for flat classloaders, but it's at least
> correct in that case.
>
> This is what we do in other frameworks: the order in which you look
> for resources needs to be well defined, but it's not in Infinispan.
>
>> Doing anything special (like META-INF/_internal/jgroups-udp.xml) is
>> completely non-intuitive
>> and will cause support issues down the road.
>> Using config/jgroups-udp.xml is standard, and would be immediately
>> understood by anyone.
> I would agree with you about this being a common expectation, but it's
> not rock-solid; we still can't state for sure that the user won't use
> "config" directory too..
>
> The only foolproof solution is to store some signature of the
> jgroups-udp.xml hardcoded in a class, or simply hardcode the whole
> file as a constant.
> We could use the maven inject plugin to maintain the xml file as
> normal source code, and "seal" some constant at build time.. but I
> don't think it's worth it as by defining the lookup order the problem
> is solved in JBoss or WildFly as we would know that the
> infinispan-core.jar classloader is different than the user one, unless
> people bundle the Infinispan jars in their app.
>
> Considering JGroups already logs the full configuration file that it's
> applying, maybe it could log some hash signature as well? This way you
> could just compare the signature to make sure it's using some "well
> known" configuration file.
>
> Sanne
>
>
>> -Dennis
>>
>> On 06/12/2014 07:44 AM, Alan Field wrote:
>>> Tristan,
>>>
>>> So the server and library configuration parsers will handle something like this?
>>>
>>>       <jgroups>
>>>           <stack-file name="jgroupsStack" path="config/jgroups-udp.xml"/>
>>>       </jgroups>
>>>
>>> If this is true, then I agree that this is a good solution as well.
>>>
>>> Thanks,
>>> Alan
>>>
>>> ----- Original Message -----
>>>> From: "Tristan Tarrant" <ttarrant at redhat.com>
>>>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>>>> Sent: Thursday, June 12, 2014 2:37:22 PM
>>>> Subject: Re: [infinispan-dev] Issue with JGroups config files in ispn-core
>>>>
>>>> I think the "internal" jgroups files should be "moved" to a separate
>>>> directory within the core jar, to be searched after the "root". So the
>>>> user can still provide a jgroups-udp.xml and it won't conflict.
>>>>
>>>> Tristan
>>>>
>>>> On 12/06/14 14:30, Martin Gencur wrote:
>>>>> Hi,
>>>>> let me mention an issue that several people faced in the past,
>>>>> independently of each other:
>>>>>
>>>>> A user app uses a custom JGroups configuration file. However, they
>>>>> choose the same name as the files which we bundle inside
>>>>> infinispan-core.jar.
>>>>> Result? People are wondering why their custom configuration does not
>>>>> take effect.
>>>>> Reason? Infinispan uses the default jgroups file bundled in infinispan-core
>>>>> Who faced the issue? (I suppose it's just a small subset:)) Me, Radim,
>>>>> Alan, Wolf Fink
>>>>>
>>>>> I believe a lot of users run into this issue.
>>>>>
>>>>> We were considering a possible solution and this one seems like it could
>>>>> work (use both 1) and 2)):
>>>>> 1) rename the config files in the distribution e.g. this way:
>>>>> jgroups-ec2.xml -> default-jgroups-ec2.xml
>>>>> jgroups-udp.xml -> default-jgroups-udp.xml
>>>>> jgroups-tcp.xml -> default-jgroups-tcp.xml
>>>>>
>>>>> Any other suggestions? internal-jgroups-udp.xml ?
>>>>> dontEverUseThisFileInYourAppAsTheCustomConfigurationFile-jgroups-udp.xml
>>>>> ? (joke)
>>>>> (simply something that users would automatically like to change once
>>>>> they use it in their app)
>>>>>
>>>>> 2) Throw a warning whenever a user wants to use a custom jgroups
>>>>> configuration file that has the same name as one of the above
>>>>>
>>>>>
>>>>> WDYT?
>>>>>
>>>>> Thanks!
>>>>> Martin
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>


From mmarkus at redhat.com  Fri Jun 13 09:52:57 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Fri, 13 Jun 2014 14:52:57 +0100
Subject: [infinispan-dev] stabilize test suite
Message-ID: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>

Hi,

The test suite is allover, so please STOP integrating into master till the suite gets green. 
Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From mudokonman at gmail.com  Fri Jun 13 10:45:42 2014
From: mudokonman at gmail.com (William Burns)
Date: Fri, 13 Jun 2014 10:45:42 -0400
Subject: [infinispan-dev] stabilize test suite
In-Reply-To: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>
References: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>
Message-ID: <CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>

I am currently looking at https://issues.jboss.org/browse/ISPN-4389
which involves the StateTransferSuppressFor* tests.

Also after that I was planning on getting to
https://issues.jboss.org/browse/ISPN-4384 which is a test that fails
spuriously CacheNotifierInitialTransferDistTest

On Fri, Jun 13, 2014 at 9:52 AM, Mircea Markus <mmarkus at redhat.com> wrote:
> Hi,
>
> The test suite is allover, so please STOP integrating into master till the suite gets green.
> Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From ttarrant at redhat.com  Fri Jun 13 11:15:18 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 13 Jun 2014 17:15:18 +0200
Subject: [infinispan-dev] stabilize test suite
In-Reply-To: <CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>
References: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>
	<CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>
Message-ID: <539B1586.4070506@redhat.com>

I'm on https://issues.jboss.org/browse/ISPN-4403

And then I'll look into why arquillian doesn't detect the hotrod server

Tristan

On 13/06/14 16:45, William Burns wrote:
> I am currently looking at https://issues.jboss.org/browse/ISPN-4389
> which involves the StateTransferSuppressFor* tests.
>
> Also after that I was planning on getting to
> https://issues.jboss.org/browse/ISPN-4384 which is a test that fails
> spuriously CacheNotifierInitialTransferDistTest
>
> On Fri, Jun 13, 2014 at 9:52 AM, Mircea Markus <mmarkus at redhat.com> wrote:
>> Hi,
>>
>> The test suite is allover, so please STOP integrating into master till the suite gets green.
>> Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.
>>
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>


From rhauch at redhat.com  Fri Jun 13 11:42:19 2014
From: rhauch at redhat.com (Randall Hauch)
Date: Fri, 13 Jun 2014 10:42:19 -0500
Subject: [infinispan-dev] stabilize test suite
In-Reply-To: <539B1586.4070506@redhat.com>
References: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>
	<CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>
	<539B1586.4070506@redhat.com>
Message-ID: <EB28C000-23CD-42A5-A3FF-3E2C3D6B7DC3@redhat.com>

This may not at all be related, but maybe it might save some time. One problem ModeShape has run into is that Arquillian seems to only detect a Wildfly server when the test is run with IPv4.

On Jun 13, 2014, at 10:15 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:

> I'm on https://issues.jboss.org/browse/ISPN-4403
> 
> And then I'll look into why arquillian doesn't detect the hotrod server
> 
> Tristan
> 
> On 13/06/14 16:45, William Burns wrote:
>> I am currently looking at https://issues.jboss.org/browse/ISPN-4389
>> which involves the StateTransferSuppressFor* tests.
>> 
>> Also after that I was planning on getting to
>> https://issues.jboss.org/browse/ISPN-4384 which is a test that fails
>> spuriously CacheNotifierInitialTransferDistTest
>> 
>> On Fri, Jun 13, 2014 at 9:52 AM, Mircea Markus <mmarkus at redhat.com> wrote:
>>> Hi,
>>> 
>>> The test suite is allover, so please STOP integrating into master till the suite gets green.
>>> Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.
>>> 
>>> Cheers,
>>> --
>>> Mircea Markus
>>> Infinispan lead (www.infinispan.org)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From galder at redhat.com  Mon Jun 16 03:47:52 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Mon, 16 Jun 2014 09:47:52 +0200
Subject: [infinispan-dev] stabilize test suite
In-Reply-To: <EB28C000-23CD-42A5-A3FF-3E2C3D6B7DC3@redhat.com>
References: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>
	<CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>
	<539B1586.4070506@redhat.com>
	<EB28C000-23CD-42A5-A3FF-3E2C3D6B7DC3@redhat.com>
Message-ID: <B15B76BF-006A-4132-82ED-AA6F26755D2C@redhat.com>

org.infinispan.security.QueryAuthorizationTest is failing to start properly, as a result of:

java.lang.NoClassDefFoundError: org/apache/lucene/search/similarities/Similarity

I?ve created https://issues.jboss.org/browse/ISPN-4405 and I?m looking into it.

Cheers,

On 13 Jun 2014, at 17:42, Randall Hauch <rhauch at redhat.com> wrote:

> This may not at all be related, but maybe it might save some time. One problem ModeShape has run into is that Arquillian seems to only detect a Wildfly server when the test is run with IPv4.
> 
> On Jun 13, 2014, at 10:15 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:
> 
>> I'm on https://issues.jboss.org/browse/ISPN-4403
>> 
>> And then I'll look into why arquillian doesn't detect the hotrod server
>> 
>> Tristan
>> 
>> On 13/06/14 16:45, William Burns wrote:
>>> I am currently looking at https://issues.jboss.org/browse/ISPN-4389
>>> which involves the StateTransferSuppressFor* tests.
>>> 
>>> Also after that I was planning on getting to
>>> https://issues.jboss.org/browse/ISPN-4384 which is a test that fails
>>> spuriously CacheNotifierInitialTransferDistTest
>>> 
>>> On Fri, Jun 13, 2014 at 9:52 AM, Mircea Markus <mmarkus at redhat.com> wrote:
>>>> Hi,
>>>> 
>>>> The test suite is allover, so please STOP integrating into master till the suite gets green.
>>>> Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.
>>>> 
>>>> Cheers,
>>>> --
>>>> Mircea Markus
>>>> Infinispan lead (www.infinispan.org)
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From galder at redhat.com  Mon Jun 16 05:17:23 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Mon, 16 Jun 2014 11:17:23 +0200
Subject: [infinispan-dev] stabilize test suite
In-Reply-To: <B15B76BF-006A-4132-82ED-AA6F26755D2C@redhat.com>
References: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>
	<CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>
	<539B1586.4070506@redhat.com>
	<EB28C000-23CD-42A5-A3FF-3E2C3D6B7DC3@redhat.com>
	<B15B76BF-006A-4132-82ED-AA6F26755D2C@redhat.com>
Message-ID: <06AEBE40-0F0E-4F4C-AE78-CBCEDE9A9166@redhat.com>

I?ve fixed that and also fixed https://issues.jboss.org/browse/ISPN-4406 too, both of which focused on the instability in integrationtests/security-manager-it/

PR for both JIRAs: https://github.com/infinispan/infinispan/pull/2642

Onto the next lot of failing tests...

Cheers,
 
On 16 Jun 2014, at 09:47, Galder Zamarre?o <galder at redhat.com> wrote:

> org.infinispan.security.QueryAuthorizationTest is failing to start properly, as a result of:
> 
> java.lang.NoClassDefFoundError: org/apache/lucene/search/similarities/Similarity
> 
> I?ve created https://issues.jboss.org/browse/ISPN-4405 and I?m looking into it.
> 
> Cheers,
> 
> On 13 Jun 2014, at 17:42, Randall Hauch <rhauch at redhat.com> wrote:
> 
>> This may not at all be related, but maybe it might save some time. One problem ModeShape has run into is that Arquillian seems to only detect a Wildfly server when the test is run with IPv4.
>> 
>> On Jun 13, 2014, at 10:15 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:
>> 
>>> I'm on https://issues.jboss.org/browse/ISPN-4403
>>> 
>>> And then I'll look into why arquillian doesn't detect the hotrod server
>>> 
>>> Tristan
>>> 
>>> On 13/06/14 16:45, William Burns wrote:
>>>> I am currently looking at https://issues.jboss.org/browse/ISPN-4389
>>>> which involves the StateTransferSuppressFor* tests.
>>>> 
>>>> Also after that I was planning on getting to
>>>> https://issues.jboss.org/browse/ISPN-4384 which is a test that fails
>>>> spuriously CacheNotifierInitialTransferDistTest
>>>> 
>>>> On Fri, Jun 13, 2014 at 9:52 AM, Mircea Markus <mmarkus at redhat.com> wrote:
>>>>> Hi,
>>>>> 
>>>>> The test suite is allover, so please STOP integrating into master till the suite gets green.
>>>>> Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.
>>>>> 
>>>>> Cheers,
>>>>> --
>>>>> Mircea Markus
>>>>> Infinispan lead (www.infinispan.org)
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From galder at redhat.com  Mon Jun 16 05:24:30 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Mon, 16 Jun 2014 11:24:30 +0200
Subject: [infinispan-dev] stabilize test suite
In-Reply-To: <06AEBE40-0F0E-4F4C-AE78-CBCEDE9A9166@redhat.com>
References: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>
	<CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>
	<539B1586.4070506@redhat.com>
	<EB28C000-23CD-42A5-A3FF-3E2C3D6B7DC3@redhat.com>
	<B15B76BF-006A-4132-82ED-AA6F26755D2C@redhat.com>
	<06AEBE40-0F0E-4F4C-AE78-CBCEDE9A9166@redhat.com>
Message-ID: <C7EC99C8-57AF-4D7F-BF28-5ECC7547BB30@redhat.com>

Next up, I?m checking what?s up with:

org.infinispan.server.test.cs.leveldb.* failures:

LevelDBCacheStoreIT.testDataRetrievableViaLevelDbApi	
LevelDBCacheStoreIT.testDataSurvivesRestart

Cheers,

On 16 Jun 2014, at 11:17, Galder Zamarre?o <galder at redhat.com> wrote:

> I?ve fixed that and also fixed https://issues.jboss.org/browse/ISPN-4406 too, both of which focused on the instability in integrationtests/security-manager-it/
> 
> PR for both JIRAs: https://github.com/infinispan/infinispan/pull/2642
> 
> Onto the next lot of failing tests...
> 
> Cheers,
> 
> On 16 Jun 2014, at 09:47, Galder Zamarre?o <galder at redhat.com> wrote:
> 
>> org.infinispan.security.QueryAuthorizationTest is failing to start properly, as a result of:
>> 
>> java.lang.NoClassDefFoundError: org/apache/lucene/search/similarities/Similarity
>> 
>> I?ve created https://issues.jboss.org/browse/ISPN-4405 and I?m looking into it.
>> 
>> Cheers,
>> 
>> On 13 Jun 2014, at 17:42, Randall Hauch <rhauch at redhat.com> wrote:
>> 
>>> This may not at all be related, but maybe it might save some time. One problem ModeShape has run into is that Arquillian seems to only detect a Wildfly server when the test is run with IPv4.
>>> 
>>> On Jun 13, 2014, at 10:15 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>> 
>>>> I'm on https://issues.jboss.org/browse/ISPN-4403
>>>> 
>>>> And then I'll look into why arquillian doesn't detect the hotrod server
>>>> 
>>>> Tristan
>>>> 
>>>> On 13/06/14 16:45, William Burns wrote:
>>>>> I am currently looking at https://issues.jboss.org/browse/ISPN-4389
>>>>> which involves the StateTransferSuppressFor* tests.
>>>>> 
>>>>> Also after that I was planning on getting to
>>>>> https://issues.jboss.org/browse/ISPN-4384 which is a test that fails
>>>>> spuriously CacheNotifierInitialTransferDistTest
>>>>> 
>>>>> On Fri, Jun 13, 2014 at 9:52 AM, Mircea Markus <mmarkus at redhat.com> wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> The test suite is allover, so please STOP integrating into master till the suite gets green.
>>>>>> Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.
>>>>>> 
>>>>>> Cheers,
>>>>>> --
>>>>>> Mircea Markus
>>>>>> Infinispan lead (www.infinispan.org)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> 
>>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From ttarrant at redhat.com  Mon Jun 16 05:32:03 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 16 Jun 2014 11:32:03 +0200
Subject: [infinispan-dev] stabilize test suite
In-Reply-To: <C7EC99C8-57AF-4D7F-BF28-5ECC7547BB30@redhat.com>
References: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>	<CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>	<539B1586.4070506@redhat.com>	<EB28C000-23CD-42A5-A3FF-3E2C3D6B7DC3@redhat.com>	<B15B76BF-006A-4132-82ED-AA6F26755D2C@redhat.com>	<06AEBE40-0F0E-4F4C-AE78-CBCEDE9A9166@redhat.com>
	<C7EC99C8-57AF-4D7F-BF28-5ECC7547BB30@redhat.com>
Message-ID: <539EB993.6060408@redhat.com>

The REST endpoint has stopped working since the upgrade to Undertow.

Tracking this as https://issues.jboss.org/browse/ISPN-4407

Tristan

On 16/06/14 11:24, Galder Zamarre?o wrote:
> Next up, I?m checking what?s up with:
>
> org.infinispan.server.test.cs.leveldb.* failures:
>
> LevelDBCacheStoreIT.testDataRetrievableViaLevelDbApi	
> LevelDBCacheStoreIT.testDataSurvivesRestart
>
> Cheers,
>
> On 16 Jun 2014, at 11:17, Galder Zamarre?o <galder at redhat.com> wrote:
>
>> I?ve fixed that and also fixed https://issues.jboss.org/browse/ISPN-4406 too, both of which focused on the instability in integrationtests/security-manager-it/
>>
>> PR for both JIRAs: https://github.com/infinispan/infinispan/pull/2642
>>
>> Onto the next lot of failing tests...
>>
>> Cheers,
>>
>> On 16 Jun 2014, at 09:47, Galder Zamarre?o <galder at redhat.com> wrote:
>>
>>> org.infinispan.security.QueryAuthorizationTest is failing to start properly, as a result of:
>>>
>>> java.lang.NoClassDefFoundError: org/apache/lucene/search/similarities/Similarity
>>>
>>> I?ve created https://issues.jboss.org/browse/ISPN-4405 and I?m looking into it.
>>>
>>> Cheers,
>>>
>>> On 13 Jun 2014, at 17:42, Randall Hauch <rhauch at redhat.com> wrote:
>>>
>>>> This may not at all be related, but maybe it might save some time. One problem ModeShape has run into is that Arquillian seems to only detect a Wildfly server when the test is run with IPv4.
>>>>
>>>> On Jun 13, 2014, at 10:15 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>>>
>>>>> I'm on https://issues.jboss.org/browse/ISPN-4403
>>>>>
>>>>> And then I'll look into why arquillian doesn't detect the hotrod server
>>>>>
>>>>> Tristan
>>>>>
>>>>> On 13/06/14 16:45, William Burns wrote:
>>>>>> I am currently looking at https://issues.jboss.org/browse/ISPN-4389
>>>>>> which involves the StateTransferSuppressFor* tests.
>>>>>>
>>>>>> Also after that I was planning on getting to
>>>>>> https://issues.jboss.org/browse/ISPN-4384 which is a test that fails
>>>>>> spuriously CacheNotifierInitialTransferDistTest
>>>>>>
>>>>>> On Fri, Jun 13, 2014 at 9:52 AM, Mircea Markus <mmarkus at redhat.com> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> The test suite is allover, so please STOP integrating into master till the suite gets green.
>>>>>>> Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> --
>>>>>>> Mircea Markus
>>>>>>> Infinispan lead (www.infinispan.org)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>> twitter.com/galderz
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>


From dan.berindei at gmail.com  Mon Jun 16 10:57:43 2014
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 16 Jun 2014 17:57:43 +0300
Subject: [infinispan-dev] Reliability of return values
In-Reply-To: <2A1770A0-9884-461D-B288-14B2F18A96E8@redhat.com>
References: <53707669.7070701@redhat.com>
	<CA+nfvwT66bci7eY2hqBcAgtba7tO7ov_dBt7t5ypk026qEwvAQ@mail.gmail.com>
	<CAFm4XO0-d_sQPgNRJxDnVb56ADvu00UEn9qj=fKD88+fj5BRwQ@mail.gmail.com>
	<5370A873.8070509@redhat.com>
	<CA+nfvwQDkzbZ7fuoLX80xtx0KgspGZVWoqD+vo=W5iFRSCcKEg@mail.gmail.com>
	<53723CD4.9060300@redhat.com>
	<CA+nfvwTKCSf1fw=SDJGswzZu29Lcdv0j6jdpFEwLgECj0aFpdg@mail.gmail.com>
	<B999DCB7-3F63-42D8-97F2-7E053E187EA7@redhat.com>
	<2A1770A0-9884-461D-B288-14B2F18A96E8@redhat.com>
Message-ID: <CA+nfvwR=eFk7cjNgo0kJUr+uSezi-jZQFkF=HNwSQvhwyJTLQA@mail.gmail.com>

On Thu, Jun 12, 2014 at 4:54 PM, Galder Zamarre?o <galder at redhat.com> wrote:

> Hi all,
>
> I?m working on the implementation of this, and the solution noted in the
> JIRA does not work for situations where you have to return a previous value
> that might have been overriden due to partial operation application.
> Example (assuming 1 owner only):
>
> 1. remote client does a put(?key?, 1) in Node-A
> 2. remote client does a replace(?key?, 2) on Node-A but the operation
> fails to replicate and gets partially applieed in Node-A only.
> 3. remote client, seeing the replace failed, retries the replace(?key?, 2)
> in Node-B. replace underneath finds that the previous value of ?key? is 2
> (since it got partially applied in Node-A), so it succeeds but the previous
> value returned is 2, which is wrong. The previous value should have been 1,
> but this value is gone?
>
> In my view, there is two ways to solve this issue:
>
> 1. Make Hot Rod caches transactional. By doign so, operations are not
> partially applied. They?re done fully cluster wide or they?re rolled back.
> I?ve verified this and the test passes once you make the cache
> transactional. The downside is of course performance. IOW, anyone using
> conditional operations, or relying on previous values, would need
> transactional caches. This should work well with the retry mechanism being
> implemented for ISPN-2956, which is still needed. A big question here is
> whether Hot Rod caches should be transactional by default or viceversa. If
> they?re transactional, our performance will go down for sure but we won?t
> have this type of issues (with retry in place). If you?re not
> transactional, you are faster but you?re exposed to these edge cases, and
> you need to consider them when deploying your app, something people might
> miss, although we could provide WARN messages when conditional operations
> or Flag.FORCE_RETURN_VALUE is used with non-transactional caches.
>

The same problem [1] can appear in embedded caches. So if there's an
argument to make HotRod caches transactional by default, it should apply to
embedded caches as well.

I like the idea of logging a warning when the FORCE_RETURN_VALUE or the
non-versioned conditional operations are used from HotRod. But we have to
be careful with the wording, because inconsistencies can appear in
transactional mode as well [2].

[1] https://issues.jboss.org/browse/ISPN-4286
[2] https://issues.jboss.org/browse/ISPN-3421


> 2. Get rid of returning previous value in the Hot Rod protocol for
> modifying operations. For conditional operations, returning true/false is
> at least enough to see if the condition was applied. So,
> replaceIfUnmodified/replace/remove(conditional), they would only return
> true/false. This would be complicated due to reliance on Map/ConcurrentMap
> APIs. Maybe something to consider for when we stop relying on JDK APIs.
>
>
I'm not sure we can completely get rid of the return values, even though
JCache doesn't extend Map it still has a getAndPut method.



> I also considered applying corrective actions but that?s very messy and
> prone to concurrency issues, so I quickly discarded that.
>

Yeah, rolling back partial modifications is definitely not an option.


>
> Any other options? Thoughts on the options above?
>

I was waiting for Sanne to say this, but how about keeping a version
history?

If we had the chain of values/versions for each key, we could look up the
version of the current operation in the chain when retrying. If it's there,
we could infer that the operation was already applied, and return the
previous value in the chain as the previous version.

Of course, there are the usual downsides:
1. Maintaining the history might be tricky, especially around state
transfers.
2. Performance will also be affected, maybe getting closer to the tx
performance.


>
> Cheers,
>
> On 26 May 2014, at 18:11, Galder Zamarre?o <galder at redhat.com> wrote:
>
> > Hi all,
> >
> > I?ve been looking into ISPN-2956 last week and I think we have a
> solution for it which requires a protocol change [1]
> >
> > Since we?re in the middle of the Hot Rod 2.0 development, this is a good
> opportunity to implement it.
> >
> > Cheers,
> >
> > [1]
> https://issues.jboss.org/browse/ISPN-2956?focusedCommentId=12970541&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12970541
> >
> > On 14 May 2014, at 09:36, Dan Berindei <dan.berindei at gmail.com> wrote:
> >
> >>
> >>
> >>
> >> On Tue, May 13, 2014 at 6:40 PM, Radim Vansa <rvansa at redhat.com> wrote:
> >> On 05/13/2014 03:58 PM, Dan Berindei wrote:
> >>>
> >>>
> >>> On Mon, May 12, 2014 at 1:54 PM, Radim Vansa <rvansa at redhat.com>
> wrote:
> >>> @Dan: It's absolutely correct to do the further writes in order to make
> >>> the cache consistent, I am not arguing against that. You've fixed the
> >>> outcome (state of cache) well. My point was that we should let the user
> >>> know that the value he gets is not 100% correct when we already know
> >>> that - and given the API, the only option to do that seems to me as
> >>> throwing an exception.
> >>>
> >>> The problem, as I see it, is that users also expect methods that throw
> an exception to *not* modify the cache.
> >>> So we would break some of the users' expectations anyway.
> >>
> >> When the response from primary owner does not arrive soon, we throw
> timeout exception and the cache is modified anyway, isn't it?
> >> If we throw ~ReturnValueUnreliableException, the user has at least some
> chance to react. Currently, for code requiring 100% reliable value, you
> can't do anything but ignore the return value, even for CAS operations.
> >>
> >>
> >> Yes, but we don't expect the user to handle a TimeoutException in any
> meaningful way. Instead, we expect the user to choose his hardware and
> configuration to avoid timeouts, if he cares about consistency. How could
> you handle an exception that tells you "I may have written the value you
> asked me to in the cache, or maybe not. Either way, you will never know
> what the previous value was. Muahahaha!" in an application that cares about
> consistency?
> >>
> >> But the proposed ReturnValueUnreliableException can't be avoided by the
> user, it has to be handled every time the cluster membership changes. So it
> would be more like WriteSkewException than TimeoutException. And when we
> throw a WriteSkewException, we don't write anything to the cache.
> >>
> >> Remember, most users do not care about the previous value at all -
> that's the reason why JCache and our HotRod client don't return the
> previous value by default. Those that do care about the previous value, use
> the conditional write operations, and those already work (well, except for
> the scenario below). So you would force everyone to handle an exception
> that they don't care about.
> >>
> >> It would make sense to throw an exception if we didn't return the
> previous value by default, and the user requested the return value
> explicitly. But we do return the value by default, so I don't think it
> would be a good idea for us.
> >>
> >>>
> >>>
> >>> @Sanne: I was not suggesting that for now - sure, value versioning is
> (I
> >>> hope) on the roadmap. But that's more complicated, I though just about
> >>> making an adjustment to the current implementation.
> >>>
> >>>
> >>> Actually, just keeping a history of values would not fix the the
> return value in all cases.
> >>>
> >>> When retrying a put on the new primary owner, the primary owner would
> still have to compare our value with the latest value, and return the
> previous value if they are equal. So we could have something like this:
> >>>
> >>> A is the originator, B is the primary owner, k = v0
> >>> A -> B: put(k, v1)
> >>> B dies before writing v, C is now primary owner
> >>> D -> C: put(k, v1) // another put operation from D, with the same value
> >>> C -> D: null
> >>> A -> C: retry_put(k, v1)
> >>> C -> A: v0 // C assumes A is overwriting its own value, so it's
> returning the previous one
> >>>
> >>> To fix that, we'd need a unique version generated by the originator -
> kind of like a transaction id ;)
> >>
> >> Is it such a problem to associate unique ID with each write? History
> implementation seems to me like the more complicated part.
> >>
> >> I also think maintaining a version history would be quite complicated,
> and it also would make it harder for users to estimate their cache's memory
> usage. That's why I was trying to show that it's not a panacea.
> >>
> >>
> >>
> >>> And to fix the HotRod use case, the HotRod client would have to be the
> one generating the version.
> >>
> >> I agree.
> >>
> >> Radim
> >>
> >>
> >>>
> >>> Cheers
> >>> Dan
> >>>
> >>>
> >>>
> >>> Radim
> >>>
> >>> On 05/12/2014 12:02 PM, Sanne Grinovero wrote:
> >>>> I don't think we are in a position to decide what is a reasonable
> >>>> compromise; we can do better.
> >>>> For example - as Radim suggested - it might seem reasonable to have
> >>>> the older value around for a little while. We'll need a little bit of
> >>>> history of values and tombstones anyway for many other reasons.
> >>>>
> >>>>
> >>>> Sanne
> >>>>
> >>>> On 12 May 2014 09:37, Dan Berindei <dan.berindei at gmail.com> wrote:
> >>>>> Radim, I would contend that the first and foremost guarantee that
> put()
> >>>>> makes is to leave the cache in a consistent state. So we can't just
> throw an
> >>>>> exception and give up, leaving k=v on one owner and k=null on
> another.
> >>>>>
> >>>>> Secondly, put(k, v) being atomic means that it either succeeds, it
> writes
> >>>>> k=v in the cache, and it returns the previous value, or it doesn't
> succeed,
> >>>>> and it doesn't write k=v in the cache. Returning the wrong previous
> value is
> >>>>> bad, but leaving k=v in the cache is just as bad, even if the all
> the owners
> >>>>> have the same value.
> >>>>>
> >>>>> And last, we can't have one node seeing k=null, then k=v, then
> k=null again,
> >>>>> when the only write we did on the cache was a put(k, v). So trying
> to undo
> >>>>> the write would not help.
> >>>>>
> >>>>> In the end, we have to make a compromise, and I think returning the
> wrong
> >>>>> value in some of the cases is a reasonable compromise. Of course, we
> should
> >>>>> document that :)
> >>>>>
> >>>>> I also believe ISPN-2956 could be fixed so that HotRod behaves just
> like
> >>>>> embedded mode after the ISPN-3422 fix, by adding a RETRY flag to the
> HotRod
> >>>>> protocol and to the cache itself.
> >>>>>
> >>>>> Incidentally, transactional caches have a similar problem when the
> >>>>> originator leaves the cluster: ISPN-3421 [1]
> >>>>> And we can't handle transactional caches any better than
> non-transactional
> >>>>> caches until we expose transactions to the HotRod client.
> >>>>>
> >>>>> [1] https://issues.jboss.org/browse/ISPN-2956
> >>>>>
> >>>>> Cheers
> >>>>> Dan
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Mon, May 12, 2014 at 10:21 AM, Radim Vansa <rvansa at redhat.com>
> wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> recently I've stumbled upon one already expected behaviour (one
> instance
> >>>>>> is [1]), but which did not got much attention.
> >>>>>>
> >>>>>> In non-tx cache, when the primary owner fails after the request has
> been
> >>>>>> replicated to backup owner, the request is retried in the new
> topology.
> >>>>>> Then, the operation is executed on the new primary (the previous
> >>>>>> backup). The outcome has been already fixed in [2], but the return
> value
> >>>>>> may be wrong. For example, when we do a put, the return value for
> the
> >>>>>> second attempt will be the currently inserted value (although the
> entry
> >>>>>> was just created). Same situation may happen for other operations.
> >>>>>>
> >>>>>> Currently, it's not possible to return the correct value (because
> it has
> >>>>>> already been overwritten and we don't keep a history of values), but
> >>>>>> shouldn't we rather throw an exception if we were not able to
> fulfil the
> >>>>>> API contract?
> >>>>>>
> >>>>>> Radim
> >>>>>>
> >>>>>> [1] https://issues.jboss.org/browse/ISPN-2956
> >>>>>> [2] https://issues.jboss.org/browse/ISPN-3422
> >>>>>>
> >>>>>> --
> >>>>>> Radim Vansa <rvansa at redhat.com>
> >>>>>> JBoss DataGrid QA
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> infinispan-dev mailing list
> >>>>>> infinispan-dev at lists.jboss.org
> >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> infinispan-dev mailing list
> >>>>> infinispan-dev at lists.jboss.org
> >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>> --
> >>> Radim Vansa <rvansa at redhat.com>
> >>> JBoss DataGrid QA
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>>
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >>
> >> --
> >> Radim Vansa
> >> <rvansa at redhat.com>
> >>
> >> JBoss DataGrid QA
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> > --
> > Galder Zamarre?o
> > galder at redhat.com
> > twitter.com/galderz
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140616/1a2de919/attachment-0001.html 

From galder at redhat.com  Tue Jun 17 04:37:49 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Tue, 17 Jun 2014 10:37:49 +0200
Subject: [infinispan-dev] stabilize test suite
In-Reply-To: <C7EC99C8-57AF-4D7F-BF28-5ECC7547BB30@redhat.com>
References: <2932E48D-7BD5-4DFA-81D2-0DD19B45CA8D@redhat.com>
	<CA+YCuUP=iKOxt6TKYFQyCLc_Y6XMKQ_ia7xxhqPzggooti2+cw@mail.gmail.com>
	<539B1586.4070506@redhat.com>
	<EB28C000-23CD-42A5-A3FF-3E2C3D6B7DC3@redhat.com>
	<B15B76BF-006A-4132-82ED-AA6F26755D2C@redhat.com>
	<06AEBE40-0F0E-4F4C-AE78-CBCEDE9A9166@redhat.com>
	<C7EC99C8-57AF-4D7F-BF28-5ECC7547BB30@redhat.com>
Message-ID: <0F7937A0-A4B6-42EE-85F5-0A997B992D63@redhat.com>

WRT LevelDB tests:
https://issues.jboss.org/browse/ISPN-4409
https://github.com/infinispan/infinispan/pull/2650

Cheers,

On 16 Jun 2014, at 11:24, Galder Zamarre?o <galder at redhat.com> wrote:

> Next up, I?m checking what?s up with:
> 
> org.infinispan.server.test.cs.leveldb.* failures:
> 
> LevelDBCacheStoreIT.testDataRetrievableViaLevelDbApi	
> LevelDBCacheStoreIT.testDataSurvivesRestart
> 
> Cheers,
> 
> On 16 Jun 2014, at 11:17, Galder Zamarre?o <galder at redhat.com> wrote:
> 
>> I?ve fixed that and also fixed https://issues.jboss.org/browse/ISPN-4406 too, both of which focused on the instability in integrationtests/security-manager-it/
>> 
>> PR for both JIRAs: https://github.com/infinispan/infinispan/pull/2642
>> 
>> Onto the next lot of failing tests...
>> 
>> Cheers,
>> 
>> On 16 Jun 2014, at 09:47, Galder Zamarre?o <galder at redhat.com> wrote:
>> 
>>> org.infinispan.security.QueryAuthorizationTest is failing to start properly, as a result of:
>>> 
>>> java.lang.NoClassDefFoundError: org/apache/lucene/search/similarities/Similarity
>>> 
>>> I?ve created https://issues.jboss.org/browse/ISPN-4405 and I?m looking into it.
>>> 
>>> Cheers,
>>> 
>>> On 13 Jun 2014, at 17:42, Randall Hauch <rhauch at redhat.com> wrote:
>>> 
>>>> This may not at all be related, but maybe it might save some time. One problem ModeShape has run into is that Arquillian seems to only detect a Wildfly server when the test is run with IPv4.
>>>> 
>>>> On Jun 13, 2014, at 10:15 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:
>>>> 
>>>>> I'm on https://issues.jboss.org/browse/ISPN-4403
>>>>> 
>>>>> And then I'll look into why arquillian doesn't detect the hotrod server
>>>>> 
>>>>> Tristan
>>>>> 
>>>>> On 13/06/14 16:45, William Burns wrote:
>>>>>> I am currently looking at https://issues.jboss.org/browse/ISPN-4389
>>>>>> which involves the StateTransferSuppressFor* tests.
>>>>>> 
>>>>>> Also after that I was planning on getting to
>>>>>> https://issues.jboss.org/browse/ISPN-4384 which is a test that fails
>>>>>> spuriously CacheNotifierInitialTransferDistTest
>>>>>> 
>>>>>> On Fri, Jun 13, 2014 at 9:52 AM, Mircea Markus <mmarkus at redhat.com> wrote:
>>>>>>> Hi,
>>>>>>> 
>>>>>>> The test suite is allover, so please STOP integrating into master till the suite gets green.
>>>>>>> Also please start looking into the failures: in order not to overlap, reply to this email with what you're looking at.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> --
>>>>>>> Mircea Markus
>>>>>>> Infinispan lead (www.infinispan.org)
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> 
>>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> 
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>> twitter.com/galderz
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From galder at redhat.com  Tue Jun 17 05:53:35 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Tue, 17 Jun 2014 11:53:35 +0200
Subject: [infinispan-dev] Reliability of return values
In-Reply-To: <CA+nfvwR=eFk7cjNgo0kJUr+uSezi-jZQFkF=HNwSQvhwyJTLQA@mail.gmail.com>
References: <53707669.7070701@redhat.com>
	<CA+nfvwT66bci7eY2hqBcAgtba7tO7ov_dBt7t5ypk026qEwvAQ@mail.gmail.com>
	<CAFm4XO0-d_sQPgNRJxDnVb56ADvu00UEn9qj=fKD88+fj5BRwQ@mail.gmail.com>
	<5370A873.8070509@redhat.com>
	<CA+nfvwQDkzbZ7fuoLX80xtx0KgspGZVWoqD+vo=W5iFRSCcKEg@mail.gmail.com>
	<53723CD4.9060300@redhat.com>
	<CA+nfvwTKCSf1fw=SDJGswzZu29Lcdv0j6jdpFEwLgECj0aFpdg@mail.gmail.com>
	<B999DCB7-3F63-42D8-97F2-7E053E187EA7@redhat.com>
	<2A1770A0-9884-461D-B288-14B2F18A96E8@redhat.com>
	<CA+nfvwR=eFk7cjNgo0kJUr+uSezi-jZQFkF=HNwSQvhwyJTLQA@mail.gmail.com>
Message-ID: <1B37D0B5-91A6-46B5-974A-9EED679FC5AF@redhat.com>


On 16 Jun 2014, at 16:57, Dan Berindei <dan.berindei at gmail.com> wrote:

> On Thu, Jun 12, 2014 at 4:54 PM, Galder Zamarre?o <galder at redhat.com> wrote:
> Hi all,
> 
> I?m working on the implementation of this, and the solution noted in the JIRA does not work for situations where you have to return a previous value that might have been overriden due to partial operation application. Example (assuming 1 owner only):
> 
> 1. remote client does a put(?key?, 1) in Node-A
> 2. remote client does a replace(?key?, 2) on Node-A but the operation fails to replicate and gets partially applieed in Node-A only.
> 3. remote client, seeing the replace failed, retries the replace(?key?, 2) in Node-B. replace underneath finds that the previous value of ?key? is 2 (since it got partially applied in Node-A), so it succeeds but the previous value returned is 2, which is wrong. The previous value should have been 1, but this value is gone?
> 
> In my view, there is two ways to solve this issue:
> 
> 1. Make Hot Rod caches transactional. By doign so, operations are not partially applied. They?re done fully cluster wide or they?re rolled back. I?ve verified this and the test passes once you make the cache transactional. The downside is of course performance. IOW, anyone using conditional operations, or relying on previous values, would need transactional caches. This should work well with the retry mechanism being implemented for ISPN-2956, which is still needed. A big question here is whether Hot Rod caches should be transactional by default or viceversa. If they?re transactional, our performance will go down for sure but we won?t have this type of issues (with retry in place). If you?re not transactional, you are faster but you?re exposed to these edge cases, and you need to consider them when deploying your app, something people might miss, although we could provide WARN messages when conditional operations or Flag.FORCE_RETURN_VALUE is used with non-transactional caches.
> 
> The same problem [1] can appear in embedded caches. So if there's an argument to make HotRod caches transactional by default, it should apply to embedded caches as well.
> 
> I like the idea of logging a warning when the FORCE_RETURN_VALUE or the non-versioned conditional operations are used from HotRod. But we have to be careful with the wording, because inconsistencies can appear in transactional mode as well [2].

I think this (default non-transaction, and WARN if using such methods with non-transactional cache) might be the best stop gap solution.

> [1] https://issues.jboss.org/browse/ISPN-4286
> [2] https://issues.jboss.org/browse/ISPN-3421

Have you considered other fixes for [2]?

> 
> 
> 2. Get rid of returning previous value in the Hot Rod protocol for modifying operations. For conditional operations, returning true/false is at least enough to see if the condition was applied. So, replaceIfUnmodified/replace/remove(conditional), they would only return true/false. This would be complicated due to reliance on Map/ConcurrentMap APIs. Maybe something to consider for when we stop relying on JDK APIs.
> 
> 
> I'm not sure we can completely get rid of the return values, even though JCache doesn't extend Map it still has a getAndPut method.

It does have such method but we could potentially make it throw an exception saying that is not supported.

> 
>  
> I also considered applying corrective actions but that?s very messy and prone to concurrency issues, so I quickly discarded that.
> 
> Yeah, rolling back partial modifications is definitely not an option.
>  
> 
> Any other options? Thoughts on the options above?
> 
> I was waiting for Sanne to say this, but how about keeping a version history?
> 
> If we had the chain of values/versions for each key, we could look up the version of the current operation in the chain when retrying. If it's there, we could infer that the operation was already applied, and return the previous value in the chain as the previous version.
> 
> Of course, there are the usual downsides:
> 1. Maintaining the history might be tricky, especially around state transfers.
> 2. Performance will also be affected, maybe getting closer to the tx performance.

Yeah, keeping version history could help, but it?d be quite a beast to implement for 7.0, including garbage collection of old versions...etc.

>  
> 
> Cheers,
> 
> On 26 May 2014, at 18:11, Galder Zamarre?o <galder at redhat.com> wrote:
> 
> > Hi all,
> >
> > I?ve been looking into ISPN-2956 last week and I think we have a solution for it which requires a protocol change [1]
> >
> > Since we?re in the middle of the Hot Rod 2.0 development, this is a good opportunity to implement it.
> >
> > Cheers,
> >
> > [1] https://issues.jboss.org/browse/ISPN-2956?focusedCommentId=12970541&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12970541
> >
> > On 14 May 2014, at 09:36, Dan Berindei <dan.berindei at gmail.com> wrote:
> >
> >>
> >>
> >>
> >> On Tue, May 13, 2014 at 6:40 PM, Radim Vansa <rvansa at redhat.com> wrote:
> >> On 05/13/2014 03:58 PM, Dan Berindei wrote:
> >>>
> >>>
> >>> On Mon, May 12, 2014 at 1:54 PM, Radim Vansa <rvansa at redhat.com> wrote:
> >>> @Dan: It's absolutely correct to do the further writes in order to make
> >>> the cache consistent, I am not arguing against that. You've fixed the
> >>> outcome (state of cache) well. My point was that we should let the user
> >>> know that the value he gets is not 100% correct when we already know
> >>> that - and given the API, the only option to do that seems to me as
> >>> throwing an exception.
> >>>
> >>> The problem, as I see it, is that users also expect methods that throw an exception to *not* modify the cache.
> >>> So we would break some of the users' expectations anyway.
> >>
> >> When the response from primary owner does not arrive soon, we throw timeout exception and the cache is modified anyway, isn't it?
> >> If we throw ~ReturnValueUnreliableException, the user has at least some chance to react. Currently, for code requiring 100% reliable value, you can't do anything but ignore the return value, even for CAS operations.
> >>
> >>
> >> Yes, but we don't expect the user to handle a TimeoutException in any meaningful way. Instead, we expect the user to choose his hardware and configuration to avoid timeouts, if he cares about consistency. How could you handle an exception that tells you "I may have written the value you asked me to in the cache, or maybe not. Either way, you will never know what the previous value was. Muahahaha!" in an application that cares about consistency?
> >>
> >> But the proposed ReturnValueUnreliableException can't be avoided by the user, it has to be handled every time the cluster membership changes. So it would be more like WriteSkewException than TimeoutException. And when we throw a WriteSkewException, we don't write anything to the cache.
> >>
> >> Remember, most users do not care about the previous value at all - that's the reason why JCache and our HotRod client don't return the previous value by default. Those that do care about the previous value, use the conditional write operations, and those already work (well, except for the scenario below). So you would force everyone to handle an exception that they don't care about.
> >>
> >> It would make sense to throw an exception if we didn't return the previous value by default, and the user requested the return value explicitly. But we do return the value by default, so I don't think it would be a good idea for us.
> >>
> >>>
> >>>
> >>> @Sanne: I was not suggesting that for now - sure, value versioning is (I
> >>> hope) on the roadmap. But that's more complicated, I though just about
> >>> making an adjustment to the current implementation.
> >>>
> >>>
> >>> Actually, just keeping a history of values would not fix the the return value in all cases.
> >>>
> >>> When retrying a put on the new primary owner, the primary owner would still have to compare our value with the latest value, and return the previous value if they are equal. So we could have something like this:
> >>>
> >>> A is the originator, B is the primary owner, k = v0
> >>> A -> B: put(k, v1)
> >>> B dies before writing v, C is now primary owner
> >>> D -> C: put(k, v1) // another put operation from D, with the same value
> >>> C -> D: null
> >>> A -> C: retry_put(k, v1)
> >>> C -> A: v0 // C assumes A is overwriting its own value, so it's returning the previous one
> >>>
> >>> To fix that, we'd need a unique version generated by the originator - kind of like a transaction id ;)
> >>
> >> Is it such a problem to associate unique ID with each write? History implementation seems to me like the more complicated part.
> >>
> >> I also think maintaining a version history would be quite complicated, and it also would make it harder for users to estimate their cache's memory usage. That's why I was trying to show that it's not a panacea.
> >>
> >>
> >>
> >>> And to fix the HotRod use case, the HotRod client would have to be the one generating the version.
> >>
> >> I agree.
> >>
> >> Radim
> >>
> >>
> >>>
> >>> Cheers
> >>> Dan
> >>>
> >>>
> >>>
> >>> Radim
> >>>
> >>> On 05/12/2014 12:02 PM, Sanne Grinovero wrote:
> >>>> I don't think we are in a position to decide what is a reasonable
> >>>> compromise; we can do better.
> >>>> For example - as Radim suggested - it might seem reasonable to have
> >>>> the older value around for a little while. We'll need a little bit of
> >>>> history of values and tombstones anyway for many other reasons.
> >>>>
> >>>>
> >>>> Sanne
> >>>>
> >>>> On 12 May 2014 09:37, Dan Berindei <dan.berindei at gmail.com> wrote:
> >>>>> Radim, I would contend that the first and foremost guarantee that put()
> >>>>> makes is to leave the cache in a consistent state. So we can't just throw an
> >>>>> exception and give up, leaving k=v on one owner and k=null on another.
> >>>>>
> >>>>> Secondly, put(k, v) being atomic means that it either succeeds, it writes
> >>>>> k=v in the cache, and it returns the previous value, or it doesn't succeed,
> >>>>> and it doesn't write k=v in the cache. Returning the wrong previous value is
> >>>>> bad, but leaving k=v in the cache is just as bad, even if the all the owners
> >>>>> have the same value.
> >>>>>
> >>>>> And last, we can't have one node seeing k=null, then k=v, then k=null again,
> >>>>> when the only write we did on the cache was a put(k, v). So trying to undo
> >>>>> the write would not help.
> >>>>>
> >>>>> In the end, we have to make a compromise, and I think returning the wrong
> >>>>> value in some of the cases is a reasonable compromise. Of course, we should
> >>>>> document that :)
> >>>>>
> >>>>> I also believe ISPN-2956 could be fixed so that HotRod behaves just like
> >>>>> embedded mode after the ISPN-3422 fix, by adding a RETRY flag to the HotRod
> >>>>> protocol and to the cache itself.
> >>>>>
> >>>>> Incidentally, transactional caches have a similar problem when the
> >>>>> originator leaves the cluster: ISPN-3421 [1]
> >>>>> And we can't handle transactional caches any better than non-transactional
> >>>>> caches until we expose transactions to the HotRod client.
> >>>>>
> >>>>> [1] https://issues.jboss.org/browse/ISPN-2956
> >>>>>
> >>>>> Cheers
> >>>>> Dan
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Mon, May 12, 2014 at 10:21 AM, Radim Vansa <rvansa at redhat.com> wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> recently I've stumbled upon one already expected behaviour (one instance
> >>>>>> is [1]), but which did not got much attention.
> >>>>>>
> >>>>>> In non-tx cache, when the primary owner fails after the request has been
> >>>>>> replicated to backup owner, the request is retried in the new topology.
> >>>>>> Then, the operation is executed on the new primary (the previous
> >>>>>> backup). The outcome has been already fixed in [2], but the return value
> >>>>>> may be wrong. For example, when we do a put, the return value for the
> >>>>>> second attempt will be the currently inserted value (although the entry
> >>>>>> was just created). Same situation may happen for other operations.
> >>>>>>
> >>>>>> Currently, it's not possible to return the correct value (because it has
> >>>>>> already been overwritten and we don't keep a history of values), but
> >>>>>> shouldn't we rather throw an exception if we were not able to fulfil the
> >>>>>> API contract?
> >>>>>>
> >>>>>> Radim
> >>>>>>
> >>>>>> [1] https://issues.jboss.org/browse/ISPN-2956
> >>>>>> [2] https://issues.jboss.org/browse/ISPN-3422
> >>>>>>
> >>>>>> --
> >>>>>> Radim Vansa <rvansa at redhat.com>
> >>>>>> JBoss DataGrid QA
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> infinispan-dev mailing list
> >>>>>> infinispan-dev at lists.jboss.org
> >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> infinispan-dev mailing list
> >>>>> infinispan-dev at lists.jboss.org
> >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>> --
> >>> Radim Vansa <rvansa at redhat.com>
> >>> JBoss DataGrid QA
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>>
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >>
> >> --
> >> Radim Vansa
> >> <rvansa at redhat.com>
> >>
> >> JBoss DataGrid QA
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> > --
> > Galder Zamarre?o
> > galder at redhat.com
> > twitter.com/galderz
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From mmarkus at redhat.com  Wed Jun 18 07:34:30 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Wed, 18 Jun 2014 12:34:30 +0100
Subject: [infinispan-dev] Reliability of return values
In-Reply-To: <1B37D0B5-91A6-46B5-974A-9EED679FC5AF@redhat.com>
References: <53707669.7070701@redhat.com>
	<CA+nfvwT66bci7eY2hqBcAgtba7tO7ov_dBt7t5ypk026qEwvAQ@mail.gmail.com>
	<CAFm4XO0-d_sQPgNRJxDnVb56ADvu00UEn9qj=fKD88+fj5BRwQ@mail.gmail.com>
	<5370A873.8070509@redhat.com>
	<CA+nfvwQDkzbZ7fuoLX80xtx0KgspGZVWoqD+vo=W5iFRSCcKEg@mail.gmail.com>
	<53723CD4.9060300@redhat.com>
	<CA+nfvwTKCSf1fw=SDJGswzZu29Lcdv0j6jdpFEwLgECj0aFpdg@mail.gmail.com>
	<B999DCB7-3F63-42D8-97F2-7E053E187EA7@redhat.com>
	<2A1770A0-9884-461D-B288-14B2F18A96E8@redhat.com>
	<CA+nfvwR=eFk7cjNgo0kJUr+uSezi-jZQFkF=HNwSQvhwyJTLQA@mail.gmail.com>
	<1B37D0B5-91A6-46B5-974A-9EED679FC5AF@redhat.com>
Message-ID: <FFA6221D-6AE5-4325-BB1A-9822CD8A885B@redhat.com>


On Jun 17, 2014, at 10:53, Galder Zamarre?o <galder at redhat.com> wrote:

> 
> On 16 Jun 2014, at 16:57, Dan Berindei <dan.berindei at gmail.com> wrote:
> 
>> On Thu, Jun 12, 2014 at 4:54 PM, Galder Zamarre?o <galder at redhat.com> wrote:
>> Hi all,
>> 
>> I?m working on the implementation of this, and the solution noted in the JIRA does not work for situations where you have to return a previous value that might have been overriden due to partial operation application. Example (assuming 1 owner only):
>> 
>> 1. remote client does a put(?key?, 1) in Node-A
>> 2. remote client does a replace(?key?, 2) on Node-A but the operation fails to replicate and gets partially applieed in Node-A only.
>> 3. remote client, seeing the replace failed, retries the replace(?key?, 2) in Node-B. replace underneath finds that the previous value of ?key? is 2 (since it got partially applied in Node-A), so it succeeds but the previous value returned is 2, which is wrong. The previous value should have been 1, but this value is gone?
>> 
>> In my view, there is two ways to solve this issue:
>> 
>> 1. Make Hot Rod caches transactional. By doign so, operations are not partially applied. They?re done fully cluster wide or they?re rolled back. I?ve verified this and the test passes once you make the cache transactional. The downside is of course performance. IOW, anyone using conditional operations, or relying on previous values, would need transactional caches. This should work well with the retry mechanism being implemented for ISPN-2956, which is still needed. A big question here is whether Hot Rod caches should be transactional by default or viceversa. If they?re transactional, our performance will go down for sure but we won?t have this type of issues (with retry in place). If you?re not transactional, you are faster but you?re exposed to these edge cases, and you need to consider them when deploying your app, something people might miss, although we could provide WARN messages when conditional operations or Flag.FORCE_RETURN_VALUE is used with non-transactional caches.
>> 
>> The same problem [1] can appear in embedded caches. So if there's an argument to make HotRod caches transactional by default, it should apply to embedded caches as well.
>> 
>> I like the idea of logging a warning when the FORCE_RETURN_VALUE or the non-versioned conditional operations are used from HotRod. But we have to be careful with the wording, because inconsistencies can appear in transactional mode as well [2].
> 
> I think this (default non-transaction, and WARN if using such methods with non-transactional cache) might be the best stop gap solution.

+1. Would be good to have [2] fixed as well, in case people want a workable workaround.

> 
>> [1] https://issues.jboss.org/browse/ISPN-4286
>> [2] https://issues.jboss.org/browse/ISPN-3421
> 
> Have you considered other fixes for [2]?
> 
>> 
>> 
>> 2. Get rid of returning previous value in the Hot Rod protocol for modifying operations. For conditional operations, returning true/false is at least enough to see if the condition was applied. So, replaceIfUnmodified/replace/remove(conditional), they would only return true/false. This would be complicated due to reliance on Map/ConcurrentMap APIs. Maybe something to consider for when we stop relying on JDK APIs.
>> 
>> 
>> I'm not sure we can completely get rid of the return values, even though JCache doesn't extend Map it still has a getAndPut method.
> 
> It does have such method but we could potentially make it throw an exception saying that is not supported.
> 
>> 
>> 
>> I also considered applying corrective actions but that?s very messy and prone to concurrency issues, so I quickly discarded that.
>> 
>> Yeah, rolling back partial modifications is definitely not an option.
>> 
>> 
>> Any other options? Thoughts on the options above?
>> 
>> I was waiting for Sanne to say this, but how about keeping a version history?
>> 
>> If we had the chain of values/versions for each key, we could look up the version of the current operation in the chain when retrying. If it's there, we could infer that the operation was already applied, and return the previous value in the chain as the previous version.
>> 
>> Of course, there are the usual downsides:
>> 1. Maintaining the history might be tricky, especially around state transfers.
>> 2. Performance will also be affected, maybe getting closer to the tx performance.
> 
> Yeah, keeping version history could help, but it?d be quite a beast to implement for 7.0, including garbage collection of old versions...etc.

Yes, also option 1 seems to make things work and is almost ready. Also with versioning, I think the performance is worse, as you always need to communicate when you no longer need a version(RPC), so that it can be garbage collected. 

> 
>> 
>> 
>> Cheers,
>> 
>> On 26 May 2014, at 18:11, Galder Zamarre?o <galder at redhat.com> wrote:
>> 
>>> Hi all,
>>> 
>>> I?ve been looking into ISPN-2956 last week and I think we have a solution for it which requires a protocol change [1]
>>> 
>>> Since we?re in the middle of the Hot Rod 2.0 development, this is a good opportunity to implement it.
>>> 
>>> Cheers,
>>> 
>>> [1] https://issues.jboss.org/browse/ISPN-2956?focusedCommentId=12970541&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12970541
>>> 
>>> On 14 May 2014, at 09:36, Dan Berindei <dan.berindei at gmail.com> wrote:
>>> 
>>>> 
>>>> 
>>>> 
>>>> On Tue, May 13, 2014 at 6:40 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>>> On 05/13/2014 03:58 PM, Dan Berindei wrote:
>>>>> 
>>>>> 
>>>>> On Mon, May 12, 2014 at 1:54 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>> @Dan: It's absolutely correct to do the further writes in order to make
>>>>> the cache consistent, I am not arguing against that. You've fixed the
>>>>> outcome (state of cache) well. My point was that we should let the user
>>>>> know that the value he gets is not 100% correct when we already know
>>>>> that - and given the API, the only option to do that seems to me as
>>>>> throwing an exception.
>>>>> 
>>>>> The problem, as I see it, is that users also expect methods that throw an exception to *not* modify the cache.
>>>>> So we would break some of the users' expectations anyway.
>>>> 
>>>> When the response from primary owner does not arrive soon, we throw timeout exception and the cache is modified anyway, isn't it?
>>>> If we throw ~ReturnValueUnreliableException, the user has at least some chance to react. Currently, for code requiring 100% reliable value, you can't do anything but ignore the return value, even for CAS operations.
>>>> 
>>>> 
>>>> Yes, but we don't expect the user to handle a TimeoutException in any meaningful way. Instead, we expect the user to choose his hardware and configuration to avoid timeouts, if he cares about consistency. How could you handle an exception that tells you "I may have written the value you asked me to in the cache, or maybe not. Either way, you will never know what the previous value was. Muahahaha!" in an application that cares about consistency?
>>>> 
>>>> But the proposed ReturnValueUnreliableException can't be avoided by the user, it has to be handled every time the cluster membership changes. So it would be more like WriteSkewException than TimeoutException. And when we throw a WriteSkewException, we don't write anything to the cache.
>>>> 
>>>> Remember, most users do not care about the previous value at all - that's the reason why JCache and our HotRod client don't return the previous value by default. Those that do care about the previous value, use the conditional write operations, and those already work (well, except for the scenario below). So you would force everyone to handle an exception that they don't care about.
>>>> 
>>>> It would make sense to throw an exception if we didn't return the previous value by default, and the user requested the return value explicitly. But we do return the value by default, so I don't think it would be a good idea for us.
>>>> 
>>>>> 
>>>>> 
>>>>> @Sanne: I was not suggesting that for now - sure, value versioning is (I
>>>>> hope) on the roadmap. But that's more complicated, I though just about
>>>>> making an adjustment to the current implementation.
>>>>> 
>>>>> 
>>>>> Actually, just keeping a history of values would not fix the the return value in all cases.
>>>>> 
>>>>> When retrying a put on the new primary owner, the primary owner would still have to compare our value with the latest value, and return the previous value if they are equal. So we could have something like this:
>>>>> 
>>>>> A is the originator, B is the primary owner, k = v0
>>>>> A -> B: put(k, v1)
>>>>> B dies before writing v, C is now primary owner
>>>>> D -> C: put(k, v1) // another put operation from D, with the same value
>>>>> C -> D: null
>>>>> A -> C: retry_put(k, v1)
>>>>> C -> A: v0 // C assumes A is overwriting its own value, so it's returning the previous one
>>>>> 
>>>>> To fix that, we'd need a unique version generated by the originator - kind of like a transaction id ;)
>>>> 
>>>> Is it such a problem to associate unique ID with each write? History implementation seems to me like the more complicated part.
>>>> 
>>>> I also think maintaining a version history would be quite complicated, and it also would make it harder for users to estimate their cache's memory usage. That's why I was trying to show that it's not a panacea.
>>>> 
>>>> 
>>>> 
>>>>> And to fix the HotRod use case, the HotRod client would have to be the one generating the version.
>>>> 
>>>> I agree.
>>>> 
>>>> Radim
>>>> 
>>>> 
>>>>> 
>>>>> Cheers
>>>>> Dan
>>>>> 
>>>>> 
>>>>> 
>>>>> Radim
>>>>> 
>>>>> On 05/12/2014 12:02 PM, Sanne Grinovero wrote:
>>>>>> I don't think we are in a position to decide what is a reasonable
>>>>>> compromise; we can do better.
>>>>>> For example - as Radim suggested - it might seem reasonable to have
>>>>>> the older value around for a little while. We'll need a little bit of
>>>>>> history of values and tombstones anyway for many other reasons.
>>>>>> 
>>>>>> 
>>>>>> Sanne
>>>>>> 
>>>>>> On 12 May 2014 09:37, Dan Berindei <dan.berindei at gmail.com> wrote:
>>>>>>> Radim, I would contend that the first and foremost guarantee that put()
>>>>>>> makes is to leave the cache in a consistent state. So we can't just throw an
>>>>>>> exception and give up, leaving k=v on one owner and k=null on another.
>>>>>>> 
>>>>>>> Secondly, put(k, v) being atomic means that it either succeeds, it writes
>>>>>>> k=v in the cache, and it returns the previous value, or it doesn't succeed,
>>>>>>> and it doesn't write k=v in the cache. Returning the wrong previous value is
>>>>>>> bad, but leaving k=v in the cache is just as bad, even if the all the owners
>>>>>>> have the same value.
>>>>>>> 
>>>>>>> And last, we can't have one node seeing k=null, then k=v, then k=null again,
>>>>>>> when the only write we did on the cache was a put(k, v). So trying to undo
>>>>>>> the write would not help.
>>>>>>> 
>>>>>>> In the end, we have to make a compromise, and I think returning the wrong
>>>>>>> value in some of the cases is a reasonable compromise. Of course, we should
>>>>>>> document that :)
>>>>>>> 
>>>>>>> I also believe ISPN-2956 could be fixed so that HotRod behaves just like
>>>>>>> embedded mode after the ISPN-3422 fix, by adding a RETRY flag to the HotRod
>>>>>>> protocol and to the cache itself.
>>>>>>> 
>>>>>>> Incidentally, transactional caches have a similar problem when the
>>>>>>> originator leaves the cluster: ISPN-3421 [1]
>>>>>>> And we can't handle transactional caches any better than non-transactional
>>>>>>> caches until we expose transactions to the HotRod client.
>>>>>>> 
>>>>>>> [1] https://issues.jboss.org/browse/ISPN-2956
>>>>>>> 
>>>>>>> Cheers
>>>>>>> Dan
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Mon, May 12, 2014 at 10:21 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> recently I've stumbled upon one already expected behaviour (one instance
>>>>>>>> is [1]), but which did not got much attention.
>>>>>>>> 
>>>>>>>> In non-tx cache, when the primary owner fails after the request has been
>>>>>>>> replicated to backup owner, the request is retried in the new topology.
>>>>>>>> Then, the operation is executed on the new primary (the previous
>>>>>>>> backup). The outcome has been already fixed in [2], but the return value
>>>>>>>> may be wrong. For example, when we do a put, the return value for the
>>>>>>>> second attempt will be the currently inserted value (although the entry
>>>>>>>> was just created). Same situation may happen for other operations.
>>>>>>>> 
>>>>>>>> Currently, it's not possible to return the correct value (because it has
>>>>>>>> already been overwritten and we don't keep a history of values), but
>>>>>>>> shouldn't we rather throw an exception if we were not able to fulfil the
>>>>>>>> API contract?
>>>>>>>> 
>>>>>>>> Radim
>>>>>>>> 
>>>>>>>> [1] https://issues.jboss.org/browse/ISPN-2956
>>>>>>>> [2] https://issues.jboss.org/browse/ISPN-3422
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Radim Vansa <rvansa at redhat.com>
>>>>>>>> JBoss DataGrid QA
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> infinispan-dev mailing list
>>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> 
>>>>> 
>>>>> --
>>>>> Radim Vansa <rvansa at redhat.com>
>>>>> JBoss DataGrid QA
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> 
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> 
>>>> --
>>>> Radim Vansa
>>>> <rvansa at redhat.com>
>>>> 
>>>> JBoss DataGrid QA
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> 
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>> twitter.com/galderz
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From sanne at infinispan.org  Wed Jun 18 12:21:41 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 18 Jun 2014 17:21:41 +0100
Subject: [infinispan-dev] CD datastore inserts slow
In-Reply-To: <7A5D7DA2-77F5-4710-9091-7A76C8A8DA40@gmail.com>
References: <CAFm4XO2gL0pcKyH0F0OTN2VATdjsOPMG+704CNgAXGCO-u9T-Q@mail.gmail.com>
	<878E3B1C-6CD2-411C-A4A5-8F379F9A4AFA@gmail.com>
	<CAFm4XO3jy3i0NUycZYJ4wAQ7XsW-3VaKRiV73-jWVFrDiqfqpg@mail.gmail.com>
	<7A5D7DA2-77F5-4710-9091-7A76C8A8DA40@gmail.com>
Message-ID: <CAFm4XO2k=LiTjEkHadkVeBbGi6rgOdvL3a6kK_MrXBA=SnTFYQ@mail.gmail.com>

Hi Ales,
I've looked at the benchmark.
The main problem is that the high amount of IndexWriter open/close
cycles are quite CPU intensive and also generate a huge amount of
temporary objects; as previously mentioned the easy workaround is to
write objects in batches or transactions, but we can do other
improvements too of course.

My mainline is ~320 seconds when running CapeDwarf as-is; when
reconfiguring the backend to use the more efficient Infinispan
IndexManager (as we had actually previously suggested!) we avoid the
open/close cost but we still have the synchronous flush, so
performance improves but is still not good: 215 seconds.

This last scenario is harder to improve further, as the bottleneck
becomes the chattiness of the Index storage in Infinispan, combined
with the CacheStore usage. We can change the default buffer sizes for
the index to improve it but it would be much more effective if you
could batch multiple changes together.
Also we should look at improving the CacheStore performance, which is
the main bottleneck, although its cost gets amplified by the
inefficient use we do at higher level.. I will try some more
experiments with this.

To compare, I reconfigured it to use local-only NTR backends (not
suited for clustering) and so doing I found another problem: all of
your caches are sharing the same index!
By reconfiguring each cache to use and independent path to isolate the
indexes, the benchmark completes in 20 seconds, and these remaining 20
seconds are all spent in CapeDwarf code, with some more object
allocations from Infinispan core.

So in short, even perfectly tuning indexing away, you can' t get
faster than 20 seconds without making some more changes in CD and
Infinispan Core as well.. would that be an acceptable goal?

Also, question: do you really need both clustering and the persistent
CacheStore for the TCK tests? disabling these aspects would massively
speedup the whole thing.

Sanne






On 3 June 2014 19:39, Ales Justin <ales.justin at gmail.com> wrote:
> Here:
>
> ---
>
> I've added this test to GAE TCK:
> *
> https://github.com/GoogleCloudPlatform/appengine-tck/blob/master/core/benchmark/src/test/java/com/google/appengine/tck/benchmark/ObjectifyBenchmarkTest.java
>
> You need CapeDwarf 2.0.0.CR2:
> * http://capedwarf.org/downloads/
>
> And clone GAE TCK:
> * https://github.com/GoogleCloudPlatform/appengine-tck
>
> Run CapeDwarf:
> * https://github.com/capedwarf/capedwarf-blue/blob/master/README.md (see
> (5))
>
> Then run the ObjectifyBenchmarkTest:
> * cd <GAE_TCK_HOME>
> * mvn clean install
> * cd core/benchmark
> * mvn clean install -Pcapedwarf,benchmark
>
> Ping me for any issues.
>
> Thanks!
>
> -Ales
>
> On 03 Jun 2014, at 17:40, Sanne Grinovero <sanne at infinispan.org> wrote:
>
> Hi Ales,
> I don't have CD installed anymore but will try it out. Do you have
> some pointers to set up and run this benchmark?
>
> Sanne
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From ales.justin at gmail.com  Thu Jun 19 08:29:09 2014
From: ales.justin at gmail.com (Ales Justin)
Date: Thu, 19 Jun 2014 14:29:09 +0200
Subject: [infinispan-dev] CD datastore inserts slow
In-Reply-To: <CAFm4XO2k=LiTjEkHadkVeBbGi6rgOdvL3a6kK_MrXBA=SnTFYQ@mail.gmail.com>
References: <CAFm4XO2gL0pcKyH0F0OTN2VATdjsOPMG+704CNgAXGCO-u9T-Q@mail.gmail.com>
	<878E3B1C-6CD2-411C-A4A5-8F379F9A4AFA@gmail.com>
	<CAFm4XO3jy3i0NUycZYJ4wAQ7XsW-3VaKRiV73-jWVFrDiqfqpg@mail.gmail.com>
	<7A5D7DA2-77F5-4710-9091-7A76C8A8DA40@gmail.com>
	<CAFm4XO2k=LiTjEkHadkVeBbGi6rgOdvL3a6kK_MrXBA=SnTFYQ@mail.gmail.com>
Message-ID: <B95E9DEB-2D7F-4F14-9D37-FE3312F30609@gmail.com>

> To compare, I reconfigured it to use local-only NTR backends (not
> suited for clustering) and so doing I found another problem: all of
> your caches are sharing the same index!
> By reconfiguring each cache to use and independent path to isolate the
> indexes, the benchmark completes in 20 seconds, and these remaining 20
> seconds are all spent in CapeDwarf code, with some more object
> allocations from Infinispan core.

How do you configure this index isolation?
(how exactly does this look in standalone.xml?)

> So in short, even perfectly tuning indexing away, you can' t get
> faster than 20 seconds without making some more changes in CD and
> Infinispan Core as well.. would that be an acceptable goal?

20sec vs 320sec definitely sounds better. ;-)

What kind of changes would one have to make in CD?

> Also, question: do you really need both clustering and the persistent
> CacheStore for the TCK tests? disabling these aspects would massively
> speedup the whole thing.

Wdym?
You mean we could do w/o cache store in tests?
Sure, but we want to be as close as possible to real usage scenario.

-Ales


From sanne at infinispan.org  Thu Jun 19 09:12:27 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 19 Jun 2014 14:12:27 +0100
Subject: [infinispan-dev] CD datastore inserts slow
In-Reply-To: <B95E9DEB-2D7F-4F14-9D37-FE3312F30609@gmail.com>
References: <CAFm4XO2gL0pcKyH0F0OTN2VATdjsOPMG+704CNgAXGCO-u9T-Q@mail.gmail.com>
	<878E3B1C-6CD2-411C-A4A5-8F379F9A4AFA@gmail.com>
	<CAFm4XO3jy3i0NUycZYJ4wAQ7XsW-3VaKRiV73-jWVFrDiqfqpg@mail.gmail.com>
	<7A5D7DA2-77F5-4710-9091-7A76C8A8DA40@gmail.com>
	<CAFm4XO2k=LiTjEkHadkVeBbGi6rgOdvL3a6kK_MrXBA=SnTFYQ@mail.gmail.com>
	<B95E9DEB-2D7F-4F14-9D37-FE3312F30609@gmail.com>
Message-ID: <CAFm4XO370A-EJFrSGx-bcr1Wbz+odAkxckZ0d+osx2xWb_obWA@mail.gmail.com>

On 19 June 2014 13:29, Ales Justin <ales.justin at gmail.com> wrote:
>> To compare, I reconfigured it to use local-only NTR backends (not
>> suited for clustering) and so doing I found another problem: all of
>> your caches are sharing the same index!
>> By reconfiguring each cache to use and independent path to isolate the
>> indexes, the benchmark completes in 20 seconds, and these remaining 20
>> seconds are all spent in CapeDwarf code, with some more object
>> allocations from Infinispan core.
>
> How do you configure this index isolation?
> (how exactly does this look in standalone.xml?)

I did it locally, can send you a pull request.

But to clarify: this would disable clustering! Do you really want
that? the first option is safer, I'd just change the IndexManager.

I can send you a pull request for both variations of the
configuration, just let me know which file it is I have to patch: I
tried to look into the repository for capedwarf-jboss-as, but the
configuration file in there seems quite different than the one in
CapeDwarf blue distribution. So I'm not sure if I'm looking in the
right place, or maybe the wrong branch?

>
>> So in short, even perfectly tuning indexing away, you can' t get
>> faster than 20 seconds without making some more changes in CD and
>> Infinispan Core as well.. would that be an acceptable goal?
>
> 20sec vs 320sec definitely sounds better. ;-)
>
> What kind of changes would one have to make in CD?

General optimisation, the usual stuff: profile it, rinse and repeat. I
didn't look at details but I remember a priority would be to optimise
some HashMap lookups for internal services and components.

>> Also, question: do you really need both clustering and the persistent
>> CacheStore for the TCK tests? disabling these aspects would massively
>> speedup the whole thing.
>
> Wdym?
> You mean we could do w/o cache store in tests?
> Sure, but we want to be as close as possible to real usage scenario.

It's probably better to try tuning the CacheStore then, but you'll
always be quite slow as CacheStore(s) significantly slow down
Infinispan.
You should also ask yourself if the CacheStore is really needed.

Sanne

>
> -Ales
>

From ales.justin at gmail.com  Thu Jun 19 09:44:53 2014
From: ales.justin at gmail.com (Ales Justin)
Date: Thu, 19 Jun 2014 15:44:53 +0200
Subject: [infinispan-dev] CD datastore inserts slow
In-Reply-To: <CAFm4XO370A-EJFrSGx-bcr1Wbz+odAkxckZ0d+osx2xWb_obWA@mail.gmail.com>
References: <CAFm4XO2gL0pcKyH0F0OTN2VATdjsOPMG+704CNgAXGCO-u9T-Q@mail.gmail.com>
	<878E3B1C-6CD2-411C-A4A5-8F379F9A4AFA@gmail.com>
	<CAFm4XO3jy3i0NUycZYJ4wAQ7XsW-3VaKRiV73-jWVFrDiqfqpg@mail.gmail.com>
	<7A5D7DA2-77F5-4710-9091-7A76C8A8DA40@gmail.com>
	<CAFm4XO2k=LiTjEkHadkVeBbGi6rgOdvL3a6kK_MrXBA=SnTFYQ@mail.gmail.com>
	<B95E9DEB-2D7F-4F14-9D37-FE3312F30609@gmail.com>
	<CAFm4XO370A-EJFrSGx-bcr1Wbz+odAkxckZ0d+osx2xWb_obWA@mail.gmail.com>
Message-ID: <F97F9B3F-D1D2-4E13-B8A2-EAC90CF6D719@gmail.com>

>>> To compare, I reconfigured it to use local-only NTR backends (not
>>> suited for clustering) and so doing I found another problem: all of
>>> your caches are sharing the same index!
>>> By reconfiguring each cache to use and independent path to isolate the
>>> indexes, the benchmark completes in 20 seconds, and these remaining 20
>>> seconds are all spent in CapeDwarf code, with some more object
>>> allocations from Infinispan core.
>> 
>> How do you configure this index isolation?
>> (how exactly does this look in standalone.xml?)
> 
> I did it locally, can send you a pull request.

Please do.

> But to clarify: this would disable clustering!

Why or how would this disable clustering?

> Do you really want
> that?

Probably not. :-)

> the first option is safer, I'd just change the IndexManager.

Which IndexManager do we use now?

> I can send you a pull request for both variations of the
> configuration, just let me know which file it is I have to patch: I
> tried to look into the repository for capedwarf-jboss-as, but the
> configuration file in there seems quite different than the one in
> CapeDwarf blue distribution.

Hmmm, they should be the same, or what's the diff?
* https://github.com/capedwarf/capedwarf-testsuite/blob/master/integrate-cd/src/main/resources/standalone/configuration/standalone-capedwarf-modules.xml
* https://github.com/capedwarf/capedwarf-jboss-as/blob/master/build/src/main/resources/standalone/configuration/standalone-capedwarf-modules.xml

> So I'm not sure if I'm looking in the
> right place, or maybe the wrong branch?

The distribution / release is done via Testsuite:
* https://github.com/capedwarf/capedwarf-testsuite
(see -Prelease profile)

> General optimisation, the usual stuff: profile it, rinse and repeat. I
> didn't look at details but I remember a priority would be to optimise
> some HashMap lookups for internal services and components.

Do you perhaps still have some profile data on which look-ups are this?

> It's probably better to try tuning the CacheStore then, but you'll
> always be quite slow as CacheStore(s) significantly slow down
> Infinispan.
> You should also ask yourself if the CacheStore is really needed.

Well, I'd leave this up to the users.

-Ales

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140619/548c4837/attachment.html 

From galder at redhat.com  Thu Jun 19 10:12:19 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Thu, 19 Jun 2014 16:12:19 +0200
Subject: [infinispan-dev] Where to put KeyValueFilterFactory and
	ConverterFactory...
Message-ID: <F2AE1601-B627-49FA-8F26-4BA988222B43@redhat.com>

Hi all,

As I work on ISPN-3950, I?ve been thinking about the location of KeyValueFilterFactory  and ConverterFactory interfaces. These interfaces provide filter/converter instances to be installed in server/hotrod, but we expect users to provide implementations of such classes and plug the server with them.

These interfaces are currently in server/hotrod project, something I?m not keen on doing any more, because users would need to get access to these interfaces and they could start fiddling with other stuff in that module, and I want to avoid that at all costs, particularly since I?m hoping to do some major refactoring of the decoders in 7.1.

So, the q is: where should KeyValueFilterFactory and ConverterFactory live? 

They?re for sure needed by server/hotrod. client/hotrod-client does not need them explicitly, but the users need to provide implementations of them and plug them to the server. With this in mind, core/ might be the best place for it. They cannot live in commons/ cos it?d would require to move KeyValueFilter/Converter and other dependant classes to commons.

Is everyone happy with ConverterFactory and KeyValueFilterFactory being in core/?

Cheers,
--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From pedro at infinispan.org  Thu Jun 19 10:18:50 2014
From: pedro at infinispan.org (Pedro Ruivo)
Date: Thu, 19 Jun 2014 15:18:50 +0100
Subject: [infinispan-dev] Where to put KeyValueFilterFactory and
	ConverterFactory...
In-Reply-To: <F2AE1601-B627-49FA-8F26-4BA988222B43@redhat.com>
References: <F2AE1601-B627-49FA-8F26-4BA988222B43@redhat.com>
Message-ID: <53A2F14A.8080901@infinispan.org>



On 06/19/2014 03:12 PM, Galder Zamarre?o wrote:
> Hi all,
>
> As I work on ISPN-3950, I?ve been thinking about the location of KeyValueFilterFactory  and ConverterFactory interfaces. These interfaces provide filter/converter instances to be installed in server/hotrod, but we expect users to provide implementations of such classes and plug the server with them.
>
> These interfaces are currently in server/hotrod project, something I?m not keen on doing any more, because users would need to get access to these interfaces and they could start fiddling with other stuff in that module, and I want to avoid that at all costs, particularly since I?m hoping to do some major refactoring of the decoders in 7.1.
>
> So, the q is: where should KeyValueFilterFactory and ConverterFactory live?
>
> They?re for sure needed by server/hotrod. client/hotrod-client does not need them explicitly, but the users need to provide implementations of them and plug them to the server. With this in mind, core/ might be the best place for it. They cannot live in commons/ cos it?d would require to move KeyValueFilter/Converter and other dependant classes to commons.
>
IMO, this is the best solution. I don't think a user should bring all 
the core/ (and probably its dependencies) to implement a couple of 
interfaces...

Cheers,
Pedro

> Is everyone happy with ConverterFactory and KeyValueFilterFactory being in core/?
>
> Cheers,
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

From ttarrant at redhat.com  Thu Jun 19 10:25:42 2014
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Thu, 19 Jun 2014 16:25:42 +0200
Subject: [infinispan-dev] Where to put KeyValueFilterFactory and
	ConverterFactory...
In-Reply-To: <F2AE1601-B627-49FA-8F26-4BA988222B43@redhat.com>
References: <F2AE1601-B627-49FA-8F26-4BA988222B43@redhat.com>
Message-ID: <53A2F2E6.4050803@redhat.com>

To me core is ideal:
when you develop these components, it's as if you were running in an 
embedded environment (which is what the server essentially is), and core 
is the closest you can get.

Tristan

On 19/06/14 16:12, Galder Zamarre?o wrote:
> Hi all,
>
> As I work on ISPN-3950, I?ve been thinking about the location of KeyValueFilterFactory  and ConverterFactory interfaces. These interfaces provide filter/converter instances to be installed in server/hotrod, but we expect users to provide implementations of such classes and plug the server with them.
>
> These interfaces are currently in server/hotrod project, something I?m not keen on doing any more, because users would need to get access to these interfaces and they could start fiddling with other stuff in that module, and I want to avoid that at all costs, particularly since I?m hoping to do some major refactoring of the decoders in 7.1.
>
> So, the q is: where should KeyValueFilterFactory and ConverterFactory live?
>
> They?re for sure needed by server/hotrod. client/hotrod-client does not need them explicitly, but the users need to provide implementations of them and plug them to the server. With this in mind, core/ might be the best place for it. They cannot live in commons/ cos it?d would require to move KeyValueFilter/Converter and other dependant classes to commons.
>
> Is everyone happy with ConverterFactory and KeyValueFilterFactory being in core/?
>
> Cheers,
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>


From mudokonman at gmail.com  Thu Jun 19 10:34:09 2014
From: mudokonman at gmail.com (William Burns)
Date: Thu, 19 Jun 2014 10:34:09 -0400
Subject: [infinispan-dev] Where to put KeyValueFilterFactory and
	ConverterFactory...
In-Reply-To: <53A2F2E6.4050803@redhat.com>
References: <F2AE1601-B627-49FA-8F26-4BA988222B43@redhat.com>
	<53A2F2E6.4050803@redhat.com>
Message-ID: <CA+YCuUPUTt5-Wh=7Bo0Zzf8PTYOucxfCS8kccr9F9cL2zfh5Gw@mail.gmail.com>

I was able to look into the details with this a bit.  At first thought
I would think having a module that both the client and server packages
could both depend on would be best.  However the issue is that these
factories produce instances that implement interfaces that live in the
core package, which means the server/client would have to depend on
core anyways.  So I am thinking core is probably the best place for
now.

Thinking about these factories even more it could be useful to use
these at some point in core to allow for non serializable
KeyValueFilter/Converter instances and instead have the factory be
serializable and just create them on each node as needed.

 - Will

On Thu, Jun 19, 2014 at 10:25 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:
> To me core is ideal:
> when you develop these components, it's as if you were running in an
> embedded environment (which is what the server essentially is), and core
> is the closest you can get.
>
> Tristan
>
> On 19/06/14 16:12, Galder Zamarre?o wrote:
>> Hi all,
>>
>> As I work on ISPN-3950, I?ve been thinking about the location of KeyValueFilterFactory  and ConverterFactory interfaces. These interfaces provide filter/converter instances to be installed in server/hotrod, but we expect users to provide implementations of such classes and plug the server with them.
>>
>> These interfaces are currently in server/hotrod project, something I?m not keen on doing any more, because users would need to get access to these interfaces and they could start fiddling with other stuff in that module, and I want to avoid that at all costs, particularly since I?m hoping to do some major refactoring of the decoders in 7.1.
>>
>> So, the q is: where should KeyValueFilterFactory and ConverterFactory live?
>>
>> They?re for sure needed by server/hotrod. client/hotrod-client does not need them explicitly, but the users need to provide implementations of them and plug them to the server. With this in mind, core/ might be the best place for it. They cannot live in commons/ cos it?d would require to move KeyValueFilter/Converter and other dependant classes to commons.
>>
>> Is everyone happy with ConverterFactory and KeyValueFilterFactory being in core/?
>>
>> Cheers,
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From sanne at infinispan.org  Fri Jun 20 08:34:14 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 20 Jun 2014 13:34:14 +0100
Subject: [infinispan-dev] Test failures
In-Reply-To: <CA+nfvwSJR6z_iB3vdDhLfmDiko8GgW7-DOu6pgcbf9nZ1EUXkQ@mail.gmail.com>
References: <CAFm4XO1R=fo_Ek9G8HL8xdA7XZZuLJNA5cGA0xBpqE+Tp5baqQ@mail.gmail.com>
	<CA+nfvwRWfZTkxcAeGs6VDT7=3TCSKUy5nfAGX9ePGRoHypwRGA@mail.gmail.com>
	<CAFm4XO265nEkRB=nV8RLcqS5WJHcrBAH+SKmAwGwOQe_Y88AWA@mail.gmail.com>
	<CA+nfvwQKdzpWhukFMdTw7bfsB0DkZg-AGPiQQfQMAjXDRJbRBQ@mail.gmail.com>
	<CAFm4XO0fdee6r4f+QT_M5rr-dD6g=j2rxMQGLCOuxyALsjRZEw@mail.gmail.com>
	<002B61E4-8A6F-49E7-B9A8-36AED6091F67@redhat.com>
	<CA+nfvwSJR6z_iB3vdDhLfmDiko8GgW7-DOu6pgcbf9nZ1EUXkQ@mail.gmail.com>
Message-ID: <CAFm4XO14mwwG4F7TZF0iANgn=f--09x+VWkCHFzPEALh0K7zDQ@mail.gmail.com>

I've been trying again, current state:

Failed tests:
  ConditionalOperationsConcurrentWriteSkewTest.testPutIfAbsent:62->ConditionalOperationsConcurrentTest.testOnCaches:129
java.lang.RuntimeException: java.util.concurrent.TimeoutException

Tests run: 4436, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Infinispan BOM .................................... SUCCESS [  0.237 s]
[INFO] Infinispan Common Parent .......................... SUCCESS [  2.192 s]
[INFO] Infinispan Checkstyle Rules ....................... SUCCESS [  3.591 s]
[INFO] Infinispan Commons ................................ SUCCESS [  6.209 s]
[INFO] Infinispan Core ................................... FAILURE [11:10 min]

..

Sanne

On 11 June 2014 08:03, Dan Berindei <dan.berindei at gmail.com> wrote:
> I'll skip straight to the conclusions in the paper:
>
>> ? Keep regression tests around for up to a year ? but most of
> those will be system-level tests rather than unit tests.
>
> I would say most of our tests qualify as regression tests, and they are
> system-level (they start a full cache manager, with a JGroups channel and
> everything).
> But how would we know that they haven't failed in a year? I mean we can find
> which tests never failed in CI, but we can't find which tests never failed
> on any dev's machine (thus helping that dev find a problem in his code).
>
>> ? Keep unit tests that test key algorithms for which there is a
> broad, formal, independent oracle of correctness, and for
> which there is ascribable business value.
>> ? Except for the preceding case, if X has business value and you
> can text X with either a system test or a unit test, use a system
> test ? context is everything.
>
> We have about 100 tests in the "unit" group, some of which should really be
> in the "functional" group (e.g. BaseStoreTest).
> It shouldn't take long to walk over those and delete some that may be tested
> better by system tests, but it won't have a visible effect on the test suite
> run time either.
>
>> ? Design a test with more care than you design the code.
>
> I admit that when reviewing PRs I pay less attention to the tests than to
> the production code. I will try to change that :)
>
>> ? Turn most unit tests into assertions.
>
> I imagine some kinds of tests can indeed be replaced with assertions in
> production code, but again we don't have so many unit tests.
>
>> ? Throw away tests that haven?t failed in a year.
>> ? Testing can?t replace good development: a high test failure
> rate suggests you should shorten development intervals,
> perhaps radically, and make sure your architecture and design
> regimens have teeth
>> ? If you find that individual functions being tested are trivial,
> double-check the way you incentivize developers?
> performance. Rewarding coverage or other meaningless
> metrics can lead to rapid architecture decay.
>> ? Be humble about what tests can achieve. Tests don?t improve
> quality: developers do.
>
>
>
>
> On Tue, Jun 10, 2014 at 6:23 PM, Galder Zamarre?o <galder at redhat.com> wrote:
>>
>> Just recently I read this:
>> http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf
>>
>> Food for thought, and maybe we should consider doing for next major. Maybe
>> if we?re moving to JUnit we could look at this in detail too?
>>
>> Cheers,
>>
>> On 09 Jun 2014, at 23:06, Sanne Grinovero <sanne at infinispan.org> wrote:
>>
>> > On 9 June 2014 14:42, Dan Berindei <dan.berindei at gmail.com> wrote:
>> >>
>> >>
>> >> On Mon, Jun 9, 2014 at 12:43 PM, Sanne Grinovero <sanne at infinispan.org>
>> >> wrote:
>> >>>
>> >>> Hi Dan!
>> >>>
>> >>> The reason is that I'm making substantial API changes in the Query
>> >>> module, and I've lost count on how many other modules and integration
>> >>> tests depend on it: I need to run all the testsuite to evaluate where
>> >>> I'm heading.. I don't need it just as last touches but continually, to
>> >>> be able to make good choices while work is in progress.
>> >>
>> >>
>> >> Wouldn't "mvn install -DskipTests" be enough for testing dependencies?
>> >> You could then use "mvn surefire:test -pl query,lucene-directory" to
>> >> run
>> >> just the tests you're interested in.
>> >> I have a script that does just that - even for core, I don't want to
>> >> think
>> >> about whether I changed something in commons or not, but I almost never
>> >> want
>> >> to run the commons tests :)
>> >
>> > I need to run all modules, and AFAIK there is no way to run the tests
>> > on all modules except one (unless you list all of them explicitly..
>> > good luck with that).
>> >
>> > What I did to go ahead in this ball of mud is to patch the core
>> > pom.xml to add "skipTests" option to the surefire configuration, at
>> > least I could test my stuff.
>> >
>> >
>> >>> Not only I'm changing API but also substantial changes in the
>> >>> dependency tree.. without a working testsuite I can't make progress.
>> >>> I'm working around it by deleting core in a temporary commit... :-/
>> >>> (and even so the suite takes more than an hour ??!)
>> >>>
>> >>
>> >> 1 hour just for the core, or for everything? I used to get about 1h for
>> >> everything, if just the core takes that long to fail it's definitely a
>> >> problem. There was also a bit of a slowdown after the JGroups
>> >> 3.5.0.Beta7
>> >> upgrade (ISPN-4355), but that should be fixed now.
>> >
>> > It's 1h and 20 minutes now to test all modules. This took 12 minutes a
>> > couple of months ago. My opinion is that even 12 minutes is too slow,
>> > I would consider it acceptable if we where running some soak tests but
>> > as far as I know we're just load testing the testing framework.
>> >
>> > Sanne
>> >
>> >>
>> >> I was also thinking of moving the failsafe plugin to the "extras"
>> >> profile,
>> >> so that we can avoid running the server integration tests in dev
>> >> builds. But
>> >> disabling the extras profile also disables the bundling for OSGi, so
>> >> perhaps
>> >> a separate profile would be better.
>> >>
>> >> Dan
>> >>
>> >>
>> >>>
>> >>> I'll test your PRs ASAP, thanks a lot.
>> >>>
>> >>> Cheers,
>> >>> Sanne
>> >>>
>> >>>
>> >>> On 9 June 2014 10:19, Dan Berindei <dan.berindei at gmail.com> wrote:
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Fri, Jun 6, 2014 at 4:31 PM, Sanne Grinovero
>> >>>> <sanne at infinispan.org>
>> >>>> wrote:
>> >>>>>
>> >>>>> I'm having several failures, these are blocking our progress on
>> >>>>> Query.
>> >>>>>
>> >>>>> Should I disable them all?
>> >>>>
>> >>>>
>> >>>> You could disable them, but I'm not quite sure how that would help
>> >>>> the
>> >>>> query
>> >>>> module... surely you don't need to run the core tests every time you
>> >>>> modify
>> >>>> something in query?
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>> Sample output of three different runs:
>> >>>>>
>> >>>>> Failed tests:
>> >>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState
>> >>>>> Thread
>> >>>>> locals st...
>> >>>>
>> >>>>
>> >>>> I couldn't reproduce this failure on my machine, but I modified
>> >>>> ThreadLocalLeakTest to ignore that particular thread-local:
>> >>>> https://github.com/infinispan/infinispan/pull/2614
>> >>>> You're the only one that reported seeing it, so please test the PR on
>> >>>> your
>> >>>> machine and integrate it.
>> >>>>
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> >>>>> ? Runtime
>> >>>>> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>> >>>>
>> >>>>
>> >>>> I've created https://issues.jboss.org/browse/ISPN-4368 for Will to
>> >>>> look
>> >>>> into, I'm not sure we need the "placeholder" key that is causing the
>> >>>> random
>> >>>> failures.
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>> Failed tests:
>> >>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState
>> >>>>> Thread
>> >>>>> locals st...
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPutIfAbsent:104->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> >>>>> ? Runtime
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> L1StateTransferOverwriteTest>BaseTxStateTransferOverwriteTest.testStateTransferInBetweenPrepareCommitWithPut:84->BaseTxStateTransferOverwriteTest.doStateTransferInBetweenPrepareCommit:265
>> >>>>> ? Runtime
>> >>>>
>> >>>>
>> >>>> The L1StateTransferOverwriteTest failure seems to have the same cause
>> >>>> as
>> >>>> StateTransferOverwriteTest failure.
>> >>>>
>> >>>>>
>> >>>>> Tests run: 5430, Failures: 3, Errors: 0, Skipped: 0
>> >>>>>
>> >>>>> Failed tests:
>> >>>>>  ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState
>> >>>>> Thread
>> >>>>> locals st...
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> CacheNotifierImplInitialTransferDistTest.testCreateAfterIterationBeganAndSegmentNotCompleteValueOwnerClustered:611->testIterationBeganAndSegmentNotComplete:510
>> >>>>> expected [11] but found [6]
>> >>>>
>> >>>>
>> >>>> I've seen this a couple times on my machine as well, I've created
>> >>>> https://issues.jboss.org/browse/ISPN-4370
>> >>>>
>> >>>>>
>> >>>>> Tests run: 5430, Failures: 2, Errors: 0, Skipped: 0
>> >>>>
>> >>>>
>> >>>>
>> >>>> I'd let Will investigate ISPN-4368 and ISPN-4370 for a bit before
>> >>>> disabling
>> >>>> those tests - he may be able to issue a PR for them quite quickly.
>> >>>>
>> >>>> Still, just because I don't like disabling tests it doesn't mean I
>> >>>> don't
>> >>>> appreciate your stability reports - keep 'em coming!
>> >>>>
>> >>>>
>> >>>> Cheers
>> >>>> Dan
>> >>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> infinispan-dev mailing list
>> >>>> infinispan-dev at lists.jboss.org
>> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>>
>> >>> _______________________________________________
>> >>> infinispan-dev mailing list
>> >>> infinispan-dev at lists.jboss.org
>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> infinispan-dev mailing list
>> >> infinispan-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From gustavonalle at gmail.com  Mon Jun 23 05:04:08 2014
From: gustavonalle at gmail.com (Gustavo Fernandes)
Date: Mon, 23 Jun 2014 10:04:08 +0100
Subject: [infinispan-dev] Hadoop and ISPN first and next steps
Message-ID: <5E4FE91D-7A99-4AFA-A0D8-82EFE870A1E0@gmail.com>

Hi all,

Last week Pedro, myself and Mircea met at London to start prototyping the integration between Hadoop and ISPN. 

We discussed several scenarios where Hadoop and ISPN would be able to work together, and decided to start with ISPN server as the source and/or sink for a Hadoop Map Reduce job

After creating an InputFormat and OutputFormat for ISPN [1], we generated some data [2] and run a sample job [3] using Hadoop v1.x, both in docker [4] and on a 4 node physical cluster (installed with the help of puppet [5]) 

We also run the same job in the same cluster with the same data, but using HDFS as data source and sink, so that we could verify correctness.

In this setup, each Hadoop slave runs the TaskTracker, Data node and ISPN server, and the idea was to generate a split [6] based on segments and redirect the map task to be executed on the nodes associated with those segments. This routing and filtering the data is still work in progress, carried on by Pedro.

Next steps? 

- For sure optimise the current Input/OutputFormat so that it can efficiently read/write data. This will allow ISPN to become part of the Hadoop ecosystem and easier to integrate it with tools like Apache Hive [7] or Pig [8].  
- Investigate closer integration for Map Reduce, potentially usable in library mode. As you might know, YARN (the overhaul of Hadoop architecture) is not only about Map Reduce, and it offers more extensions points than Hadoop Map Reduce v1
- I read with great interest the Spark paper [9]. Spark provides a DSL with functional language constructs like map, flatMap and filter to process distributed data in memory. In this scenario, Map Reduce is just a special case achieved by chaining functions [10]. As Spark is much more than Map Reduce, and can run many machine learning algorithms efficiently, I was wondering if we should shift attention to Spark rather than focusing too much on Map Reduce. Thoughts?


[1] https://github.com/pruivo/infinispan-hadoop-integration/tree/master/src/main/java/org/infinispan/hadoopintegration/mapreduce
[2] http://www.skorks.com/2010/03/how-to-quickly-generate-a-large-file-on-the-command-line-with-linux/
[3] https://github.com/pruivo/hadoop-wordcount-example/tree/master/src/main/java/com/gustavonalle/hadoop
[4] https://github.com/gustavonalle/docker/tree/master/hadoop
[5] https://gist.github.com/gustavonalle/95dfdd771f31e1e2bf9d
[6] https://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/InputSplit.html
[7] https://hive.apache.org/
[8] http://pig.apache.org/
[9] http://www.cs.berkeley.edu/~matei/papers/2012/nsdi_spark.pdf
[10] https://spark.apache.org/docs/0.9.0/quick-start.html#more-on-rdd-operations
   
Cheers,
Gustavo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140623/4c753f0b/attachment.html 

From galder at redhat.com  Tue Jun 24 10:27:07 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Tue, 24 Jun 2014 16:27:07 +0200
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
Message-ID: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>

Hi all,

Last few days I?ve been having fun fixing [1] and the solution I?ve got to has some links to [2].

The problem is that when getCacheEntry() is called, the entry returned could be partially correct. Up until now, we?d send back a cache entry back to the client that the user could not modify, but it was linked to the data container cache entry, which is mutable and hence could change on the fly. As a result of this, the value could sometimes be updated but the version would belong to a previous value for example.

A first attempt to fix it was to provide a true immutable cache entry view which was initialised on construction, but even here there was the chance of having value and version mistmatch, because getCacheEntry does not acquire locks, so an ongoing update could result in the cache entry being constructed when the update was half way, so, it would have the right value but the old version information.

All this didn?t work well with the replaceIfUmodified logic in [3]. For example, you could easily get this situation:

Current Cache contents: key=k1, value=A, version=1

T1: Hot Rod client calls: replace(k1, value=B, old-version=1)
T2: Hot Rod client calls: replace(k1, value=B, old-version=1)
T1: Server calls getCacheEntry and retrieves value=A,version=1
T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=A, new-value=B)
T1: Server updates value to B but version is still 1
T2: Server calls getCacheEntry and retrieves value=B,version=1 (wrong!)
T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=B, new-value=B)
T1: Server updates version to 2 
T1: Returns Success, replace worked
T2: Returns Success, replace worked

The end result is that cache contained B, but both replaces returned true. This is wrong and would fail to consistenly keep a counter in the value part. Imagine the value being a counter of number of times the replace succeeded. In the test developed by Takayoshi, N times replace() would return true, but the final value would be N-1 or N-2, or N-5 :|

To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag. This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.

This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:

1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.

2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].

Finally, this was not a problem when the value stored in Hot Rod was a ByteArrayValue wrapping the byte array and the version, because the value was treated atomically, and in hindsight, maybe adding getCacheEntry might have been premature, but this method has proven useful for other use cases too (rolling upgrades?etc).

Cheers,

[1] https://issues.jboss.org/browse/ISPN-4424
[2] https://issues.jboss.org/browse/ISPN-2956
[3] https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
[4] https://github.com/galderz/infinispan/tree/t_4424
[5] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From mmarkus at redhat.com  Tue Jun 24 10:51:05 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Tue, 24 Jun 2014 15:51:05 +0100
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
Message-ID: <F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>


On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com> wrote:

> Hi all,
> 
> Last few days I?ve been having fun fixing [1] and the solution I?ve got to has some links to [2].
> 
> The problem is that when getCacheEntry() is called, the entry returned could be partially correct. Up until now, we?d send back a cache entry back to the client that the user could not modify, but it was linked to the data container cache entry, which is mutable and hence could change on the fly. As a result of this, the value could sometimes be updated but the version would belong to a previous value for example.
> 
> A first attempt to fix it was to provide a true immutable cache entry view which was initialised on construction, but even here there was the chance of having value and version mistmatch, because getCacheEntry does not acquire locks, so an ongoing update could result in the cache entry being constructed when the update was half way, so, it would have the right value but the old version information.
> 
> All this didn?t work well with the replaceIfUmodified logic in [3]. For example, you could easily get this situation:
> 
> Current Cache contents: key=k1, value=A, version=1
> 
> T1: Hot Rod client calls: replace(k1, value=B, old-version=1)
> T2: Hot Rod client calls: replace(k1, value=B, old-version=1)
> T1: Server calls getCacheEntry and retrieves value=A,version=1
> T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=A, new-value=B)
> T1: Server updates value to B but version is still 1
> T2: Server calls getCacheEntry and retrieves value=B,version=1 (wrong!)
> T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=B, new-value=B)
> T1: Server updates version to 2 
> T1: Returns Success, replace worked
> T2: Returns Success, replace worked
> 
> The end result is that cache contained B, but both replaces returned true. This is wrong and would fail to consistenly keep a counter in the value part. Imagine the value being a counter of number of times the replace succeeded. In the test developed by Takayoshi, N times replace() would return true, but the final value would be N-1 or N-2, or N-5 :|
> 
> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag.

so the entire underlaying cache needs to be transactional then?

> This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
> 
> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
> 
> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
> 
> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].
> 
> Finally, this was not a problem when the value stored in Hot Rod was a ByteArrayValue wrapping the byte array and the version, because the value was treated atomically, and in hindsight, maybe adding getCacheEntry might have been premature, but this method has proven useful for other use cases too (rolling upgrades?etc).
> 
> Cheers,
> 
> [1] https://issues.jboss.org/browse/ISPN-4424
> [2] https://issues.jboss.org/browse/ISPN-2956
> [3] https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
> [4] https://github.com/galderz/infinispan/tree/t_4424
> [5] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From galder at redhat.com  Tue Jun 24 11:50:48 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Tue, 24 Jun 2014 17:50:48 +0200
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
	<F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>
Message-ID: <47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>


On 24 Jun 2014, at 16:51, Mircea Markus <mmarkus at redhat.com> wrote:

> 
> On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com> wrote:
> 
>> Hi all,
>> 
>> Last few days I?ve been having fun fixing [1] and the solution I?ve got to has some links to [2].
>> 
>> The problem is that when getCacheEntry() is called, the entry returned could be partially correct. Up until now, we?d send back a cache entry back to the client that the user could not modify, but it was linked to the data container cache entry, which is mutable and hence could change on the fly. As a result of this, the value could sometimes be updated but the version would belong to a previous value for example.
>> 
>> A first attempt to fix it was to provide a true immutable cache entry view which was initialised on construction, but even here there was the chance of having value and version mistmatch, because getCacheEntry does not acquire locks, so an ongoing update could result in the cache entry being constructed when the update was half way, so, it would have the right value but the old version information.
>> 
>> All this didn?t work well with the replaceIfUmodified logic in [3]. For example, you could easily get this situation:
>> 
>> Current Cache contents: key=k1, value=A, version=1
>> 
>> T1: Hot Rod client calls: replace(k1, value=B, old-version=1)
>> T2: Hot Rod client calls: replace(k1, value=B, old-version=1)
>> T1: Server calls getCacheEntry and retrieves value=A,version=1
>> T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=A, new-value=B)
>> T1: Server updates value to B but version is still 1
>> T2: Server calls getCacheEntry and retrieves value=B,version=1 (wrong!)
>> T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=B, new-value=B)
>> T1: Server updates version to 2 
>> T1: Returns Success, replace worked
>> T2: Returns Success, replace worked
>> 
>> The end result is that cache contained B, but both replaces returned true. This is wrong and would fail to consistenly keep a counter in the value part. Imagine the value being a counter of number of times the replace succeeded. In the test developed by Takayoshi, N times replace() would return true, but the final value would be N-1 or N-2, or N-5 :|
>> 
>> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag.
> 
> so the entire underlaying cache needs to be transactional then?

Yeah, it needs to be transactional, but the code I?ve written also deals with the fact that the cache might not be transactional. I?ll probably add a WARN message when it?s not transactional. This goes in hand with the recommendations for ISPN-2956, whereby failover for conditional operations relying on return values require transactional caches to properly deal with failover situations.

To sum up, if using conditional operations or CRUD methods with Flag.FORCE_RETURN_VALUE, caches should be transactional. Moreover, to achieve concurrency guarantees of counter tests such as the one tested for 4424, locking needs to be pessimistic too. If not using conditional operations or CRUD methods without the flag, the cache could be non-transactional.

> 
>> This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
>> 
>> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
>> 
>> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
>> 
>> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].
>> 
>> Finally, this was not a problem when the value stored in Hot Rod was a ByteArrayValue wrapping the byte array and the version, because the value was treated atomically, and in hindsight, maybe adding getCacheEntry might have been premature, but this method has proven useful for other use cases too (rolling upgrades?etc).
>> 
>> Cheers,
>> 
>> [1] https://issues.jboss.org/browse/ISPN-4424
>> [2] https://issues.jboss.org/browse/ISPN-2956
>> [3] https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
>> [4] https://github.com/galderz/infinispan/tree/t_4424
>> [5] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> Cheers,
> -- 
> Mircea Markus
> Infinispan lead (www.infinispan.org)
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From mmarkus at redhat.com  Tue Jun 24 12:11:41 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Tue, 24 Jun 2014 17:11:41 +0100
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
	<F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>
	<47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>
Message-ID: <0DD54957-976C-4985-A1A8-E8ED7DC6B320@redhat.com>


On Jun 24, 2014, at 16:50, Galder Zamarre?o <galder at redhat.com> wrote:

> 
> On 24 Jun 2014, at 16:51, Mircea Markus <mmarkus at redhat.com> wrote:
> 
>> 
>> On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com> wrote:
>> 
>>> Hi all,
>>> 
>>> Last few days I?ve been having fun fixing [1] and the solution I?ve got to has some links to [2].
>>> 
>>> The problem is that when getCacheEntry() is called, the entry returned could be partially correct. Up until now, we?d send back a cache entry back to the client that the user could not modify, but it was linked to the data container cache entry, which is mutable and hence could change on the fly. As a result of this, the value could sometimes be updated but the version would belong to a previous value for example.
>>> 
>>> A first attempt to fix it was to provide a true immutable cache entry view which was initialised on construction, but even here there was the chance of having value and version mistmatch, because getCacheEntry does not acquire locks, so an ongoing update could result in the cache entry being constructed when the update was half way, so, it would have the right value but the old version information.
>>> 
>>> All this didn?t work well with the replaceIfUmodified logic in [3]. For example, you could easily get this situation:
>>> 
>>> Current Cache contents: key=k1, value=A, version=1
>>> 
>>> T1: Hot Rod client calls: replace(k1, value=B, old-version=1)
>>> T2: Hot Rod client calls: replace(k1, value=B, old-version=1)
>>> T1: Server calls getCacheEntry and retrieves value=A,version=1
>>> T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=A, new-value=B)
>>> T1: Server updates value to B but version is still 1
>>> T2: Server calls getCacheEntry and retrieves value=B,version=1 (wrong!)
>>> T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=B, new-value=B)
>>> T1: Server updates version to 2 
>>> T1: Returns Success, replace worked
>>> T2: Returns Success, replace worked
>>> 
>>> The end result is that cache contained B, but both replaces returned true. This is wrong and would fail to consistenly keep a counter in the value part. Imagine the value being a counter of number of times the replace succeeded. In the test developed by Takayoshi, N times replace() would return true, but the final value would be N-1 or N-2, or N-5 :|
>>> 
>>> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag.
>> 
>> so the entire underlaying cache needs to be transactional then?
> 
> Yeah, it needs to be transactional, but the code I?ve written also deals with the fact that the cache might not be transactional. I?ll probably add a WARN message when it?s not transactional. This goes in hand with the recommendations for ISPN-2956, whereby failover for conditional operations relying on return values require transactional caches to properly deal with failover situations.
> 
> To sum up, if using conditional operations or CRUD methods with Flag.FORCE_RETURN_VALUE, caches should be transactional. Moreover, to achieve concurrency guarantees of counter tests such as the one tested for 4424, locking needs to be pessimistic too. If not using conditional operations or CRUD methods without the flag, the cache could be non-transactional.

Thanks for the analysis. I think we should go with your patch for ISPN 7.0 and consider the proper solution for the future, as you suggest below. 
+1 for the warning, users should be made aware for the limitation.
> 
>> 
>>> This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
>>> 
>>> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
>>> 
>>> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
>>> 
>>> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].


Shall we raise a new JIRA for the permanent solution then?

>>> 
>>> Finally, this was not a problem when the value stored in Hot Rod was a ByteArrayValue wrapping the byte array and the version, because the value was treated atomically, and in hindsight, maybe adding getCacheEntry might have been premature, but this method has proven useful for other use cases too (rolling upgrades?etc).
>>> 
>>> Cheers,
>>> 
>>> [1] https://issues.jboss.org/browse/ISPN-4424
>>> [2] https://issues.jboss.org/browse/ISPN-2956
>>> [3] https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
>>> [4] https://github.com/galderz/infinispan/tree/t_4424
>>> [5] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>> twitter.com/galderz
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> Cheers,
>> -- 
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From pedro at infinispan.org  Tue Jun 24 13:51:49 2014
From: pedro at infinispan.org (Pedro Ruivo)
Date: Tue, 24 Jun 2014 18:51:49 +0100
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <0DD54957-976C-4985-A1A8-E8ED7DC6B320@redhat.com>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>	<F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>	<47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>
	<0DD54957-976C-4985-A1A8-E8ED7DC6B320@redhat.com>
Message-ID: <53A9BAB5.9080102@infinispan.org>



On 06/24/2014 05:11 PM, Mircea Markus wrote:
>
> On Jun 24, 2014, at 16:50, Galder Zamarre?o <galder at redhat.com> wrote:
>
>>
>> On 24 Jun 2014, at 16:51, Mircea Markus <mmarkus at redhat.com> wrote:
>>
>>>
>>> On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com> wrote:
>>>
>>>> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag.
>>>
>>> so the entire underlaying cache needs to be transactional then?
>>
>> Yeah, it needs to be transactional, but the code I?ve written also deals with the fact that the cache might not be transactional. I?ll probably add a WARN message when it?s not transactional. This goes in hand with the recommendations for ISPN-2956, whereby failover for conditional operations relying on return values require transactional caches to properly deal with failover situations.
>>
>> To sum up, if using conditional operations or CRUD methods with Flag.FORCE_RETURN_VALUE, caches should be transactional. Moreover, to achieve concurrency guarantees of counter tests such as the one tested for 4424, locking needs to be pessimistic too. If not using conditional operations or CRUD methods without the flag, the cache could be non-transactional.
>
> Thanks for the analysis. I think we should go with your patch for ISPN 7.0 and consider the proper solution for the future, as you suggest below.
> +1 for the warning, users should be made aware for the limitation.

+1 for the patch.

Another suggestion: in *InternalEntryFactory.update()*, you can 
synchronize in the cache entry, and create a new method 
*InternalCacheEntry copy(InternalCacheEntry)* that makes a copy while it 
also synchronizes in the existing cache entry. I think in this way, you 
don't need the cache to be transactional neither to force the lock on 
reads. Also, I would suggest the copy() to be invoked in your case (or 
in the conditional commands accesses to DataContainer?).

>>
>>>
>>>> This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
>>>>
>>>> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
>>>>
>>>> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
>>>>
>>>> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].
>
>
> Shall we raise a new JIRA for the permanent solution then?

+1

I think the immutable entries would be better, but it will destroy 
ReadCommitted isolation. The ReadCommitted is based on the mutability of 
the entry (i.e. it will not invoke the data container if the entry 
exists in transaction context). When (and if) is removed, then we can 
make the entries immutable.

>
>>>>
>>>> Finally, this was not a problem when the value stored in Hot Rod was a ByteArrayValue wrapping the byte array and the version, because the value was treated atomically, and in hindsight, maybe adding getCacheEntry might have been premature, but this method has proven useful for other use cases too (rolling upgrades?etc).
>>>>
>>>> Cheers,
>>>>
>>>> [1] https://issues.jboss.org/browse/ISPN-4424
>>>> [2] https://issues.jboss.org/browse/ISPN-2956
>>>> [3] https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
>>>> [4] https://github.com/galderz/infinispan/tree/t_4424
>>>> [5] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
>>>> --
>>>> Galder Zamarre?o
>>>> galder at redhat.com
>>>> twitter.com/galderz
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> Cheers,
>>> --
>>> Mircea Markus
>>> Infinispan lead (www.infinispan.org)
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> Cheers,
>

From mudokonman at gmail.com  Tue Jun 24 14:04:18 2014
From: mudokonman at gmail.com (William Burns)
Date: Tue, 24 Jun 2014 14:04:18 -0400
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <53A9BAB5.9080102@infinispan.org>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
	<F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>
	<47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>
	<0DD54957-976C-4985-A1A8-E8ED7DC6B320@redhat.com>
	<53A9BAB5.9080102@infinispan.org>
Message-ID: <CA+YCuUOX-Mv2AGE7ShE=1aU=m49+fWsMzXk+h6ND0pvKS5o83w@mail.gmail.com>

I must admit this seems a bit heavy handed to have to enable
transactions, when are using them solely for the purpose of having
implicit transactions.

Can we not instead just tweak the NonTransactionalLockingInterceptor
to obey the FORCE_WRITE_LOCK, so it would guarantee a consistent value
in the face of a concurrent write when doing getCacheEntry?

Although thinking again on the locking, I don't think it alone is
sufficient either.  As the cache entry is serialized after releasing
the lock, which means there is still a window when only the value may
be changed on an owner node.  We really need immutable CacheEntries
returned from getCacheEntry even with locking to work properly.

 - Will

On Tue, Jun 24, 2014 at 1:51 PM, Pedro Ruivo <pedro at infinispan.org> wrote:
>
>
> On 06/24/2014 05:11 PM, Mircea Markus wrote:
>>
>> On Jun 24, 2014, at 16:50, Galder Zamarre?o <galder at redhat.com> wrote:
>>
>>>
>>> On 24 Jun 2014, at 16:51, Mircea Markus <mmarkus at redhat.com> wrote:
>>>
>>>>
>>>> On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com> wrote:
>>>>
>>>>> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag.
>>>>
>>>> so the entire underlaying cache needs to be transactional then?
>>>
>>> Yeah, it needs to be transactional, but the code I?ve written also deals with the fact that the cache might not be transactional. I?ll probably add a WARN message when it?s not transactional. This goes in hand with the recommendations for ISPN-2956, whereby failover for conditional operations relying on return values require transactional caches to properly deal with failover situations.
>>>
>>> To sum up, if using conditional operations or CRUD methods with Flag.FORCE_RETURN_VALUE, caches should be transactional. Moreover, to achieve concurrency guarantees of counter tests such as the one tested for 4424, locking needs to be pessimistic too. If not using conditional operations or CRUD methods without the flag, the cache could be non-transactional.
>>
>> Thanks for the analysis. I think we should go with your patch for ISPN 7.0 and consider the proper solution for the future, as you suggest below.
>> +1 for the warning, users should be made aware for the limitation.
>
> +1 for the patch.
>
> Another suggestion: in *InternalEntryFactory.update()*, you can

I don't know if that would cover all the places since we also set the
value in the various WriteCommands themselves.

> synchronize in the cache entry, and create a new method
> *InternalCacheEntry copy(InternalCacheEntry)* that makes a copy while it
> also synchronizes in the existing cache entry. I think in this way, you
> don't need the cache to be transactional neither to force the lock on
> reads. Also, I would suggest the copy() to be invoked in your case (or
> in the conditional commands accesses to DataContainer?).

Yeah I was thinking of this approach as well, we would either have to
make the copy or synchronize in the serialization.  I personally think
making a copy would be better, but the entire copy operation would
have to be synchronized then as you mentioned.

>
>>>
>>>>
>>>>> This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
>>>>>
>>>>> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
>>>>>
>>>>> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
>>>>>
>>>>> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].
>>
>>
>> Shall we raise a new JIRA for the permanent solution then?
>
> +1
>
> I think the immutable entries would be better, but it will destroy
> ReadCommitted isolation. The ReadCommitted is based on the mutability of
> the entry (i.e. it will not invoke the data container if the entry
> exists in transaction context). When (and if) is removed, then we can
> make the entries immutable.

If we only make getCacheEntry immutable, it wouldn't affect the normal
get operations.  I was thinking we could probably make the copy only
in the GetKeyValueCommand.

>
>>
>>>>>
>>>>> Finally, this was not a problem when the value stored in Hot Rod was a ByteArrayValue wrapping the byte array and the version, because the value was treated atomically, and in hindsight, maybe adding getCacheEntry might have been premature, but this method has proven useful for other use cases too (rolling upgrades?etc).
>>>>>
>>>>> Cheers,
>>>>>
>>>>> [1] https://issues.jboss.org/browse/ISPN-4424
>>>>> [2] https://issues.jboss.org/browse/ISPN-2956
>>>>> [3] https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
>>>>> [4] https://github.com/galderz/infinispan/tree/t_4424
>>>>> [5] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
>>>>> --
>>>>> Galder Zamarre?o
>>>>> galder at redhat.com
>>>>> twitter.com/galderz
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> Cheers,
>>>> --
>>>> Mircea Markus
>>>> Infinispan lead (www.infinispan.org)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>> twitter.com/galderz
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> Cheers,
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From dan.berindei at gmail.com  Wed Jun 25 02:45:13 2014
From: dan.berindei at gmail.com (Dan Berindei)
Date: Wed, 25 Jun 2014 09:45:13 +0300
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <CA+YCuUOX-Mv2AGE7ShE=1aU=m49+fWsMzXk+h6ND0pvKS5o83w@mail.gmail.com>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
	<F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>
	<47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>
	<0DD54957-976C-4985-A1A8-E8ED7DC6B320@redhat.com>
	<53A9BAB5.9080102@infinispan.org>
	<CA+YCuUOX-Mv2AGE7ShE=1aU=m49+fWsMzXk+h6ND0pvKS5o83w@mail.gmail.com>
Message-ID: <CA+nfvwRPa6SvXOeGM9a_HU27+jXEQdV_stJzjBf34-vqUp08gQ@mail.gmail.com>

On Tue, Jun 24, 2014 at 9:04 PM, William Burns <mudokonman at gmail.com> wrote:

> I must admit this seems a bit heavy handed to have to enable
> transactions, when are using them solely for the purpose of having
> implicit transactions.
>
> Can we not instead just tweak the NonTransactionalLockingInterceptor
> to obey the FORCE_WRITE_LOCK, so it would guarantee a consistent value
> in the face of a concurrent write when doing getCacheEntry?
>
>
I suggested that as well, but ISPN-2956 means we can't guarantee the
atomicity of non-tx conditional operations anyway.
And HotRod doesn't work nicely with optimistic locking (yet), so we have to
require pessimistic locking. I'm not sure about total order, though.


> Although thinking again on the locking, I don't think it alone is
> sufficient either.  As the cache entry is serialized after releasing
> the lock, which means there is still a window when only the value may
> be changed on an owner node.  We really need immutable CacheEntries
> returned from getCacheEntry even with locking to work properly.
>

Galder didn't mention this, but his proposal also copies the entry in
GetKeyValueCommand.perform, so the serialization happens on an
ImmutableCacheEntryView.


>  - Will
>
> On Tue, Jun 24, 2014 at 1:51 PM, Pedro Ruivo <pedro at infinispan.org> wrote:
> >
> >
> > On 06/24/2014 05:11 PM, Mircea Markus wrote:
> >>
> >> On Jun 24, 2014, at 16:50, Galder Zamarre?o <galder at redhat.com> wrote:
> >>
> >>>
> >>> On 24 Jun 2014, at 16:51, Mircea Markus <mmarkus at redhat.com> wrote:
> >>>
> >>>>
> >>>> On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com>
> wrote:
> >>>>
> >>>>> To fix this, I?ve been working with Dan on some solutions and we?ve
> taken inspiration of the new requirements appearing as a result of
> ISPN-2956. To be able to deal with partial application of conditional
> operations properly, transactional caches are needed. So, the solution that
> can be seen in [4] takes that, and creates a transaction around the
> replaceIfUmodified and forces the getCacheEntry() call to acquire lock via
> FORCE_WRITE_LOCK flag.
> >>>>
> >>>> so the entire underlaying cache needs to be transactional then?
> >>>
> >>> Yeah, it needs to be transactional, but the code I?ve written also
> deals with the fact that the cache might not be transactional. I?ll
> probably add a WARN message when it?s not transactional. This goes in hand
> with the recommendations for ISPN-2956, whereby failover for conditional
> operations relying on return values require transactional caches to
> properly deal with failover situations.
> >>>
> >>> To sum up, if using conditional operations or CRUD methods with
> Flag.FORCE_RETURN_VALUE, caches should be transactional. Moreover, to
> achieve concurrency guarantees of counter tests such as the one tested for
> 4424, locking needs to be pessimistic too. If not using conditional
> operations or CRUD methods without the flag, the cache could be
> non-transactional.
> >>
> >> Thanks for the analysis. I think we should go with your patch for ISPN
> 7.0 and consider the proper solution for the future, as you suggest below.
> >> +1 for the warning, users should be made aware for the limitation.
> >
> > +1 for the patch.
> >
> > Another suggestion: in *InternalEntryFactory.update()*, you can
>
> I don't know if that would cover all the places since we also set the
> value in the various WriteCommands themselves.
>

I think the WriteCommands only modify wrapped entries.

It would be really bad if a put(k, v) in a read-committed tx cache could
modify the cache entry before the tx is committed. Making this distinction
clear would be another reason to make InternalCacheEntry immutable...


> > synchronize in the cache entry, and create a new method
> > *InternalCacheEntry copy(InternalCacheEntry)* that makes a copy while it
> > also synchronizes in the existing cache entry. I think in this way, you
> > don't need the cache to be transactional neither to force the lock on
> > reads. Also, I would suggest the copy() to be invoked in your case (or
> > in the conditional commands accesses to DataContainer?).
>

Nitpicking: copy() only exists in the CopyableDeltaAware interface, and the
commands themselves should not access the DataContainer.

This sounds like it might work, and in fact it's quite similar to
Takayoshi's initial proposal (though he targeted it specifically to
HotRod's replaceIfUnmodified). It's a bit scary because
CacheEntry.setValue() is called in so many places, but I think everything
but InternalEntryFactoryImpl.update() should be working on context entries,
so they don't need synchronization. Still, ISPN-2956...


> Yeah I was thinking of this approach as well, we would either have to
> make the copy or synchronize in the serialization.  I personally think
> making a copy would be better, but the entire copy operation would
> have to be synchronized then as you mentioned.
>
> >
> >>>
> >>>>
> >>>>> This solves the issue explained above, but of course it has an
> impact of the performance. The test now runs about 1.5 or 2 times slower.
> >>>>>
> >>>>> This is probably the best that we can do in the 7.0 time frame, but
> there?s several things that could improve this:
> >>>>>
> >>>>> 1. True immutable entries in the cache. If the entries in the cache
> were truly immutable, there would be no risk of sending back a partially
> correct entry back to the client.
> >>>>>
> >>>>> 2. A cache replace method that does not compare objects based on
> equality (the current replace()), but a replace method that takes a
> function. The function could compare that the old entry?s version and the
> cached entry?s version match. The function would only be executed inside
> the inner container, with all the locks held properly. I already hinted
> something similar in [5].
> >>
> >>
> >> Shall we raise a new JIRA for the permanent solution then?
> >
> > +1
> >
> > I think the immutable entries would be better, but it will destroy
> > ReadCommitted isolation. The ReadCommitted is based on the mutability of
> > the entry (i.e. it will not invoke the data container if the entry
> > exists in transaction context). When (and if) is removed, then we can
> > make the entries immutable.
>

Agree, it would be a significant change. Perhaps context entries in
read-committed caches could hold a reference to the actual
EquivalentConcurrentHashMapV8.MapEntry if the entry was local, but that
would add more complexity. So I would rather see it removed completely.


>
> If we only make getCacheEntry immutable, it wouldn't affect the normal
> get operations.  I was thinking we could probably make the copy only
> in the GetKeyValueCommand.
>

It would still affect every write operation. And we would need a way to
distinguish between GetKeyValueCommands for getCacheEntry and generated by
ClusteredGetCommands, since both kinds return InternalCacheEntries.

On the other hand, I think we have the same race with the write skew check
in optimistic caches. We read the value and version for write skew checking
without holding the key lock, so in theory it's possible to read an
outdated value and the current version. So we need some kind of
synchronization when reading the entry for optimistic mode, anyway.

But instead of a synchronized block, I propose making the value and
metadata references volatile. It would require care to always read the
metadata first and write it last, but reads would still be "free" on x86.
HotRod would still need transactions, but it wouldn't need
FORCE_WRITE_LOCK, so this might be a little faster than Galder's version.


>
> >
> >>
> >>>>>
> >>>>> Finally, this was not a problem when the value stored in Hot Rod was
> a ByteArrayValue wrapping the byte array and the version, because the value
> was treated atomically, and in hindsight, maybe adding getCacheEntry might
> have been premature, but this method has proven useful for other use cases
> too (rolling upgrades?etc).
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> [1] https://issues.jboss.org/browse/ISPN-4424
> >>>>> [2] https://issues.jboss.org/browse/ISPN-2956
> >>>>> [3]
> https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
> >>>>> [4] https://github.com/galderz/infinispan/tree/t_4424
> >>>>> [5]
> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
> >>>>> --
> >>>>> Galder Zamarre?o
> >>>>> galder at redhat.com
> >>>>> twitter.com/galderz
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> infinispan-dev mailing list
> >>>>> infinispan-dev at lists.jboss.org
> >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>>
> >>>> Cheers,
> >>>> --
> >>>> Mircea Markus
> >>>> Infinispan lead (www.infinispan.org)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>> --
> >>> Galder Zamarre?o
> >>> galder at redhat.com
> >>> twitter.com/galderz
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >> Cheers,
> >>
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140625/08784bc5/attachment-0001.html 

From galder at redhat.com  Wed Jun 25 03:12:25 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Wed, 25 Jun 2014 09:12:25 +0200
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
Message-ID: <70A5496E-2891-47BE-9D38-0043520C6053@redhat.com>

FYI: Takayoshi has run some further testing and found that with 40 threads there was still some deviation, so it needs further investigation.

On 24 Jun 2014, at 16:27, Galder Zamarre?o <galder at redhat.com> wrote:

> Hi all,
> 
> Last few days I?ve been having fun fixing [1] and the solution I?ve got to has some links to [2].
> 
> The problem is that when getCacheEntry() is called, the entry returned could be partially correct. Up until now, we?d send back a cache entry back to the client that the user could not modify, but it was linked to the data container cache entry, which is mutable and hence could change on the fly. As a result of this, the value could sometimes be updated but the version would belong to a previous value for example.
> 
> A first attempt to fix it was to provide a true immutable cache entry view which was initialised on construction, but even here there was the chance of having value and version mistmatch, because getCacheEntry does not acquire locks, so an ongoing update could result in the cache entry being constructed when the update was half way, so, it would have the right value but the old version information.
> 
> All this didn?t work well with the replaceIfUmodified logic in [3]. For example, you could easily get this situation:
> 
> Current Cache contents: key=k1, value=A, version=1
> 
> T1: Hot Rod client calls: replace(k1, value=B, old-version=1)
> T2: Hot Rod client calls: replace(k1, value=B, old-version=1)
> T1: Server calls getCacheEntry and retrieves value=A,version=1
> T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=A, new-value=B)
> T1: Server updates value to B but version is still 1
> T2: Server calls getCacheEntry and retrieves value=B,version=1 (wrong!)
> T1: Cached and stream versions match (both are 1), so call replace(k1, old-value=B, new-value=B)
> T1: Server updates version to 2 
> T1: Returns Success, replace worked
> T2: Returns Success, replace worked
> 
> The end result is that cache contained B, but both replaces returned true. This is wrong and would fail to consistenly keep a counter in the value part. Imagine the value being a counter of number of times the replace succeeded. In the test developed by Takayoshi, N times replace() would return true, but the final value would be N-1 or N-2, or N-5 :|
> 
> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag. This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
> 
> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
> 
> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
> 
> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].
> 
> Finally, this was not a problem when the value stored in Hot Rod was a ByteArrayValue wrapping the byte array and the version, because the value was treated atomically, and in hindsight, maybe adding getCacheEntry might have been premature, but this method has proven useful for other use cases too (rolling upgrades?etc).
> 
> Cheers,
> 
> [1] https://issues.jboss.org/browse/ISPN-4424
> [2] https://issues.jboss.org/browse/ISPN-2956
> [3] https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
> [4] https://github.com/galderz/infinispan/tree/t_4424
> [5] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From galder at redhat.com  Wed Jun 25 03:33:55 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Wed, 25 Jun 2014 09:33:55 +0200
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <CA+nfvwRPa6SvXOeGM9a_HU27+jXEQdV_stJzjBf34-vqUp08gQ@mail.gmail.com>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>
	<F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>
	<47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>
	<0DD54957-976C-4985-A1A8-E8ED7DC6B320@redhat.com>
	<53A9BAB5.9080102@infinispan.org>
	<CA+YCuUOX-Mv2AGE7ShE=1aU=m49+fWsMzXk+h6ND0pvKS5o83w@mail.gmail.com>
	<CA+nfvwRPa6SvXOeGM9a_HU27+jXEQdV_stJzjBf34-vqUp08gQ@mail.gmail.com>
Message-ID: <EDF65B97-FF5B-4203-A9E3-63D53EEDE23E@redhat.com>


On 25 Jun 2014, at 08:45, Dan Berindei <dan.berindei at gmail.com> wrote:

> 
> 
> 
> On Tue, Jun 24, 2014 at 9:04 PM, William Burns <mudokonman at gmail.com> wrote:
> I must admit this seems a bit heavy handed to have to enable
> transactions, when are using them solely for the purpose of having
> implicit transactions.
> 
> Can we not instead just tweak the NonTransactionalLockingInterceptor
> to obey the FORCE_WRITE_LOCK, so it would guarantee a consistent value
> in the face of a concurrent write when doing getCacheEntry?
> 
> 
> I suggested that as well, but ISPN-2956 means we can't guarantee the atomicity of non-tx conditional operations anyway.
> And HotRod doesn't work nicely with optimistic locking (yet), so we have to require pessimistic locking. I'm not sure about total order, though.

Indeed, I agree that transactions is a bit heavy handed, but the fact that transactions are needed to deal with ISPN-2956 properly, made me lean that way.

> Although thinking again on the locking, I don't think it alone is
> sufficient either.  As the cache entry is serialized after releasing
> the lock, which means there is still a window when only the value may
> be changed on an owner node.  We really need immutable CacheEntries
> returned from getCacheEntry even with locking to work properly.
> 
> Galder didn't mention this, but his proposal also copies the entry in GetKeyValueCommand.perform, so the serialization happens on an ImmutableCacheEntryView.

Yeah, ImmutableCacheEntryView effectively caches the cache entry?s values at construction time. Assuming there?s lock in place, that should be safe.

The thing to note here is that replaceIfUmodified() needs both the version and value part to be correct, hence why we added the transaction and force write lock, to avoid one not matching up with the other. The other piece of the puzzle is getVersioned() operation, that the client calls to find an entry?s version and use that. Since the only thing sent to the server is the version (and not the value), I?ve stayed away from making getVersioned() calling getCacheEntry with FORCE_WRITE_LOCK and a transaction.

Cheers,

> 
> 
>  - Will
> 
> On Tue, Jun 24, 2014 at 1:51 PM, Pedro Ruivo <pedro at infinispan.org> wrote:
> >
> >
> > On 06/24/2014 05:11 PM, Mircea Markus wrote:
> >>
> >> On Jun 24, 2014, at 16:50, Galder Zamarre?o <galder at redhat.com> wrote:
> >>
> >>>
> >>> On 24 Jun 2014, at 16:51, Mircea Markus <mmarkus at redhat.com> wrote:
> >>>
> >>>>
> >>>> On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com> wrote:
> >>>>
> >>>>> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag.
> >>>>
> >>>> so the entire underlaying cache needs to be transactional then?
> >>>
> >>> Yeah, it needs to be transactional, but the code I?ve written also deals with the fact that the cache might not be transactional. I?ll probably add a WARN message when it?s not transactional. This goes in hand with the recommendations for ISPN-2956, whereby failover for conditional operations relying on return values require transactional caches to properly deal with failover situations.
> >>>
> >>> To sum up, if using conditional operations or CRUD methods with Flag.FORCE_RETURN_VALUE, caches should be transactional. Moreover, to achieve concurrency guarantees of counter tests such as the one tested for 4424, locking needs to be pessimistic too. If not using conditional operations or CRUD methods without the flag, the cache could be non-transactional.
> >>
> >> Thanks for the analysis. I think we should go with your patch for ISPN 7.0 and consider the proper solution for the future, as you suggest below.
> >> +1 for the warning, users should be made aware for the limitation.
> >
> > +1 for the patch.
> >
> > Another suggestion: in *InternalEntryFactory.update()*, you can
> 
> I don't know if that would cover all the places since we also set the
> value in the various WriteCommands themselves.
> 
> I think the WriteCommands only modify wrapped entries. 
> 
> It would be really bad if a put(k, v) in a read-committed tx cache could modify the cache entry before the tx is committed. Making this distinction clear would be another reason to make InternalCacheEntry immutable...
> 
> 
> > synchronize in the cache entry, and create a new method
> > *InternalCacheEntry copy(InternalCacheEntry)* that makes a copy while it
> > also synchronizes in the existing cache entry. I think in this way, you
> > don't need the cache to be transactional neither to force the lock on
> > reads. Also, I would suggest the copy() to be invoked in your case (or
> > in the conditional commands accesses to DataContainer?).
> 
> Nitpicking: copy() only exists in the CopyableDeltaAware interface, and the commands themselves should not access the DataContainer.
> 
> This sounds like it might work, and in fact it's quite similar to Takayoshi's initial proposal (though he targeted it specifically to HotRod's replaceIfUnmodified). It's a bit scary because CacheEntry.setValue() is called in so many places, but I think everything but InternalEntryFactoryImpl.update() should be working on context entries, so they don't need synchronization. Still, ISPN-2956...
> 
> 
> Yeah I was thinking of this approach as well, we would either have to
> make the copy or synchronize in the serialization.  I personally think
> making a copy would be better, but the entire copy operation would
> have to be synchronized then as you mentioned.
> 
> >
> >>>
> >>>>
> >>>>> This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
> >>>>>
> >>>>> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
> >>>>>
> >>>>> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
> >>>>>
> >>>>> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].
> >>
> >>
> >> Shall we raise a new JIRA for the permanent solution then?
> >
> > +1
> >
> > I think the immutable entries would be better, but it will destroy
> > ReadCommitted isolation. The ReadCommitted is based on the mutability of
> > the entry (i.e. it will not invoke the data container if the entry
> > exists in transaction context). When (and if) is removed, then we can
> > make the entries immutable.
> 
> Agree, it would be a significant change. Perhaps context entries in read-committed caches could hold a reference to the actual EquivalentConcurrentHashMapV8.MapEntry if the entry was local, but that would add more complexity. So I would rather see it removed completely.
>  
> 
> If we only make getCacheEntry immutable, it wouldn't affect the normal
> get operations.  I was thinking we could probably make the copy only
> in the GetKeyValueCommand.
> 
> It would still affect every write operation. And we would need a way to distinguish between GetKeyValueCommands for getCacheEntry and generated by ClusteredGetCommands, since both kinds return InternalCacheEntries.
> 
> On the other hand, I think we have the same race with the write skew check in optimistic caches. We read the value and version for write skew checking without holding the key lock, so in theory it's possible to read an outdated value and the current version. So we need some kind of synchronization when reading the entry for optimistic mode, anyway. 
> 
> But instead of a synchronized block, I propose making the value and metadata references volatile. It would require care to always read the metadata first and write it last, but reads would still be "free" on x86. HotRod would still need transactions, but it wouldn't need FORCE_WRITE_LOCK, so this might be a little faster than Galder's version.
>  
> 
> >
> >>
> >>>>>
> >>>>> Finally, this was not a problem when the value stored in Hot Rod was a ByteArrayValue wrapping the byte array and the version, because the value was treated atomically, and in hindsight, maybe adding getCacheEntry might have been premature, but this method has proven useful for other use cases too (rolling upgrades?etc).
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> [1] https://issues.jboss.org/browse/ISPN-4424
> >>>>> [2] https://issues.jboss.org/browse/ISPN-2956
> >>>>> [3] https://github.com/infinispan/infinispan/blob/master/server/core/src/main/scala/org/infinispan/server/core/AbstractProtocolDecoder.scala#L302
> >>>>> [4] https://github.com/galderz/infinispan/tree/t_4424
> >>>>> [5] https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/AdvancedCache.java#L374
> >>>>> --
> >>>>> Galder Zamarre?o
> >>>>> galder at redhat.com
> >>>>> twitter.com/galderz
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> infinispan-dev mailing list
> >>>>> infinispan-dev at lists.jboss.org
> >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>>
> >>>> Cheers,
> >>>> --
> >>>> Mircea Markus
> >>>> Infinispan lead (www.infinispan.org)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>
> >>>
> >>> --
> >>> Galder Zamarre?o
> >>> galder at redhat.com
> >>> twitter.com/galderz
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >> Cheers,
> >>
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From galder at redhat.com  Wed Jun 25 04:21:33 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Wed, 25 Jun 2014 10:21:33 +0200
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <53A9BAB5.9080102@infinispan.org>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>	<F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>	<47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>
	<0DD54957-976C-4985-A1A8-E8ED7DC6B320@redhat.com>
	<53A9BAB5.9080102@infinispan.org>
Message-ID: <8046734C-4ED0-4D79-A2C7-30DAEFC81747@redhat.com>


On 24 Jun 2014, at 19:51, Pedro Ruivo <pedro at infinispan.org> wrote:

> 
> 
> On 06/24/2014 05:11 PM, Mircea Markus wrote:
>> 
>> On Jun 24, 2014, at 16:50, Galder Zamarre?o <galder at redhat.com> wrote:
>> 
>>> 
>>> On 24 Jun 2014, at 16:51, Mircea Markus <mmarkus at redhat.com> wrote:
>>> 
>>>> 
>>>> On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com> wrote:
>>>> 
>>>>> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag.
>>>> 
>>>> so the entire underlaying cache needs to be transactional then?
>>> 
>>> Yeah, it needs to be transactional, but the code I?ve written also deals with the fact that the cache might not be transactional. I?ll probably add a WARN message when it?s not transactional. This goes in hand with the recommendations for ISPN-2956, whereby failover for conditional operations relying on return values require transactional caches to properly deal with failover situations.
>>> 
>>> To sum up, if using conditional operations or CRUD methods with Flag.FORCE_RETURN_VALUE, caches should be transactional. Moreover, to achieve concurrency guarantees of counter tests such as the one tested for 4424, locking needs to be pessimistic too. If not using conditional operations or CRUD methods without the flag, the cache could be non-transactional.
>> 
>> Thanks for the analysis. I think we should go with your patch for ISPN 7.0 and consider the proper solution for the future, as you suggest below.
>> +1 for the warning, users should be made aware for the limitation.
> 
> +1 for the patch.
> 
> Another suggestion: in *InternalEntryFactory.update()*, you can 
> synchronize in the cache entry, and create a new method 
> *InternalCacheEntry copy(InternalCacheEntry)* that makes a copy while it 
> also synchronizes in the existing cache entry. I think in this way, you 
> don't need the cache to be transactional neither to force the lock on 
> reads. Also, I would suggest the copy() to be invoked in your case (or 
> in the conditional commands accesses to DataContainer?).

^ This is an interesting idea too. While transactions are necessary to combat ISPN-2956, fixing this via transactions, as you all pointed out, might be a bit too much and also might hurt performance wise.

I?m doing further testing to see if I see Takayoshi?s problems locally. In the mean time, I?ll try to prototype an alternative solution based around this.

> 
>>> 
>>>> 
>>>>> This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
>>>>> 
>>>>> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
>>>>> 
>>>>> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
>>>>> 
>>>>> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].
>> 
>> 
>> Shall we raise a new JIRA for the permanent solution then?
> 
> +1
> 
> I think the immutable entries would be better, but it will destroy 
> ReadCommitted isolation. The ReadCommitted is based on the mutability of 
> the entry (i.e. it will not invoke the data container if the entry 
> exists in transaction context). When (and if) is removed, then we can 
> make the entries immutable.

Yeah, I think RC is the biggest barrier right now. 

Thanks everyone for their input so far!! :D

Cheers,
--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From wfink at redhat.com  Wed Jun 25 07:08:53 2014
From: wfink at redhat.com (Wolf-Dieter Fink)
Date: Wed, 25 Jun 2014 13:08:53 +0200
Subject: [infinispan-dev] Can not compile current upstream because of test
	failures
Message-ID: <53AAADC5.8050702@redhat.com>

Hi,

I use latest upstream of git at github.com:infinispan/infinispan.git
   commit bc949a3acb354ff9ac3202b450c521dc6740b415
   Author: Martin Gencur <mgencur at redhat.com>
   Date:   Thu Jun 19 15:59:27 2014 +0200


and the build failed "build.sh clean install", see below.
Is there a general problem with that tests? Or is it a problem in my 
environment?


Also you can not build with "-Dmaven.test.skip=true" as the next module 
failed because of dependencies
[INFO] Infinispan Server - Test Suite .................... FAILURE [1.687s]
   ....
[INFO] 
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] 
------------------------------------------------------------------------
[ERROR] Failed to execute goal on project test-suite: Could not resolve 
dependencies for project 
org.infinispan.server:test-suite:jar:7.0.0-SNAPSHOT: Failure to find 
org.infinispan:infinispan-security-integrationtests:jar:tests:7.0.0-SNAPSHOT 
in http://maven.repository.redhat.com/earlyaccess/all/ was cached in the 
local repository, resolution will not be reattempted until the update 
interval of redhat-earlyaccess-repository-group has elapsed or updates 
are forced -> [Help 1]



---------  "./build.sh clean install"  failure 
---------------------------------
testReaderRemove(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
Time elapsed: 0.053 sec  <<< ERROR!
java.lang.Exception: Unexpected exception, 
expected<java.security.PrivilegedActionException> but 
was<javax.security.auth.login.LoginException>
     at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
     at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
     at 
org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
     at 
org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
     at 
javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
     at 
javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
     at javax.naming.InitialContext.init(InitialContext.java:242)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:356)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
     at 
javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
     at java.security.AccessController.doPrivileged(Native Method)
     at 
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
     at 
org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
     at 
org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)

testAdminCRUD(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
Time elapsed: 0.054 sec  <<< ERROR!
javax.security.auth.login.LoginException: Unable to create new 
InitialLdapContext
     at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
     at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
     at 
org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
     at 
org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
     at 
javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
     at 
javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
     at javax.naming.InitialContext.init(InitialContext.java:242)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:356)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
     at 
javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
     at java.security.AccessController.doPrivileged(Native Method)
     at 
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
     at 
org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
     at 
org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)

testUnauthenticatedWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
Time elapsed: 0.052 sec  <<< ERROR!
java.lang.Exception: Unexpected exception, 
expected<java.lang.SecurityException> but 
was<javax.security.auth.login.LoginException>
     at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
     at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
     at 
org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
     at 
org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
     at 
javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
     at 
javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
     at javax.naming.InitialContext.init(InitialContext.java:242)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:356)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
     at 
javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
     at java.security.AccessController.doPrivileged(Native Method)
     at 
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
     at 
org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
     at 
org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)

testWriterWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
Time elapsed: 0.053 sec  <<< ERROR!
javax.security.auth.login.LoginException: Unable to create new 
InitialLdapContext
     at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
     at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
     at 
org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
     at 
org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
     at 
javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
     at 
javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
     at javax.naming.InitialContext.init(InitialContext.java:242)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:356)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
     at 
javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
     at java.security.AccessController.doPrivileged(Native Method)
     at 
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
     at 
org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
     at 
org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)

testWriterCreateWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
Time elapsed: 0.053 sec  <<< ERROR!
javax.security.auth.login.LoginException: Unable to create new 
InitialLdapContext
     at 
sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
     at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
     at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
     at 
com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
     at 
org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
     at 
org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
     at 
javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
     at 
javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
     at javax.naming.InitialContext.init(InitialContext.java:242)
     at 
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:356)
     at 
org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
     at 
javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
     at java.security.AccessController.doPrivileged(Native Method)
     at 
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
     at 
org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
     at 
org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)

Running 
org.infinispan.test.integration.security.embedded.LdapAuthenticationIT
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.591 
sec - in 
org.infinispan.test.integration.security.embedded.LdapAuthenticationIT

Results :

Tests in error:
   KrbLdapAuthenticationIT.testUnprivilegedWrite ?  Unexpected 
exception, expecte...
   KrbLdapAuthenticationIT.testUnauthenticatedRemove ?  Unexpected 
exception, exp...
   KrbLdapAuthenticationIT.testUnprivilegedRemove ?  Unexpected 
exception, expect...
KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
? Login
   KrbLdapAuthenticationIT.testUnauthenticatedRead ?  Unexpected 
exception, expec...
   KrbLdapAuthenticationIT.testWriterRead ?  Unexpected exception, 
expected<java....
KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
? Login
   KrbLdapAuthenticationIT.testReaderWrite ?  Unexpected exception, 
expected<java...
   KrbLdapAuthenticationIT.testUnprivilegedRead ?  Unexpected exception, 
expected...
   KrbLdapAuthenticationIT.testReaderRemove ?  Unexpected exception, 
expected<jav...
KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
? Login
   KrbLdapAuthenticationIT.testUnauthenticatedWrite ?  Unexpected 
exception, expe...
KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
? Login
KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
? Login



From mmarkus at redhat.com  Wed Jun 25 07:21:44 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Wed, 25 Jun 2014 12:21:44 +0100
Subject: [infinispan-dev] cutting ISPN 7.0.0A.Alpha5 Friday
Message-ID: <69F3C637-F8E2-47D9-9828-AE1AE1792566@redhat.com>

Now that the suite is stable would be good to cut ISPN 7.0.0.Alpha5. There are still quite some PRs pending, let's focus on that for now and release on Friday.

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From galder at redhat.com  Wed Jun 25 07:57:05 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Wed, 25 Jun 2014 13:57:05 +0200
Subject: [infinispan-dev] Dangers of getCacheEntry (ISPN-4424)
In-Reply-To: <8046734C-4ED0-4D79-A2C7-30DAEFC81747@redhat.com>
References: <9A73804B-5CDC-4E47-A81B-7EE0E341DB47@redhat.com>	<F8BAED2E-FB10-40EF-9ABF-F43568BD3FB9@redhat.com>	<47232952-8DAB-436A-AB22-F832A80DF763@redhat.com>
	<0DD54957-976C-4985-A1A8-E8ED7DC6B320@redhat.com>
	<53A9BAB5.9080102@infinispan.org>
	<8046734C-4ED0-4D79-A2C7-30DAEFC81747@redhat.com>
Message-ID: <898180FA-D147-419A-8654-CE83C8D4971E@redhat.com>

Pedro?s suggestion looks good [1]

It?s passed tests both locally (20 threads, each sending 200 operations) and in a remote server (40 threads, each sending 1000 operations).

Cheers,

[1] https://github.com/infinispan/infinispan/pull/2671

On 25 Jun 2014, at 10:21, Galder Zamarre?o <galder at redhat.com> wrote:

> 
> On 24 Jun 2014, at 19:51, Pedro Ruivo <pedro at infinispan.org> wrote:
> 
>> 
>> 
>> On 06/24/2014 05:11 PM, Mircea Markus wrote:
>>> 
>>> On Jun 24, 2014, at 16:50, Galder Zamarre?o <galder at redhat.com> wrote:
>>> 
>>>> 
>>>> On 24 Jun 2014, at 16:51, Mircea Markus <mmarkus at redhat.com> wrote:
>>>> 
>>>>> 
>>>>> On Jun 24, 2014, at 15:27, Galder Zamarre?o <galder at redhat.com> wrote:
>>>>> 
>>>>>> To fix this, I?ve been working with Dan on some solutions and we?ve taken inspiration of the new requirements appearing as a result of ISPN-2956. To be able to deal with partial application of conditional operations properly, transactional caches are needed. So, the solution that can be seen in [4] takes that, and creates a transaction around the replaceIfUmodified and forces the getCacheEntry() call to acquire lock via FORCE_WRITE_LOCK flag.
>>>>> 
>>>>> so the entire underlaying cache needs to be transactional then?
>>>> 
>>>> Yeah, it needs to be transactional, but the code I?ve written also deals with the fact that the cache might not be transactional. I?ll probably add a WARN message when it?s not transactional. This goes in hand with the recommendations for ISPN-2956, whereby failover for conditional operations relying on return values require transactional caches to properly deal with failover situations.
>>>> 
>>>> To sum up, if using conditional operations or CRUD methods with Flag.FORCE_RETURN_VALUE, caches should be transactional. Moreover, to achieve concurrency guarantees of counter tests such as the one tested for 4424, locking needs to be pessimistic too. If not using conditional operations or CRUD methods without the flag, the cache could be non-transactional.
>>> 
>>> Thanks for the analysis. I think we should go with your patch for ISPN 7.0 and consider the proper solution for the future, as you suggest below.
>>> +1 for the warning, users should be made aware for the limitation.
>> 
>> +1 for the patch.
>> 
>> Another suggestion: in *InternalEntryFactory.update()*, you can 
>> synchronize in the cache entry, and create a new method 
>> *InternalCacheEntry copy(InternalCacheEntry)* that makes a copy while it 
>> also synchronizes in the existing cache entry. I think in this way, you 
>> don't need the cache to be transactional neither to force the lock on 
>> reads. Also, I would suggest the copy() to be invoked in your case (or 
>> in the conditional commands accesses to DataContainer?).
> 
> ^ This is an interesting idea too. While transactions are necessary to combat ISPN-2956, fixing this via transactions, as you all pointed out, might be a bit too much and also might hurt performance wise.
> 
> I?m doing further testing to see if I see Takayoshi?s problems locally. In the mean time, I?ll try to prototype an alternative solution based around this.
> 
>> 
>>>> 
>>>>> 
>>>>>> This solves the issue explained above, but of course it has an impact of the performance. The test now runs about 1.5 or 2 times slower.
>>>>>> 
>>>>>> This is probably the best that we can do in the 7.0 time frame, but there?s several things that could improve this:
>>>>>> 
>>>>>> 1. True immutable entries in the cache. If the entries in the cache were truly immutable, there would be no risk of sending back a partially correct entry back to the client.
>>>>>> 
>>>>>> 2. A cache replace method that does not compare objects based on equality (the current replace()), but a replace method that takes a function. The function could compare that the old entry?s version and the cached entry?s version match. The function would only be executed inside the inner container, with all the locks held properly. I already hinted something similar in [5].
>>> 
>>> 
>>> Shall we raise a new JIRA for the permanent solution then?
>> 
>> +1
>> 
>> I think the immutable entries would be better, but it will destroy 
>> ReadCommitted isolation. The ReadCommitted is based on the mutability of 
>> the entry (i.e. it will not invoke the data container if the entry 
>> exists in transaction context). When (and if) is removed, then we can 
>> make the entries immutable.
> 
> Yeah, I think RC is the biggest barrier right now. 
> 
> Thanks everyone for their input so far!! :D
> 
> Cheers,
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From galder at redhat.com  Wed Jun 25 08:01:24 2014
From: galder at redhat.com (=?iso-8859-1?Q?Galder_Zamarre=F1o?=)
Date: Wed, 25 Jun 2014 14:01:24 +0200
Subject: [infinispan-dev] cutting ISPN 7.0.0A.Alpha5 Friday
In-Reply-To: <69F3C637-F8E2-47D9-9828-AE1AE1792566@redhat.com>
References: <69F3C637-F8E2-47D9-9828-AE1AE1792566@redhat.com>
Message-ID: <3B7BE75B-B099-4D3E-B9A7-C5442CC3DB21@redhat.com>


On 25 Jun 2014, at 13:21, Mircea Markus <mmarkus at redhat.com> wrote:

> Now that the suite is stable would be good to cut ISPN 7.0.0.Alpha5. There are still quite some PRs pending, let's focus on that for now and release on Friday.

Sounds good, but avoid duplicate work, particularly when it comes to testing a PR. 

If you are testing a PR, mention it so that others can focus purely on the code.

Cheers,

> 
> Cheers,
> -- 
> Mircea Markus
> Infinispan lead (www.infinispan.org)
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From galder at redhat.com  Thu Jun 26 10:12:05 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Thu, 26 Jun 2014 16:12:05 +0200
Subject: [infinispan-dev] Hadoop and ISPN first and next steps
In-Reply-To: <5E4FE91D-7A99-4AFA-A0D8-82EFE870A1E0@gmail.com>
References: <5E4FE91D-7A99-4AFA-A0D8-82EFE870A1E0@gmail.com>
Message-ID: <429A0DED-0D09-45EE-8A32-06FEA7E7FA97@redhat.com>


On 23 Jun 2014, at 11:04, Gustavo Fernandes <gustavonalle at gmail.com> wrote:

> - I read with great interest the Spark paper [9]. Spark provides a DSL with functional language constructs like map, flatMap and filter to process distributed data in memory. In this scenario, Map Reduce is just a special case achieved by chaining functions [10]. As Spark is much more than Map Reduce, and can run many machine learning algorithms efficiently, I was wondering if we should shift attention to Spark rather than focusing too much on Map Reduce. Thoughts?

I?m not an expert on these topics, but I like the look and the approach of Spark :). The fact that it?s not tight to a single paradigm is particularly interesting, and secondly, the fact that it?s tries to make the most out of functional constructs, which seem to provide more elegant ways of dealing with data.

Cheers,
--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From galder at redhat.com  Thu Jun 26 11:08:06 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Thu, 26 Jun 2014 17:08:06 +0200
Subject: [infinispan-dev] Can not compile current upstream because of
	test failures
In-Reply-To: <53AAADC5.8050702@redhat.com>
References: <53AAADC5.8050702@redhat.com>
Message-ID: <18584DFB-85C5-4ABB-8868-E61F038FD93A@redhat.com>

I think this is a problem of JDK8, I think I had that before? what JDK are you using?

On 25 Jun 2014, at 13:08, Wolf-Dieter Fink <wfink at redhat.com> wrote:

> Hi,
> 
> I use latest upstream of git at github.com:infinispan/infinispan.git
>   commit bc949a3acb354ff9ac3202b450c521dc6740b415
>   Author: Martin Gencur <mgencur at redhat.com>
>   Date:   Thu Jun 19 15:59:27 2014 +0200
> 
> 
> and the build failed "build.sh clean install", see below.
> Is there a general problem with that tests? Or is it a problem in my 
> environment?
> 
> 
> Also you can not build with "-Dmaven.test.skip=true" as the next module 
> failed because of dependencies
> [INFO] Infinispan Server - Test Suite .................... FAILURE [1.687s]
>   ....
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] 
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal on project test-suite: Could not resolve 
> dependencies for project 
> org.infinispan.server:test-suite:jar:7.0.0-SNAPSHOT: Failure to find 
> org.infinispan:infinispan-security-integrationtests:jar:tests:7.0.0-SNAPSHOT 
> in http://maven.repository.redhat.com/earlyaccess/all/ was cached in the 
> local repository, resolution will not be reattempted until the update 
> interval of redhat-earlyaccess-repository-group has elapsed or updates 
> are forced -> [Help 1]
> 
> 
> 
> ---------  "./build.sh clean install"  failure 
> ---------------------------------
> testReaderRemove(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
> Time elapsed: 0.053 sec  <<< ERROR!
> java.lang.Exception: Unexpected exception, 
> expected<java.security.PrivilegedActionException> but 
> was<javax.security.auth.login.LoginException>
>     at 
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>     at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
>     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
>     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
>     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
>     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
>     at 
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
>     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
>     at 
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
>     at 
> javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
>     at 
> javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
>     at javax.naming.InitialContext.init(InitialContext.java:242)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:356)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
>     at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
>     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
>     at 
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
>     at 
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> 
> testAdminCRUD(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
> Time elapsed: 0.054 sec  <<< ERROR!
> javax.security.auth.login.LoginException: Unable to create new 
> InitialLdapContext
>     at 
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>     at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
>     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
>     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
>     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
>     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
>     at 
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
>     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
>     at 
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
>     at 
> javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
>     at 
> javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
>     at javax.naming.InitialContext.init(InitialContext.java:242)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:356)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
>     at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
>     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
>     at 
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
>     at 
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> 
> testUnauthenticatedWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
> Time elapsed: 0.052 sec  <<< ERROR!
> java.lang.Exception: Unexpected exception, 
> expected<java.lang.SecurityException> but 
> was<javax.security.auth.login.LoginException>
>     at 
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>     at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
>     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
>     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
>     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
>     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
>     at 
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
>     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
>     at 
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
>     at 
> javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
>     at 
> javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
>     at javax.naming.InitialContext.init(InitialContext.java:242)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:356)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
>     at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
>     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
>     at 
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
>     at 
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> 
> testWriterWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
> Time elapsed: 0.053 sec  <<< ERROR!
> javax.security.auth.login.LoginException: Unable to create new 
> InitialLdapContext
>     at 
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>     at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
>     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
>     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
>     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
>     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
>     at 
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
>     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
>     at 
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
>     at 
> javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
>     at 
> javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
>     at javax.naming.InitialContext.init(InitialContext.java:242)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:356)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
>     at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
>     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
>     at 
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
>     at 
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> 
> testWriterCreateWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT) 
> Time elapsed: 0.053 sec  <<< ERROR!
> javax.security.auth.login.LoginException: Unable to create new 
> InitialLdapContext
>     at 
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>     at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>     at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
>     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
>     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
>     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
>     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
>     at 
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
>     at 
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
>     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
>     at 
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
>     at 
> javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
>     at 
> javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
>     at javax.naming.InitialContext.init(InitialContext.java:242)
>     at 
> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:356)
>     at 
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
>     at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
>     at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
>     at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
>     at 
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
>     at 
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> 
> Running 
> org.infinispan.test.integration.security.embedded.LdapAuthenticationIT
> Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.591 
> sec - in 
> org.infinispan.test.integration.security.embedded.LdapAuthenticationIT
> 
> Results :
> 
> Tests in error:
>   KrbLdapAuthenticationIT.testUnprivilegedWrite ?  Unexpected 
> exception, expecte...
>   KrbLdapAuthenticationIT.testUnauthenticatedRemove ?  Unexpected 
> exception, exp...
>   KrbLdapAuthenticationIT.testUnprivilegedRemove ?  Unexpected 
> exception, expect...
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
> ? Login
>   KrbLdapAuthenticationIT.testUnauthenticatedRead ?  Unexpected 
> exception, expec...
>   KrbLdapAuthenticationIT.testWriterRead ?  Unexpected exception, 
> expected<java....
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
> ? Login
>   KrbLdapAuthenticationIT.testReaderWrite ?  Unexpected exception, 
> expected<java...
>   KrbLdapAuthenticationIT.testUnprivilegedRead ?  Unexpected exception, 
> expected...
>   KrbLdapAuthenticationIT.testReaderRemove ?  Unexpected exception, 
> expected<jav...
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
> ? Login
>   KrbLdapAuthenticationIT.testUnauthenticatedWrite ?  Unexpected 
> exception, expe...
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
> ? Login
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68 
> ? Login
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From dan.berindei at gmail.com  Thu Jun 26 11:25:08 2014
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 26 Jun 2014 18:25:08 +0300
Subject: [infinispan-dev] Can not compile current upstream because of
	test failures
In-Reply-To: <18584DFB-85C5-4ABB-8868-E61F038FD93A@redhat.com>
References: <53AAADC5.8050702@redhat.com>
	<18584DFB-85C5-4ABB-8868-E61F038FD93A@redhat.com>
Message-ID: <CA+nfvwQROFJ=d_YBbLPabXDmW=WayGSYERP_S49xYhA-kPFEEA@mail.gmail.com>

On Thu, Jun 26, 2014 at 6:08 PM, Galder Zamarre?o <galder at redhat.com> wrote:

> I think this is a problem of JDK8, I think I had that before? what JDK are
> you using?
>
> On 25 Jun 2014, at 13:08, Wolf-Dieter Fink <wfink at redhat.com> wrote:
>
> > Hi,
> >
> > I use latest upstream of git at github.com:infinispan/infinispan.git
> >   commit bc949a3acb354ff9ac3202b450c521dc6740b415
> >   Author: Martin Gencur <mgencur at redhat.com>
> >   Date:   Thu Jun 19 15:59:27 2014 +0200
> >
> >
> > and the build failed "build.sh clean install", see below.
> > Is there a general problem with that tests? Or is it a problem in my
> > environment?
> >
> >
> > Also you can not build with "-Dmaven.test.skip=true" as the next module
> > failed because of dependencies
>

mvn clean install -DskipTests

will work, it builds and packages the tests but it does not run them.



> > [INFO] Infinispan Server - Test Suite .................... FAILURE
> [1.687s]
> >   ....
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] BUILD FAILURE
> > [INFO]
> > ------------------------------------------------------------------------
> > [ERROR] Failed to execute goal on project test-suite: Could not resolve
> > dependencies for project
> > org.infinispan.server:test-suite:jar:7.0.0-SNAPSHOT: Failure to find
> >
> org.infinispan:infinispan-security-integrationtests:jar:tests:7.0.0-SNAPSHOT
> > in http://maven.repository.redhat.com/earlyaccess/all/ was cached in the
> > local repository, resolution will not be reattempted until the update
> > interval of redhat-earlyaccess-repository-group has elapsed or updates
> > are forced -> [Help 1]
> >
> >
> >
> > ---------  "./build.sh clean install"  failure
> > ---------------------------------
> >
> testReaderRemove(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT)
> > Time elapsed: 0.053 sec  <<< ERROR!
> > java.lang.Exception: Unexpected exception,
> > expected<java.security.PrivilegedActionException> but
> > was<javax.security.auth.login.LoginException>
> >     at
> > sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> >     at
> >
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> >     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
> >     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
> >     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
> >     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
> >     at
> >
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
> >     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
> >     at
> >
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
> >     at
> > javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
> >     at
> > javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
> >     at javax.naming.InitialContext.init(InitialContext.java:242)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:356)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >     at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke(Method.java:606)
> >     at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> >     at
> > javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at
> > javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> >     at
> javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> >     at
> >
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
> >     at
> >
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> >
> >
> testAdminCRUD(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT)
> > Time elapsed: 0.054 sec  <<< ERROR!
> > javax.security.auth.login.LoginException: Unable to create new
> > InitialLdapContext
> >     at
> > sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> >     at
> >
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> >     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
> >     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
> >     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
> >     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
> >     at
> >
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
> >     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
> >     at
> >
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
> >     at
> > javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
> >     at
> > javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
> >     at javax.naming.InitialContext.init(InitialContext.java:242)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:356)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >     at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke(Method.java:606)
> >     at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> >     at
> > javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at
> > javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> >     at
> javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> >     at
> >
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
> >     at
> >
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> >
> >
> testUnauthenticatedWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT)
> > Time elapsed: 0.052 sec  <<< ERROR!
> > java.lang.Exception: Unexpected exception,
> > expected<java.lang.SecurityException> but
> > was<javax.security.auth.login.LoginException>
> >     at
> > sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> >     at
> >
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> >     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
> >     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
> >     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
> >     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
> >     at
> >
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
> >     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
> >     at
> >
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
> >     at
> > javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
> >     at
> > javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
> >     at javax.naming.InitialContext.init(InitialContext.java:242)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:356)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >     at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke(Method.java:606)
> >     at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> >     at
> > javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at
> > javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> >     at
> javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> >     at
> >
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
> >     at
> >
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> >
> >
> testWriterWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT)
> > Time elapsed: 0.053 sec  <<< ERROR!
> > javax.security.auth.login.LoginException: Unable to create new
> > InitialLdapContext
> >     at
> > sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> >     at
> >
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> >     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
> >     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
> >     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
> >     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
> >     at
> >
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
> >     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
> >     at
> >
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
> >     at
> > javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
> >     at
> > javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
> >     at javax.naming.InitialContext.init(InitialContext.java:242)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:356)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >     at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke(Method.java:606)
> >     at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> >     at
> > javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at
> > javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> >     at
> javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> >     at
> >
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
> >     at
> >
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> >
> >
> testWriterCreateWrite(org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT)
> > Time elapsed: 0.053 sec  <<< ERROR!
> > javax.security.auth.login.LoginException: Unable to create new
> > InitialLdapContext
> >     at
> > sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> >     at
> > sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> >     at
> >
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> >     at com.sun.jndi.ldap.sasl.LdapSasl.saslBind(LdapSasl.java:123)
> >     at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:235)
> >     at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740)
> >     at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
> >     at
> > com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
> >     at
> >
> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
> >     at
> >
> org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:114)
> >     at org.jboss.as.naming.InitialContext.init(InitialContext.java:99)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at org.jboss.as.naming.InitialContext.<init>(InitialContext.java:90)
> >     at
> >
> org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:44)
> >     at
> > javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
> >     at
> > javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
> >     at javax.naming.InitialContext.init(InitialContext.java:242)
> >     at
> > javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.constructLdapContext(AdvancedLdapLoginModule.java:432)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.innerLogin(AdvancedLdapLoginModule.java:343)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule$AuthorizeAction.run(AdvancedLdapLoginModule.java:792)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:356)
> >     at
> >
> org.jboss.security.negotiation.AdvancedLdapLoginModule.login(AdvancedLdapLoginModule.java:287)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >     at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke(Method.java:606)
> >     at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> >     at
> > javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> >     at
> javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at
> > javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> >     at
> javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> >     at
> >
> org.infinispan.test.integration.security.embedded.AbstractAuthentication.authenticateWithKrb(AbstractAuthentication.java:68)
> >     at
> >
> org.infinispan.test.integration.security.embedded.KrbLdapAuthenticationIT.getAdminSubject(KrbLdapAuthenticationIT.java:71)
> >
> > Running
> > org.infinispan.test.integration.security.embedded.LdapAuthenticationIT
> > Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.591
> > sec - in
> > org.infinispan.test.integration.security.embedded.LdapAuthenticationIT
> >
> > Results :
> >
> > Tests in error:
> >   KrbLdapAuthenticationIT.testUnprivilegedWrite ?  Unexpected
> > exception, expecte...
> >   KrbLdapAuthenticationIT.testUnauthenticatedRemove ?  Unexpected
> > exception, exp...
> >   KrbLdapAuthenticationIT.testUnprivilegedRemove ?  Unexpected
> > exception, expect...
> >
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68
> > ? Login
> >   KrbLdapAuthenticationIT.testUnauthenticatedRead ?  Unexpected
> > exception, expec...
> >   KrbLdapAuthenticationIT.testWriterRead ?  Unexpected exception,
> > expected<java....
> >
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68
> > ? Login
> >   KrbLdapAuthenticationIT.testReaderWrite ?  Unexpected exception,
> > expected<java...
> >   KrbLdapAuthenticationIT.testUnprivilegedRead ?  Unexpected exception,
> > expected...
> >   KrbLdapAuthenticationIT.testReaderRemove ?  Unexpected exception,
> > expected<jav...
> >
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68
> > ? Login
> >   KrbLdapAuthenticationIT.testUnauthenticatedWrite ?  Unexpected
> > exception, expe...
> >
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68
> > ? Login
> >
> KrbLdapAuthenticationIT>AbstractAuthentication.setupCache:99->getAdminSubject:71->AbstractAuthentication.authenticateWithKrb:68
> > ? Login
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140626/f0c4921c/attachment-0001.html 

From sanne at infinispan.org  Thu Jun 26 14:40:09 2014
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 26 Jun 2014 19:40:09 +0100
Subject: [infinispan-dev] Dropping support for older versions of Apache
	Lucene
Message-ID: <CAFm4XO3e2A1B9a=OX07VxWNfkb4hmGKcjYvBtCktwv=bg5vX5Q@mail.gmail.com>

All,
please note I've sent the pull request which removes support for
Apache Lucene versions 2.x and 3.x [1]

The currently recommended version is Apache Lucene 4.8.1; if someone
has really strong needs to keep support for Lucene 3 please speak up
now as the patch is rather large and complex.

However the primary reason for us to drop this, is that it makes it
very tricky and time consuming to develop a better integration with
Lucene 4+ so I don't think it will be possible to maintain the support
for the older version, unless someone volunteers for it.

Note that the Query engine and all such extras we built on it, were
already migrated to Lucene 4 so there is no loss of other
functionality.

Regards,
Sanne

1 - https://github.com/infinispan/infinispan/pull/2676

From vjuranek at redhat.com  Thu Jun 26 15:35:12 2014
From: vjuranek at redhat.com (Vojtech Juranek)
Date: Thu, 26 Jun 2014 21:35:12 +0200
Subject: [infinispan-dev] Can not compile current upstream because of
	test failures
In-Reply-To: <18584DFB-85C5-4ABB-8868-E61F038FD93A@redhat.com>
References: <53AAADC5.8050702@redhat.com>
	<18584DFB-85C5-4ABB-8868-E61F038FD93A@redhat.com>
Message-ID: <1638505.z1lzGSorvt@localhost>

> I think this is a problem of JDK8, I think I had that before?

IMHO you mean ISPN-4224 [1] which was fixed by 73b35dd37d [2]

> what JDK are you using?

+1 - cannot reproduce on my env (linux, Orcale JDK 7), so either another issue 
specific to used JDK or some environment issue

[1] https://issues.jboss.org/browse/ISPN-4224
[2] 
https://github.com/infinispan/infinispan/commit/73b35dd37df3c538f2abf9e49b6c4f438b4ef232
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part.
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140626/eb60b3e1/attachment.bin 

From rory.odonnell at oracle.com  Fri Jun 27 03:55:02 2014
From: rory.odonnell at oracle.com (Rory O'Donnell Oracle, Dublin Ireland)
Date: Fri, 27 Jun 2014 08:55:02 +0100
Subject: [infinispan-dev] Early Access builds for JDK 9 b19,
 JDK 8u20 b20 are available on java.net
Message-ID: <53AD2356.9070209@oracle.com>

Hi Galder,

Early Access builds for JDK 9 b19 <https://jdk9.java.net/download/>  and 
JDK 8u20 b20 <https://jdk8.java.net/download.html>  are available on 
java.net.

As we enter the later phases of development for JDK 8u20 , please log 
any show
stoppers as soon as possible.

Rgds, Rory

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20140627/651d1bb0/attachment.html 

From mmarkus at redhat.com  Fri Jun 27 08:54:44 2014
From: mmarkus at redhat.com (Mircea Markus)
Date: Fri, 27 Jun 2014 13:54:44 +0100
Subject: [infinispan-dev] cutting ISPN 7.0.0A.Alpha5 Friday
In-Reply-To: <3B7BE75B-B099-4D3E-B9A7-C5442CC3DB21@redhat.com>
References: <69F3C637-F8E2-47D9-9828-AE1AE1792566@redhat.com>
	<3B7BE75B-B099-4D3E-B9A7-C5442CC3DB21@redhat.com>
Message-ID: <311D227F-4E44-43BC-B708-A00CBAA1474B@redhat.com>

Still 2 failures on CI machine so no release yet[1].
Dan/Will please take a look.

[1] http://ci.infinispan.org/viewLog.html?buildId=9237&tab=buildResultsDiv&buildTypeId=bt8

On Jun 25, 2014, at 13:01, Galder Zamarre?o <galder at redhat.com> wrote:

> 
> On 25 Jun 2014, at 13:21, Mircea Markus <mmarkus at redhat.com> wrote:
> 
>> Now that the suite is stable would be good to cut ISPN 7.0.0.Alpha5. There are still quite some PRs pending, let's focus on that for now and release on Friday.
> 
> Sounds good, but avoid duplicate work, particularly when it comes to testing a PR. 
> 
> If you are testing a PR, mention it so that others can focus purely on the code.
> 
> Cheers,
> 
>> 
>> Cheers,
>> -- 
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> twitter.com/galderz
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Cheers,
-- 
Mircea Markus
Infinispan lead (www.infinispan.org)






From mudokonman at gmail.com  Fri Jun 27 10:49:36 2014
From: mudokonman at gmail.com (William Burns)
Date: Fri, 27 Jun 2014 10:49:36 -0400
Subject: [infinispan-dev] cutting ISPN 7.0.0A.Alpha5 Friday
In-Reply-To: <311D227F-4E44-43BC-B708-A00CBAA1474B@redhat.com>
References: <69F3C637-F8E2-47D9-9828-AE1AE1792566@redhat.com>
	<3B7BE75B-B099-4D3E-B9A7-C5442CC3DB21@redhat.com>
	<311D227F-4E44-43BC-B708-A00CBAA1474B@redhat.com>
Message-ID: <CA+YCuUOJD3XqiNPDLCCWbyhom=-b9kFvtTb+0M=EOX6yus5c5g@mail.gmail.com>

On Fri, Jun 27, 2014 at 8:54 AM, Mircea Markus <mmarkus at redhat.com> wrote:
> Still 2 failures on CI machine so no release yet[1].
> Dan/Will please take a look.
>
> [1] http://ci.infinispan.org/viewLog.html?buildId=9237&tab=buildResultsDiv&buildTypeId=bt8

Yeah I logged [1] for this a bit ago.  I was never able to reproduce
this issue prior.  But talking with Dan, it came up that maybe our
laptops have too many cores :P.  So I tried taskset and it was able to
reproduce the issue quite easily.  I just wanted to point this out in
case if someone else runs into a non-deterministic test like this and
it might help out.  Command line I used [2]

[1] https://issues.jboss.org/browse/ISPN-4435
[2] taskset -c 0 mvn test
-Dtest=CacheNotifierImplInitialTransferDistTest -pl core -PtraceTests

>
> On Jun 25, 2014, at 13:01, Galder Zamarre?o <galder at redhat.com> wrote:
>
>>
>> On 25 Jun 2014, at 13:21, Mircea Markus <mmarkus at redhat.com> wrote:
>>
>>> Now that the suite is stable would be good to cut ISPN 7.0.0.Alpha5. There are still quite some PRs pending, let's focus on that for now and release on Friday.
>>
>> Sounds good, but avoid duplicate work, particularly when it comes to testing a PR.
>>
>> If you are testing a PR, mention it so that others can focus purely on the code.
>>
>> Cheers,
>>
>>>
>>> Cheers,
>>> --
>>> Mircea Markus
>>> Infinispan lead (www.infinispan.org)
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> twitter.com/galderz
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From galder at redhat.com  Mon Jun 30 04:36:05 2014
From: galder at redhat.com (=?windows-1252?Q?Galder_Zamarre=F1o?=)
Date: Mon, 30 Jun 2014 10:36:05 +0200
Subject: [infinispan-dev] Where to put KeyValueFilterFactory and
	ConverterFactory...
In-Reply-To: <CA+YCuUPUTt5-Wh=7Bo0Zzf8PTYOucxfCS8kccr9F9cL2zfh5Gw@mail.gmail.com>
References: <F2AE1601-B627-49FA-8F26-4BA988222B43@redhat.com>
	<53A2F2E6.4050803@redhat.com>
	<CA+YCuUPUTt5-Wh=7Bo0Zzf8PTYOucxfCS8kccr9F9cL2zfh5Gw@mail.gmail.com>
Message-ID: <D59EA01F-0083-4285-9E81-2825352879E5@redhat.com>

2 votes for core vs 1 for commons, going for core/

On 19 Jun 2014, at 16:34, William Burns <mudokonman at gmail.com> wrote:

> I was able to look into the details with this a bit.  At first thought
> I would think having a module that both the client and server packages
> could both depend on would be best.  However the issue is that these
> factories produce instances that implement interfaces that live in the
> core package, which means the server/client would have to depend on
> core anyways.  So I am thinking core is probably the best place for
> now.
> 
> Thinking about these factories even more it could be useful to use
> these at some point in core to allow for non serializable
> KeyValueFilter/Converter instances and instead have the factory be
> serializable and just create them on each node as needed.

^ Very good point, in fact, why not do this now? It could solve my problem of [1] in a very clean way. If each node instantiates the KeyValueFilter/Converter instances locally, I can plug the marshaller more easily? If I can plug the factory, I could try to initialise the factory with the marshaller and pass that to the cache, which involves the callback to create the KeyValueFilter/Converter instances, and I can use the marshaller in the factory to initialise these instances.

[1] https://issues.jboss.org/browse/ISPN-4378

> 
> - Will
> 
> On Thu, Jun 19, 2014 at 10:25 AM, Tristan Tarrant <ttarrant at redhat.com> wrote:
>> To me core is ideal:
>> when you develop these components, it's as if you were running in an
>> embedded environment (which is what the server essentially is), and core
>> is the closest you can get.
>> 
>> Tristan
>> 
>> On 19/06/14 16:12, Galder Zamarre?o wrote:
>>> Hi all,
>>> 
>>> As I work on ISPN-3950, I?ve been thinking about the location of KeyValueFilterFactory  and ConverterFactory interfaces. These interfaces provide filter/converter instances to be installed in server/hotrod, but we expect users to provide implementations of such classes and plug the server with them.
>>> 
>>> These interfaces are currently in server/hotrod project, something I?m not keen on doing any more, because users would need to get access to these interfaces and they could start fiddling with other stuff in that module, and I want to avoid that at all costs, particularly since I?m hoping to do some major refactoring of the decoders in 7.1.
>>> 
>>> So, the q is: where should KeyValueFilterFactory and ConverterFactory live?
>>> 
>>> They?re for sure needed by server/hotrod. client/hotrod-client does not need them explicitly, but the users need to provide implementations of them and plug them to the server. With this in mind, core/ might be the best place for it. They cannot live in commons/ cos it?d would require to move KeyValueFilter/Converter and other dependant classes to commons.
>>> 
>>> Is everyone happy with ConverterFactory and KeyValueFilterFactory being in core/?
>>> 
>>> Cheers,
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>> twitter.com/galderz
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com
twitter.com/galderz



From vblagoje at redhat.com  Mon Jun 30 10:47:23 2014
From: vblagoje at redhat.com (Vladimir Blagojevic)
Date: Mon, 30 Jun 2014 10:47:23 -0400
Subject: [infinispan-dev] Hadoop and ISPN first and next steps
In-Reply-To: <429A0DED-0D09-45EE-8A32-06FEA7E7FA97@redhat.com>
References: <5E4FE91D-7A99-4AFA-A0D8-82EFE870A1E0@gmail.com>
	<429A0DED-0D09-45EE-8A32-06FEA7E7FA97@redhat.com>
Message-ID: <53B1787B.50905@redhat.com>

On 2014-06-26, 10:12 AM, Galder Zamarre?o wrote:
> On 23 Jun 2014, at 11:04, Gustavo Fernandes <gustavonalle at gmail.com> wrote:
>
>> - I read with great interest the Spark paper [9]. Spark provides a DSL with functional language constructs like map, flatMap and filter to process distributed data in memory. In this scenario, Map Reduce is just a special case achieved by chaining functions [10]. As Spark is much more than Map Reduce, and can run many machine learning algorithms efficiently, I was wondering if we should shift attention to Spark rather than focusing too much on Map Reduce. Thoughts?
> I?m not an expert on these topics, but I like the look and the approach of Spark :). The fact that it?s not tight to a single paradigm is particularly interesting, and secondly, the fact that it?s tries to make the most out of functional constructs, which seem to provide more elegant ways of dealing with data.
>
>
Gustavo thanks for your email and the references. I like Spark as well! 
I read the Spark paper over the weekend, definitely not an easy digest 
and I will continue to read about this topic but this seems to be the 
direction we should steer ourselves - data analytics platform!

As for Hadoop implementation not sure that it make sense to 
implement/support Hadoop v1.x unless it is super easy and low 
maintenance. How hard would it be to implement YARN?

Regards,
Vladimir