From dan.berindei at gmail.com  Mon Mar  2 03:55:47 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 2 Mar 2015 10:55:47 +0200
Subject: [infinispan-dev] Asynchronous cache's "void put()" call
 expectations changed from 6.0.0 to 6.0.1/7.0
In-Reply-To: <128708287.14780333.1409660354923.JavaMail.zimbra@redhat.com>
References: <E1767D5C-B205-4DF1-81F5-103420C18C04@redhat.com>
	<128708287.14780333.1409660354923.JavaMail.zimbra@redhat.com>
Message-ID: <CA+nfvwS6AEZJOAxogDRS76R4NG6BSEJVazypxMRsVZkErdMxUw@mail.gmail.com>

Reviving the thread...

Paul, this is a little more complicated than just the return value of the
put. In Infinispan prior to 6.0.1, this would have always worked with a
replicated asynchronous cache:

cache.put(k, v); // return value ignored
v1 = cache.get(k);
assertEquals(v, v1);

The behaviour of Infinispan prior to 6.0.1 was to write the value to the
local cache synchronously, and only replicate the value to the other owners
asynchronously.
Of course, if the cache was distributed and the originator was not an
owner, the assertion could have failed anyway, because the remote get could
be executed before the remote put.
The get(k) is not guaranteed to return the value inserted by the put(k, v),
but it did work most of the time.

Since 6.0.1, the originator doesn't update the local value synchronously
any more, unless it is the primary owner. If it is a backup owner, it will
send the update asynchronously to the primary owner, which will then update
the backup owners (including the originator) synchronously, while holding
the lock. This was supposed to change in 5.3.0, as replicated mode was
already using lock delegation in 5.3.0, but EntryWrappingInterceptor had an
outdated check that meant the originator did both the synchronous update
and the asynchronous one. The fix in 6.0.1 was just to remove the
synchronous update.

Personally I think the pre-6.0.1 behaviour was a bit dangerous, because the
get would return the updated value *almost* all of the time. So there was a
chance that tests would work, and the app would only fail in production.
putIfAbsent will also behave more like an atomic operation with 6.0.1+, at
least as long as the primary owner of the key doesn't change.

That's not to say the current behaviour is perfect. I'm open to changing
REPL_ASYNC back to updating the local data container first (if owner) and
then sending the write command asynchronously to all the members in
parallel, like we did in 5.2.0 and before.

We could also change DIST_ASYNC to behave the same, because the current
behaviour is a little too close to what putAsync does in DIST_SYNC mode,
and people using async replication should be ok with nodes applying
modifications in a different order (it happens on primary owner change
anyway).

However, I'm not sure what we should do for atomic operations like
replace(key, oldValue, newValues). I would suggest throwing a
UnsupportedOperationException, because the operation is very unlikely to be
atomic. It's true that PFER is also specified as behaving the same as
putIfAbsent, but the PFER check is going to be on the originator only so we
can just remove that bit from the Javadoc.

Another option would be to change PFER only to behave like in 5.3.0 and
6.0.0. We could say PFER performs two put operations, one that's local-only
and one that's replicated but asynchronous (updating the value on the
originator a second time). Or we could say PFER always updates the value on
the local node only...

Cheers
Dan



> On Tue, Sep 2, 2014 at 3:19 PM, Paul Ferraro <paul.ferraro at redhat.com>
wrote:

> ----- Original Message -----
> > From: "Galder Zamarre?o" <galder at redhat.com>
> > To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>, "Paul
> Ferraro" <paul.ferraro at redhat.com>
> > Sent: Monday, September 1, 2014 11:08:45 AM
> > Subject: Asynchronous cache's "void put()" call expectations changed
> from 6.0.0 to 6.0.1/7.0
> >
> > Hi all,
> >
> > @Paul, this might be important for WF if using async repl caches (the
> same I
> > think applies to distributed async caches too)
>
> Luckily, Dan warned me that of this behavioral change well in advance.
> Any time we need a reliable return value from a Cache.put(...), we use
> Flag.FORCE_SYNCHRONOUS so that the same code will work for sync and async
> caches alike.


> > Today I?ve been trying to upgrade Infinispan version in Hibernate master
> from
> > 6.0.0.Final to 7.0.0.Beta1. Overall, it?s all worked fine but there?s one
> > test that has started failing.
> >
> > Essentialy, this is a clustered test for a repl async cache (w/ cluster
> cache
> > loader) where a non-owner cache node does put() and immediately, on the
> same
> > cache, it calls a get(). The test is failing because the get() does not
> see
> > the effects of the put(), even if both operations are called on the same
> > cache instance.
> >
> > According to Dan, this should have been happening since [1] was
> implemented,
> > but it?s really started happening since [2] when lock delegation was
> enabled
> > for replicated caches (EntryWrappingInterceptor.isUsingLockDelegation is
> now
> > true whereas in 6.0.0 it was false).
> >
> > Not sure we set expectations in this regard, but clearly it?s big change
> in
> > terms of expectations on when ?void put()? completes for async repl
> caches.
> > I?m not sure how we should handle this, but it definitely needs some
> > discussion and adjuts documentation/javadoc if needed. Can we do
> something
> > differently?
> >
> > Indepent of how we resolve this, this is the result of once again of
> trying
> > to shoehole async behaviour into sync APIs. Any async caches (DIST, INV,
> > REPL) should really be accessed exclusively via the AsyncCache API, where
> > you can return quickly and use the future, and any listener to attach to
> it
> > (a bit ala Java8?s CompletableFuture.map lambda calls) as a way to signal
> > that the operation has completed, and then you have an API and cache mode
> > that make sense and is consistent with how async APIs work.
> >
> > Right now, when a repl async cache?s ?void put()? call is not very well
> > defined. Does it return when message has been put on the network? What
> > impact does it have in the local cache contents?
> >
> > Also, a very big problem of the change of behaviour is that if left like
> > that, you are forcing users to code differently, using the same ?void
> put()?
> > API depending on the configuration (whether async/sync). As clearly
> shown by
> > the issue above, this is very confusing. It?s a lot more logical IMO, and
> > I?ve already sent an email on this very same topic [3] back in January,
> that
> > whether a cache is sync or async should be based purely on the API used
> and
> > forget about the static configuration flag on whether the cache is async
> or
> > sync.
>
> I would agree would this last statement.  Consistent semantics are a good
> thing.
> If you do change this, however, just let me know well in advance.
>
> > Cheers,
> >
> > [1] https://issues.jboss.org/browse/ISPN-2772
> > [2] https://issues.jboss.org/browse/ISPN-3354
> > [3]
> http://lists.jboss.org/pipermail/infinispan-dev/2014-January/014448.html
> > --
> > Galder Zamarre?o
> > galder at redhat.com
> > twitter.com/galderz
> >
> >
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20150302/6f5c0f36/attachment-0001.html 

From rvansa at redhat.com  Mon Mar  2 05:20:48 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Mon, 02 Mar 2015 11:20:48 +0100
Subject: [infinispan-dev] Asynchronous cache's "void put()" call
 expectations changed from 6.0.0 to 6.0.1/7.0
In-Reply-To: <CA+nfvwS6AEZJOAxogDRS76R4NG6BSEJVazypxMRsVZkErdMxUw@mail.gmail.com>
References: <E1767D5C-B205-4DF1-81F5-103420C18C04@redhat.com>	<128708287.14780333.1409660354923.JavaMail.zimbra@redhat.com>
	<CA+nfvwS6AEZJOAxogDRS76R4NG6BSEJVazypxMRsVZkErdMxUw@mail.gmail.com>
Message-ID: <54F43980.5000102@redhat.com>

I second Galder's opinion that sync/async should be rather a matter of 
API than configuration, because you have to deal with that in code. 
Therefore, when planning a behaviour of any operation, I would suggest 
that it should be possible to host sync and async operations (currently 
forcable through a flag) with the proper semantics - different paths of 
messages would complicate this a lot (e.g. the guarantee that eventually 
all owners share the same value).

The most natural semantics would be IMO that async operations return 
without any guarantee *when* this operation will be applied (and even 
*if* since it can silently fail with error), with optional way to find 
out that it has been applied (as soon as we add the Java 8 API). And the 
effect should be the same as for sync version. It's a subject for 
discussion whether the retry should be done upon topology change (but it 
needs to be transparent to user anyway).

My 2c

Radim

On 03/02/2015 09:55 AM, Dan Berindei wrote:
> Reviving the thread...
>
> Paul, this is a little more complicated than just the return value of 
> the put. In Infinispan prior to 6.0.1, this would have always worked 
> with a replicated asynchronous cache:
>
> cache.put(k, v); // return value ignored
> v1 = cache.get(k);
> assertEquals(v, v1);
>
> The behaviour of Infinispan prior to 6.0.1 was to write the value to 
> the local cache synchronously, and only replicate the value to the 
> other owners asynchronously.
> Of course, if the cache was distributed and the originator was not an 
> owner, the assertion could have failed anyway, because the remote get 
> could be executed before the remote put.
> The get(k) is not guaranteed to return the value inserted by the 
> put(k, v), but it did work most of the time.
>
> Since 6.0.1, the originator doesn't update the local value 
> synchronously any more, unless it is the primary owner. If it is a 
> backup owner, it will send the update asynchronously to the primary 
> owner, which will then update the backup owners (including the 
> originator) synchronously, while holding the lock. This was supposed 
> to change in 5.3.0, as replicated mode was already using lock 
> delegation in 5.3.0, but EntryWrappingInterceptor had an outdated 
> check that meant the originator did both the synchronous update and 
> the asynchronous one. The fix in 6.0.1 was just to remove the 
> synchronous update.
>
> Personally I think the pre-6.0.1 behaviour was a bit dangerous, 
> because the get would return the updated value *almost* all of the 
> time. So there was a chance that tests would work, and the app would 
> only fail in production. putIfAbsent will also behave more like an 
> atomic operation with 6.0.1+, at least as long as the primary owner of 
> the key doesn't change.
>
> That's not to say the current behaviour is perfect. I'm open to 
> changing REPL_ASYNC back to updating the local data container first 
> (if owner) and then sending the write command asynchronously to all 
> the members in parallel, like we did in 5.2.0 and before.
>
> We could also change DIST_ASYNC to behave the same, because the 
> current behaviour is a little too close to what putAsync does in 
> DIST_SYNC mode, and people using async replication should be ok with 
> nodes applying modifications in a different order (it happens on 
> primary owner change anyway).
>
> However, I'm not sure what we should do for atomic operations like 
> replace(key, oldValue, newValues). I would suggest throwing a 
> UnsupportedOperationException, because the operation is very unlikely 
> to be atomic. It's true that PFER is also specified as behaving the 
> same as putIfAbsent, but the PFER check is going to be on the 
> originator only so we can just remove that bit from the Javadoc.
>
> Another option would be to change PFER only to behave like in 5.3.0 
> and 6.0.0. We could say PFER performs two put operations, one that's 
> local-only and one that's replicated but asynchronous (updating the 
> value on the originator a second time). Or we could say PFER always 
> updates the value on the local node only...
>
> Cheers
> Dan
>
> On Tue, Sep 2, 2014 at 3:19 PM, Paul Ferraro <paul.ferraro at redhat.com 
> <mailto:paul.ferraro at redhat.com>> wrote:
>
>     ----- Original Message -----
>     > From: "Galder Zamarre?o" <galder at redhat.com
>     <mailto:galder at redhat.com>>
>     > To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>>, "Paul Ferraro"
>     <paul.ferraro at redhat.com <mailto:paul.ferraro at redhat.com>>
>     > Sent: Monday, September 1, 2014 11:08:45 AM
>     > Subject: Asynchronous cache's "void put()" call expectations
>     changed from 6.0.0 to 6.0.1/7.0
>     >
>     > Hi all,
>     >
>     > @Paul, this might be important for WF if using async repl caches
>     (the same I
>     > think applies to distributed async caches too)
>
>     Luckily, Dan warned me that of this behavioral change well in advance.
>     Any time we need a reliable return value from a Cache.put(...), we
>     use Flag.FORCE_SYNCHRONOUS so that the same code will work for
>     sync and async caches alike.
>
>
>     > Today I?ve been trying to upgrade Infinispan version in
>     Hibernate master from
>     > 6.0.0.Final to 7.0.0.Beta1. Overall, it?s all worked fine but
>     there?s one
>     > test that has started failing.
>     >
>     > Essentialy, this is a clustered test for a repl async cache (w/
>     cluster cache
>     > loader) where a non-owner cache node does put() and immediately,
>     on the same
>     > cache, it calls a get(). The test is failing because the get()
>     does not see
>     > the effects of the put(), even if both operations are called on
>     the same
>     > cache instance.
>     >
>     > According to Dan, this should have been happening since [1] was
>     implemented,
>     > but it?s really started happening since [2] when lock delegation
>     was enabled
>     > for replicated caches
>     (EntryWrappingInterceptor.isUsingLockDelegation is now
>     > true whereas in 6.0.0 it was false).
>     >
>     > Not sure we set expectations in this regard, but clearly it?s
>     big change in
>     > terms of expectations on when ?void put()? completes for async
>     repl caches.
>     > I?m not sure how we should handle this, but it definitely needs some
>     > discussion and adjuts documentation/javadoc if needed. Can we do
>     something
>     > differently?
>     >
>     > Indepent of how we resolve this, this is the result of once
>     again of trying
>     > to shoehole async behaviour into sync APIs. Any async caches
>     (DIST, INV,
>     > REPL) should really be accessed exclusively via the AsyncCache
>     API, where
>     > you can return quickly and use the future, and any listener to
>     attach to it
>     > (a bit ala Java8?s CompletableFuture.map lambda calls) as a way
>     to signal
>     > that the operation has completed, and then you have an API and
>     cache mode
>     > that make sense and is consistent with how async APIs work.
>     >
>     > Right now, when a repl async cache?s ?void put()? call is not
>     very well
>     > defined. Does it return when message has been put on the
>     network? What
>     > impact does it have in the local cache contents?
>     >
>     > Also, a very big problem of the change of behaviour is that if
>     left like
>     > that, you are forcing users to code differently, using the same
>     ?void put()?
>     > API depending on the configuration (whether async/sync). As
>     clearly shown by
>     > the issue above, this is very confusing. It?s a lot more logical
>     IMO, and
>     > I?ve already sent an email on this very same topic [3] back in
>     January, that
>     > whether a cache is sync or async should be based purely on the
>     API used and
>     > forget about the static configuration flag on whether the cache
>     is async or
>     > sync.
>
>     I would agree would this last statement. Consistent semantics are
>     a good thing.
>     If you do change this, however, just let me know well in advance.
>
>     > Cheers,
>     >
>     > [1] https://issues.jboss.org/browse/ISPN-2772
>     > [2] https://issues.jboss.org/browse/ISPN-3354
>     > [3]
>     http://lists.jboss.org/pipermail/infinispan-dev/2014-January/014448.html
>     > --
>     > Galder Zamarre?o
>     > galder at redhat.com <mailto:galder at redhat.com>
>     > twitter.com/galderz <http://twitter.com/galderz>
>     >
>     >
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Mon Mar  2 06:08:46 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 2 Mar 2015 13:08:46 +0200
Subject: [infinispan-dev] Asynchronous cache's "void put()" call
 expectations changed from 6.0.0 to 6.0.1/7.0
In-Reply-To: <54F43980.5000102@redhat.com>
References: <E1767D5C-B205-4DF1-81F5-103420C18C04@redhat.com>
	<128708287.14780333.1409660354923.JavaMail.zimbra@redhat.com>
	<CA+nfvwS6AEZJOAxogDRS76R4NG6BSEJVazypxMRsVZkErdMxUw@mail.gmail.com>
	<54F43980.5000102@redhat.com>
Message-ID: <CA+nfvwSDR07uC6CPVrv04QnRdMEay-HF7f2uA+rPVRNBAfaPYA@mail.gmail.com>

The AsyncCache interface already has an xxxAsync method for every
write operation in the Cache interface. Those operations really behave
just like the sync operations, except they return a NotifyingFuture -
basically what .

The FORCE_ASYNCHRONOUS flag is different, because you have no way to
find out if the operation was successful. In fact, it behaves as if
the cache was REPL_ASYNC. This can indeed cause some confusion, as the
async operation can break consistency guarantees for concurrent sync
operations. I'm all for removing REPL_ASYNC and the FORCE_ASYNCHRONOUS
flag, but I think users will complain about their removal because
putAsync is slower and they don't care/know about potential
consistency problems.

There's also the question of putForExternalRead, which is currently
defined to always replicate the value asynchronously. If REPL_ASYNC is
removed, it doesn't make sense to keep the async replication code only
for putForExternalRead, so it would have to behave like putAsync
(maybe with an extra CACHE_MODE_LOCAL put).

Cheers
Dan


On Mon, Mar 2, 2015 at 12:20 PM, Radim Vansa <rvansa at redhat.com> wrote:
> I second Galder's opinion that sync/async should be rather a matter of
> API than configuration, because you have to deal with that in code.
> Therefore, when planning a behaviour of any operation, I would suggest
> that it should be possible to host sync and async operations (currently
> forcable through a flag) with the proper semantics - different paths of
> messages would complicate this a lot (e.g. the guarantee that eventually
> all owners share the same value).
>
> The most natural semantics would be IMO that async operations return
> without any guarantee *when* this operation will be applied (and even
> *if* since it can silently fail with error), with optional way to find
> out that it has been applied (as soon as we add the Java 8 API). And the
> effect should be the same as for sync version. It's a subject for
> discussion whether the retry should be done upon topology change (but it
> needs to be transparent to user anyway).
>
> My 2c
>
> Radim
>
> On 03/02/2015 09:55 AM, Dan Berindei wrote:
>> Reviving the thread...
>>
>> Paul, this is a little more complicated than just the return value of
>> the put. In Infinispan prior to 6.0.1, this would have always worked
>> with a replicated asynchronous cache:
>>
>> cache.put(k, v); // return value ignored
>> v1 = cache.get(k);
>> assertEquals(v, v1);
>>
>> The behaviour of Infinispan prior to 6.0.1 was to write the value to
>> the local cache synchronously, and only replicate the value to the
>> other owners asynchronously.
>> Of course, if the cache was distributed and the originator was not an
>> owner, the assertion could have failed anyway, because the remote get
>> could be executed before the remote put.
>> The get(k) is not guaranteed to return the value inserted by the
>> put(k, v), but it did work most of the time.
>>
>> Since 6.0.1, the originator doesn't update the local value
>> synchronously any more, unless it is the primary owner. If it is a
>> backup owner, it will send the update asynchronously to the primary
>> owner, which will then update the backup owners (including the
>> originator) synchronously, while holding the lock. This was supposed
>> to change in 5.3.0, as replicated mode was already using lock
>> delegation in 5.3.0, but EntryWrappingInterceptor had an outdated
>> check that meant the originator did both the synchronous update and
>> the asynchronous one. The fix in 6.0.1 was just to remove the
>> synchronous update.
>>
>> Personally I think the pre-6.0.1 behaviour was a bit dangerous,
>> because the get would return the updated value *almost* all of the
>> time. So there was a chance that tests would work, and the app would
>> only fail in production. putIfAbsent will also behave more like an
>> atomic operation with 6.0.1+, at least as long as the primary owner of
>> the key doesn't change.
>>
>> That's not to say the current behaviour is perfect. I'm open to
>> changing REPL_ASYNC back to updating the local data container first
>> (if owner) and then sending the write command asynchronously to all
>> the members in parallel, like we did in 5.2.0 and before.
>>
>> We could also change DIST_ASYNC to behave the same, because the
>> current behaviour is a little too close to what putAsync does in
>> DIST_SYNC mode, and people using async replication should be ok with
>> nodes applying modifications in a different order (it happens on
>> primary owner change anyway).
>>
>> However, I'm not sure what we should do for atomic operations like
>> replace(key, oldValue, newValues). I would suggest throwing a
>> UnsupportedOperationException, because the operation is very unlikely
>> to be atomic. It's true that PFER is also specified as behaving the
>> same as putIfAbsent, but the PFER check is going to be on the
>> originator only so we can just remove that bit from the Javadoc.
>>
>> Another option would be to change PFER only to behave like in 5.3.0
>> and 6.0.0. We could say PFER performs two put operations, one that's
>> local-only and one that's replicated but asynchronous (updating the
>> value on the originator a second time). Or we could say PFER always
>> updates the value on the local node only...
>>
>> Cheers
>> Dan
>>
>> On Tue, Sep 2, 2014 at 3:19 PM, Paul Ferraro <paul.ferraro at redhat.com
>> <mailto:paul.ferraro at redhat.com>> wrote:
>>
>>     ----- Original Message -----
>>     > From: "Galder Zamarre?o" <galder at redhat.com
>>     <mailto:galder at redhat.com>>
>>     > To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org
>>     <mailto:infinispan-dev at lists.jboss.org>>, "Paul Ferraro"
>>     <paul.ferraro at redhat.com <mailto:paul.ferraro at redhat.com>>
>>     > Sent: Monday, September 1, 2014 11:08:45 AM
>>     > Subject: Asynchronous cache's "void put()" call expectations
>>     changed from 6.0.0 to 6.0.1/7.0
>>     >
>>     > Hi all,
>>     >
>>     > @Paul, this might be important for WF if using async repl caches
>>     (the same I
>>     > think applies to distributed async caches too)
>>
>>     Luckily, Dan warned me that of this behavioral change well in advance.
>>     Any time we need a reliable return value from a Cache.put(...), we
>>     use Flag.FORCE_SYNCHRONOUS so that the same code will work for
>>     sync and async caches alike.
>>
>>
>>     > Today I?ve been trying to upgrade Infinispan version in
>>     Hibernate master from
>>     > 6.0.0.Final to 7.0.0.Beta1. Overall, it?s all worked fine but
>>     there?s one
>>     > test that has started failing.
>>     >
>>     > Essentialy, this is a clustered test for a repl async cache (w/
>>     cluster cache
>>     > loader) where a non-owner cache node does put() and immediately,
>>     on the same
>>     > cache, it calls a get(). The test is failing because the get()
>>     does not see
>>     > the effects of the put(), even if both operations are called on
>>     the same
>>     > cache instance.
>>     >
>>     > According to Dan, this should have been happening since [1] was
>>     implemented,
>>     > but it?s really started happening since [2] when lock delegation
>>     was enabled
>>     > for replicated caches
>>     (EntryWrappingInterceptor.isUsingLockDelegation is now
>>     > true whereas in 6.0.0 it was false).
>>     >
>>     > Not sure we set expectations in this regard, but clearly it?s
>>     big change in
>>     > terms of expectations on when ?void put()? completes for async
>>     repl caches.
>>     > I?m not sure how we should handle this, but it definitely needs some
>>     > discussion and adjuts documentation/javadoc if needed. Can we do
>>     something
>>     > differently?
>>     >
>>     > Indepent of how we resolve this, this is the result of once
>>     again of trying
>>     > to shoehole async behaviour into sync APIs. Any async caches
>>     (DIST, INV,
>>     > REPL) should really be accessed exclusively via the AsyncCache
>>     API, where
>>     > you can return quickly and use the future, and any listener to
>>     attach to it
>>     > (a bit ala Java8?s CompletableFuture.map lambda calls) as a way
>>     to signal
>>     > that the operation has completed, and then you have an API and
>>     cache mode
>>     > that make sense and is consistent with how async APIs work.
>>     >
>>     > Right now, when a repl async cache?s ?void put()? call is not
>>     very well
>>     > defined. Does it return when message has been put on the
>>     network? What
>>     > impact does it have in the local cache contents?
>>     >
>>     > Also, a very big problem of the change of behaviour is that if
>>     left like
>>     > that, you are forcing users to code differently, using the same
>>     ?void put()?
>>     > API depending on the configuration (whether async/sync). As
>>     clearly shown by
>>     > the issue above, this is very confusing. It?s a lot more logical
>>     IMO, and
>>     > I?ve already sent an email on this very same topic [3] back in
>>     January, that
>>     > whether a cache is sync or async should be based purely on the
>>     API used and
>>     > forget about the static configuration flag on whether the cache
>>     is async or
>>     > sync.
>>
>>     I would agree would this last statement. Consistent semantics are
>>     a good thing.
>>     If you do change this, however, just let me know well in advance.
>>
>>     > Cheers,
>>     >
>>     > [1] https://issues.jboss.org/browse/ISPN-2772
>>     > [2] https://issues.jboss.org/browse/ISPN-3354
>>     > [3]
>>     http://lists.jboss.org/pipermail/infinispan-dev/2014-January/014448.html
>>     > --
>>     > Galder Zamarre?o
>>     > galder at redhat.com <mailto:galder at redhat.com>
>>     > twitter.com/galderz <http://twitter.com/galderz>
>>     >
>>     >
>>
>>     _______________________________________________
>>     infinispan-dev mailing list
>>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From rvansa at redhat.com  Mon Mar  2 07:11:47 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Mon, 02 Mar 2015 13:11:47 +0100
Subject: [infinispan-dev] Asynchronous cache's "void put()" call
 expectations changed from 6.0.0 to 6.0.1/7.0
In-Reply-To: <CA+nfvwSDR07uC6CPVrv04QnRdMEay-HF7f2uA+rPVRNBAfaPYA@mail.gmail.com>
References: <E1767D5C-B205-4DF1-81F5-103420C18C04@redhat.com>	<128708287.14780333.1409660354923.JavaMail.zimbra@redhat.com>	<CA+nfvwS6AEZJOAxogDRS76R4NG6BSEJVazypxMRsVZkErdMxUw@mail.gmail.com>	<54F43980.5000102@redhat.com>
	<CA+nfvwSDR07uC6CPVrv04QnRdMEay-HF7f2uA+rPVRNBAfaPYA@mail.gmail.com>
Message-ID: <54F45383.4050203@redhat.com>

On 03/02/2015 12:08 PM, Dan Berindei wrote:
> The AsyncCache interface already has an xxxAsync method for every
> write operation in the Cache interface. Those operations really behave
> just like the sync operations, except they return a NotifyingFuture -
> basically what .

But the implementation is sub-optimal now, as it only delegates the sync 
operation to threadpool, requiring context switch. But this will go away 
with the reactive-Infinispan design as discussed in Berlin. And 
NotifyingFeature will be replaced with standard CompletionFuture. I 
though this was already agreed upon.

>
> The FORCE_ASYNCHRONOUS flag is different, because you have no way to
> find out if the operation was successful. In fact, it behaves as if
> the cache was REPL_ASYNC. This can indeed cause some confusion, as the
> async operation can break consistency guarantees for concurrent sync
> operations. I'm all for removing REPL_ASYNC and the FORCE_ASYNCHRONOUS
> flag, but I think users will complain about their removal because
> putAsync is slower and they don't care/know about potential
> consistency problems.

Slower? You mean that you can achieve lower throughput, right? When you 
declare that you don't need any ack (you should get a Future that would 
throw an exception on get() etc.) you don't have to send the 
confirmation from primary to originator. Do you also omit confirmations 
from backup to primary in *_ASYNC mode? I have a vague idea that this 
can work with relying on JGroups' primary -> backup ordering of 
messages, but during topology change this could be messed up (leading to 
non-consistent cache until the entry is overridden). If the users insist 
on fast and dirty cache, the configuration option shouldn't be <sync 
.../> || <async ... /> but rather <inconsistency allowed="true||false"/> 
- this would make the behaviour more explicit.

Radim

>
> There's also the question of putForExternalRead, which is currently
> defined to always replicate the value asynchronously. If REPL_ASYNC is
> removed, it doesn't make sense to keep the async replication code only
> for putForExternalRead, so it would have to behave like putAsync
> (maybe with an extra CACHE_MODE_LOCAL put).
>
> Cheers
> Dan
>
>
> On Mon, Mar 2, 2015 at 12:20 PM, Radim Vansa <rvansa at redhat.com> wrote:
>> I second Galder's opinion that sync/async should be rather a matter of
>> API than configuration, because you have to deal with that in code.
>> Therefore, when planning a behaviour of any operation, I would suggest
>> that it should be possible to host sync and async operations (currently
>> forcable through a flag) with the proper semantics - different paths of
>> messages would complicate this a lot (e.g. the guarantee that eventually
>> all owners share the same value).
>>
>> The most natural semantics would be IMO that async operations return
>> without any guarantee *when* this operation will be applied (and even
>> *if* since it can silently fail with error), with optional way to find
>> out that it has been applied (as soon as we add the Java 8 API). And the
>> effect should be the same as for sync version. It's a subject for
>> discussion whether the retry should be done upon topology change (but it
>> needs to be transparent to user anyway).
>>
>> My 2c
>>
>> Radim
>>
>> On 03/02/2015 09:55 AM, Dan Berindei wrote:
>>> Reviving the thread...
>>>
>>> Paul, this is a little more complicated than just the return value of
>>> the put. In Infinispan prior to 6.0.1, this would have always worked
>>> with a replicated asynchronous cache:
>>>
>>> cache.put(k, v); // return value ignored
>>> v1 = cache.get(k);
>>> assertEquals(v, v1);
>>>
>>> The behaviour of Infinispan prior to 6.0.1 was to write the value to
>>> the local cache synchronously, and only replicate the value to the
>>> other owners asynchronously.
>>> Of course, if the cache was distributed and the originator was not an
>>> owner, the assertion could have failed anyway, because the remote get
>>> could be executed before the remote put.
>>> The get(k) is not guaranteed to return the value inserted by the
>>> put(k, v), but it did work most of the time.
>>>
>>> Since 6.0.1, the originator doesn't update the local value
>>> synchronously any more, unless it is the primary owner. If it is a
>>> backup owner, it will send the update asynchronously to the primary
>>> owner, which will then update the backup owners (including the
>>> originator) synchronously, while holding the lock. This was supposed
>>> to change in 5.3.0, as replicated mode was already using lock
>>> delegation in 5.3.0, but EntryWrappingInterceptor had an outdated
>>> check that meant the originator did both the synchronous update and
>>> the asynchronous one. The fix in 6.0.1 was just to remove the
>>> synchronous update.
>>>
>>> Personally I think the pre-6.0.1 behaviour was a bit dangerous,
>>> because the get would return the updated value *almost* all of the
>>> time. So there was a chance that tests would work, and the app would
>>> only fail in production. putIfAbsent will also behave more like an
>>> atomic operation with 6.0.1+, at least as long as the primary owner of
>>> the key doesn't change.
>>>
>>> That's not to say the current behaviour is perfect. I'm open to
>>> changing REPL_ASYNC back to updating the local data container first
>>> (if owner) and then sending the write command asynchronously to all
>>> the members in parallel, like we did in 5.2.0 and before.
>>>
>>> We could also change DIST_ASYNC to behave the same, because the
>>> current behaviour is a little too close to what putAsync does in
>>> DIST_SYNC mode, and people using async replication should be ok with
>>> nodes applying modifications in a different order (it happens on
>>> primary owner change anyway).
>>>
>>> However, I'm not sure what we should do for atomic operations like
>>> replace(key, oldValue, newValues). I would suggest throwing a
>>> UnsupportedOperationException, because the operation is very unlikely
>>> to be atomic. It's true that PFER is also specified as behaving the
>>> same as putIfAbsent, but the PFER check is going to be on the
>>> originator only so we can just remove that bit from the Javadoc.
>>>
>>> Another option would be to change PFER only to behave like in 5.3.0
>>> and 6.0.0. We could say PFER performs two put operations, one that's
>>> local-only and one that's replicated but asynchronous (updating the
>>> value on the originator a second time). Or we could say PFER always
>>> updates the value on the local node only...
>>>
>>> Cheers
>>> Dan
>>>
>>> On Tue, Sep 2, 2014 at 3:19 PM, Paul Ferraro <paul.ferraro at redhat.com
>>> <mailto:paul.ferraro at redhat.com>> wrote:
>>>
>>>      ----- Original Message -----
>>>      > From: "Galder Zamarre?o" <galder at redhat.com
>>>      <mailto:galder at redhat.com>>
>>>      > To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org
>>>      <mailto:infinispan-dev at lists.jboss.org>>, "Paul Ferraro"
>>>      <paul.ferraro at redhat.com <mailto:paul.ferraro at redhat.com>>
>>>      > Sent: Monday, September 1, 2014 11:08:45 AM
>>>      > Subject: Asynchronous cache's "void put()" call expectations
>>>      changed from 6.0.0 to 6.0.1/7.0
>>>      >
>>>      > Hi all,
>>>      >
>>>      > @Paul, this might be important for WF if using async repl caches
>>>      (the same I
>>>      > think applies to distributed async caches too)
>>>
>>>      Luckily, Dan warned me that of this behavioral change well in advance.
>>>      Any time we need a reliable return value from a Cache.put(...), we
>>>      use Flag.FORCE_SYNCHRONOUS so that the same code will work for
>>>      sync and async caches alike.
>>>
>>>
>>>      > Today I?ve been trying to upgrade Infinispan version in
>>>      Hibernate master from
>>>      > 6.0.0.Final to 7.0.0.Beta1. Overall, it?s all worked fine but
>>>      there?s one
>>>      > test that has started failing.
>>>      >
>>>      > Essentialy, this is a clustered test for a repl async cache (w/
>>>      cluster cache
>>>      > loader) where a non-owner cache node does put() and immediately,
>>>      on the same
>>>      > cache, it calls a get(). The test is failing because the get()
>>>      does not see
>>>      > the effects of the put(), even if both operations are called on
>>>      the same
>>>      > cache instance.
>>>      >
>>>      > According to Dan, this should have been happening since [1] was
>>>      implemented,
>>>      > but it?s really started happening since [2] when lock delegation
>>>      was enabled
>>>      > for replicated caches
>>>      (EntryWrappingInterceptor.isUsingLockDelegation is now
>>>      > true whereas in 6.0.0 it was false).
>>>      >
>>>      > Not sure we set expectations in this regard, but clearly it?s
>>>      big change in
>>>      > terms of expectations on when ?void put()? completes for async
>>>      repl caches.
>>>      > I?m not sure how we should handle this, but it definitely needs some
>>>      > discussion and adjuts documentation/javadoc if needed. Can we do
>>>      something
>>>      > differently?
>>>      >
>>>      > Indepent of how we resolve this, this is the result of once
>>>      again of trying
>>>      > to shoehole async behaviour into sync APIs. Any async caches
>>>      (DIST, INV,
>>>      > REPL) should really be accessed exclusively via the AsyncCache
>>>      API, where
>>>      > you can return quickly and use the future, and any listener to
>>>      attach to it
>>>      > (a bit ala Java8?s CompletableFuture.map lambda calls) as a way
>>>      to signal
>>>      > that the operation has completed, and then you have an API and
>>>      cache mode
>>>      > that make sense and is consistent with how async APIs work.
>>>      >
>>>      > Right now, when a repl async cache?s ?void put()? call is not
>>>      very well
>>>      > defined. Does it return when message has been put on the
>>>      network? What
>>>      > impact does it have in the local cache contents?
>>>      >
>>>      > Also, a very big problem of the change of behaviour is that if
>>>      left like
>>>      > that, you are forcing users to code differently, using the same
>>>      ?void put()?
>>>      > API depending on the configuration (whether async/sync). As
>>>      clearly shown by
>>>      > the issue above, this is very confusing. It?s a lot more logical
>>>      IMO, and
>>>      > I?ve already sent an email on this very same topic [3] back in
>>>      January, that
>>>      > whether a cache is sync or async should be based purely on the
>>>      API used and
>>>      > forget about the static configuration flag on whether the cache
>>>      is async or
>>>      > sync.
>>>
>>>      I would agree would this last statement. Consistent semantics are
>>>      a good thing.
>>>      If you do change this, however, just let me know well in advance.
>>>
>>>      > Cheers,
>>>      >
>>>      > [1] https://issues.jboss.org/browse/ISPN-2772
>>>      > [2] https://issues.jboss.org/browse/ISPN-3354
>>>      > [3]
>>>      http://lists.jboss.org/pipermail/infinispan-dev/2014-January/014448.html
>>>      > --
>>>      > Galder Zamarre?o
>>>      > galder at redhat.com <mailto:galder at redhat.com>
>>>      > twitter.com/galderz <http://twitter.com/galderz>
>>>      >
>>>      >
>>>
>>>      _______________________________________________
>>>      infinispan-dev mailing list
>>>      infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Mon Mar  2 08:16:14 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 2 Mar 2015 15:16:14 +0200
Subject: [infinispan-dev] Asynchronous cache's "void put()" call
 expectations changed from 6.0.0 to 6.0.1/7.0
In-Reply-To: <54F45383.4050203@redhat.com>
References: <E1767D5C-B205-4DF1-81F5-103420C18C04@redhat.com>
	<128708287.14780333.1409660354923.JavaMail.zimbra@redhat.com>
	<CA+nfvwS6AEZJOAxogDRS76R4NG6BSEJVazypxMRsVZkErdMxUw@mail.gmail.com>
	<54F43980.5000102@redhat.com>
	<CA+nfvwSDR07uC6CPVrv04QnRdMEay-HF7f2uA+rPVRNBAfaPYA@mail.gmail.com>
	<54F45383.4050203@redhat.com>
Message-ID: <CA+nfvwR_ig7sXXwyQcFHso07_K8JP-aiiLCuam3prvHkVkv3ow@mail.gmail.com>

On Mon, Mar 2, 2015 at 2:11 PM, Radim Vansa <rvansa at redhat.com> wrote:
> On 03/02/2015 12:08 PM, Dan Berindei wrote:
>> The AsyncCache interface already has an xxxAsync method for every
>> write operation in the Cache interface. Those operations really behave
>> just like the sync operations, except they return a NotifyingFuture -
>> basically what .
>
> But the implementation is sub-optimal now, as it only delegates the sync
> operation to threadpool, requiring context switch. But this will go away
> with the reactive-Infinispan design as discussed in Berlin. And
> NotifyingFeature will be replaced with standard CompletionFuture. I
> though this was already agreed upon.
>

Sure, it's not an optimal implementation, I was just trying to point
out that we already have a proper async API. We can improve it a bit
once we move to Java 8, but not significantly.

>>
>> The FORCE_ASYNCHRONOUS flag is different, because you have no way to
>> find out if the operation was successful. In fact, it behaves as if
>> the cache was REPL_ASYNC. This can indeed cause some confusion, as the
>> async operation can break consistency guarantees for concurrent sync
>> operations. I'm all for removing REPL_ASYNC and the FORCE_ASYNCHRONOUS
>> flag, but I think users will complain about their removal because
>> putAsync is slower and they don't care/know about potential
>> consistency problems.
>
> Slower? You mean that you can achieve lower throughput, right? When you
> declare that you don't need any ack (you should get a Future that would
> throw an exception on get() etc.) you don't have to send the
> confirmation from primary to originator. Do you also omit confirmations
> from backup to primary in *_ASYNC mode? I have a vague idea that this
> can work with relying on JGroups' primary -> backup ordering of
> messages, but during topology change this could be messed up (leading to
> non-consistent cache until the entry is overridden). If the users insist
> on fast and dirty cache, the configuration option shouldn't be <sync
> .../> || <async ... /> but rather <inconsistency allowed="true||false"/>
> - this would make the behaviour more explicit.

Yes, REPL_ASYNC/DIST_ASYNC currently rely on JGroups delivering
messages in order. And it's not just that they don't require any ack
messages, but they also need to keep the lock on the primary for a
much shorter time (just until the command is sent to the backups vs.
until the command is ack-ed by all backups).

Of course, as you pointed out, this breaks when the primary owner
changes. That's why I suggested that if we keep the async replication
option, we stop using lock delegation for it and we instead send the
command directly from the originator to all the owners (basically the
model we had before 5.2).

However, DIST_SYNC also allows inconsistencies in some cases,
especially when it comes to the return value of atomic operations. So
<inconsistency allowed="false"/> wouldn't be quite right. <unsafe
asyncReplication="false|true"/> looks a little better to me.


>
> Radim
>
>>
>> There's also the question of putForExternalRead, which is currently
>> defined to always replicate the value asynchronously. If REPL_ASYNC is
>> removed, it doesn't make sense to keep the async replication code only
>> for putForExternalRead, so it would have to behave like putAsync
>> (maybe with an extra CACHE_MODE_LOCAL put).
>>
>> Cheers
>> Dan
>>
>>
>> On Mon, Mar 2, 2015 at 12:20 PM, Radim Vansa <rvansa at redhat.com> wrote:
>>> I second Galder's opinion that sync/async should be rather a matter of
>>> API than configuration, because you have to deal with that in code.
>>> Therefore, when planning a behaviour of any operation, I would suggest
>>> that it should be possible to host sync and async operations (currently
>>> forcable through a flag) with the proper semantics - different paths of
>>> messages would complicate this a lot (e.g. the guarantee that eventually
>>> all owners share the same value).
>>>
>>> The most natural semantics would be IMO that async operations return
>>> without any guarantee *when* this operation will be applied (and even
>>> *if* since it can silently fail with error), with optional way to find
>>> out that it has been applied (as soon as we add the Java 8 API). And the
>>> effect should be the same as for sync version. It's a subject for
>>> discussion whether the retry should be done upon topology change (but it
>>> needs to be transparent to user anyway).
>>>
>>> My 2c
>>>
>>> Radim
>>>
>>> On 03/02/2015 09:55 AM, Dan Berindei wrote:
>>>> Reviving the thread...
>>>>
>>>> Paul, this is a little more complicated than just the return value of
>>>> the put. In Infinispan prior to 6.0.1, this would have always worked
>>>> with a replicated asynchronous cache:
>>>>
>>>> cache.put(k, v); // return value ignored
>>>> v1 = cache.get(k);
>>>> assertEquals(v, v1);
>>>>
>>>> The behaviour of Infinispan prior to 6.0.1 was to write the value to
>>>> the local cache synchronously, and only replicate the value to the
>>>> other owners asynchronously.
>>>> Of course, if the cache was distributed and the originator was not an
>>>> owner, the assertion could have failed anyway, because the remote get
>>>> could be executed before the remote put.
>>>> The get(k) is not guaranteed to return the value inserted by the
>>>> put(k, v), but it did work most of the time.
>>>>
>>>> Since 6.0.1, the originator doesn't update the local value
>>>> synchronously any more, unless it is the primary owner. If it is a
>>>> backup owner, it will send the update asynchronously to the primary
>>>> owner, which will then update the backup owners (including the
>>>> originator) synchronously, while holding the lock. This was supposed
>>>> to change in 5.3.0, as replicated mode was already using lock
>>>> delegation in 5.3.0, but EntryWrappingInterceptor had an outdated
>>>> check that meant the originator did both the synchronous update and
>>>> the asynchronous one. The fix in 6.0.1 was just to remove the
>>>> synchronous update.
>>>>
>>>> Personally I think the pre-6.0.1 behaviour was a bit dangerous,
>>>> because the get would return the updated value *almost* all of the
>>>> time. So there was a chance that tests would work, and the app would
>>>> only fail in production. putIfAbsent will also behave more like an
>>>> atomic operation with 6.0.1+, at least as long as the primary owner of
>>>> the key doesn't change.
>>>>
>>>> That's not to say the current behaviour is perfect. I'm open to
>>>> changing REPL_ASYNC back to updating the local data container first
>>>> (if owner) and then sending the write command asynchronously to all
>>>> the members in parallel, like we did in 5.2.0 and before.
>>>>
>>>> We could also change DIST_ASYNC to behave the same, because the
>>>> current behaviour is a little too close to what putAsync does in
>>>> DIST_SYNC mode, and people using async replication should be ok with
>>>> nodes applying modifications in a different order (it happens on
>>>> primary owner change anyway).
>>>>
>>>> However, I'm not sure what we should do for atomic operations like
>>>> replace(key, oldValue, newValues). I would suggest throwing a
>>>> UnsupportedOperationException, because the operation is very unlikely
>>>> to be atomic. It's true that PFER is also specified as behaving the
>>>> same as putIfAbsent, but the PFER check is going to be on the
>>>> originator only so we can just remove that bit from the Javadoc.
>>>>
>>>> Another option would be to change PFER only to behave like in 5.3.0
>>>> and 6.0.0. We could say PFER performs two put operations, one that's
>>>> local-only and one that's replicated but asynchronous (updating the
>>>> value on the originator a second time). Or we could say PFER always
>>>> updates the value on the local node only...
>>>>
>>>> Cheers
>>>> Dan
>>>>
>>>> On Tue, Sep 2, 2014 at 3:19 PM, Paul Ferraro <paul.ferraro at redhat.com
>>>> <mailto:paul.ferraro at redhat.com>> wrote:
>>>>
>>>>      ----- Original Message -----
>>>>      > From: "Galder Zamarre?o" <galder at redhat.com
>>>>      <mailto:galder at redhat.com>>
>>>>      > To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org
>>>>      <mailto:infinispan-dev at lists.jboss.org>>, "Paul Ferraro"
>>>>      <paul.ferraro at redhat.com <mailto:paul.ferraro at redhat.com>>
>>>>      > Sent: Monday, September 1, 2014 11:08:45 AM
>>>>      > Subject: Asynchronous cache's "void put()" call expectations
>>>>      changed from 6.0.0 to 6.0.1/7.0
>>>>      >
>>>>      > Hi all,
>>>>      >
>>>>      > @Paul, this might be important for WF if using async repl caches
>>>>      (the same I
>>>>      > think applies to distributed async caches too)
>>>>
>>>>      Luckily, Dan warned me that of this behavioral change well in advance.
>>>>      Any time we need a reliable return value from a Cache.put(...), we
>>>>      use Flag.FORCE_SYNCHRONOUS so that the same code will work for
>>>>      sync and async caches alike.
>>>>
>>>>
>>>>      > Today I?ve been trying to upgrade Infinispan version in
>>>>      Hibernate master from
>>>>      > 6.0.0.Final to 7.0.0.Beta1. Overall, it?s all worked fine but
>>>>      there?s one
>>>>      > test that has started failing.
>>>>      >
>>>>      > Essentialy, this is a clustered test for a repl async cache (w/
>>>>      cluster cache
>>>>      > loader) where a non-owner cache node does put() and immediately,
>>>>      on the same
>>>>      > cache, it calls a get(). The test is failing because the get()
>>>>      does not see
>>>>      > the effects of the put(), even if both operations are called on
>>>>      the same
>>>>      > cache instance.
>>>>      >
>>>>      > According to Dan, this should have been happening since [1] was
>>>>      implemented,
>>>>      > but it?s really started happening since [2] when lock delegation
>>>>      was enabled
>>>>      > for replicated caches
>>>>      (EntryWrappingInterceptor.isUsingLockDelegation is now
>>>>      > true whereas in 6.0.0 it was false).
>>>>      >
>>>>      > Not sure we set expectations in this regard, but clearly it?s
>>>>      big change in
>>>>      > terms of expectations on when ?void put()? completes for async
>>>>      repl caches.
>>>>      > I?m not sure how we should handle this, but it definitely needs some
>>>>      > discussion and adjuts documentation/javadoc if needed. Can we do
>>>>      something
>>>>      > differently?
>>>>      >
>>>>      > Indepent of how we resolve this, this is the result of once
>>>>      again of trying
>>>>      > to shoehole async behaviour into sync APIs. Any async caches
>>>>      (DIST, INV,
>>>>      > REPL) should really be accessed exclusively via the AsyncCache
>>>>      API, where
>>>>      > you can return quickly and use the future, and any listener to
>>>>      attach to it
>>>>      > (a bit ala Java8?s CompletableFuture.map lambda calls) as a way
>>>>      to signal
>>>>      > that the operation has completed, and then you have an API and
>>>>      cache mode
>>>>      > that make sense and is consistent with how async APIs work.
>>>>      >
>>>>      > Right now, when a repl async cache?s ?void put()? call is not
>>>>      very well
>>>>      > defined. Does it return when message has been put on the
>>>>      network? What
>>>>      > impact does it have in the local cache contents?
>>>>      >
>>>>      > Also, a very big problem of the change of behaviour is that if
>>>>      left like
>>>>      > that, you are forcing users to code differently, using the same
>>>>      ?void put()?
>>>>      > API depending on the configuration (whether async/sync). As
>>>>      clearly shown by
>>>>      > the issue above, this is very confusing. It?s a lot more logical
>>>>      IMO, and
>>>>      > I?ve already sent an email on this very same topic [3] back in
>>>>      January, that
>>>>      > whether a cache is sync or async should be based purely on the
>>>>      API used and
>>>>      > forget about the static configuration flag on whether the cache
>>>>      is async or
>>>>      > sync.
>>>>
>>>>      I would agree would this last statement. Consistent semantics are
>>>>      a good thing.
>>>>      If you do change this, however, just let me know well in advance.
>>>>
>>>>      > Cheers,
>>>>      >
>>>>      > [1] https://issues.jboss.org/browse/ISPN-2772
>>>>      > [2] https://issues.jboss.org/browse/ISPN-3354
>>>>      > [3]
>>>>      http://lists.jboss.org/pipermail/infinispan-dev/2014-January/014448.html
>>>>      > --
>>>>      > Galder Zamarre?o
>>>>      > galder at redhat.com <mailto:galder at redhat.com>
>>>>      > twitter.com/galderz <http://twitter.com/galderz>
>>>>      >
>>>>      >
>>>>
>>>>      _______________________________________________
>>>>      infinispan-dev mailing list
>>>>      infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From stkousso at redhat.com  Mon Mar  2 08:43:57 2015
From: stkousso at redhat.com (Stelios Koussouris)
Date: Mon, 2 Mar 2015 08:43:57 -0500 (EST)
Subject: [infinispan-dev] HotRodClient SocketTimeout behavior during
	Continuous Queries
In-Reply-To: <534506269.9991279.1425025831773.JavaMail.zimbra@redhat.com>
References: <534506269.9991279.1425025831773.JavaMail.zimbra@redhat.com>
Message-ID: <1877694297.10910580.1425303837766.JavaMail.zimbra@redhat.com>

Looking at Remote Listeners and Continuous Queries last week I noticed that after the default 60secs of inactivity the socket maintained open for server to client push of grid events was closed with java.net.SocketTimeoutException and the client had no way of recovering.

I am not sure if this was intentional feature so that inactive connections are closed but the way to overcome this was to set sockettimeout(0).

However, even this solution is not perfect as in an environment where client/server are separated by a firewall the "inactive" connection could be closed by the firewall.

It be useful to "refresh" the connection at SOCKET_TIMEOUT-x Time to ensure this doesn't occur.

Stelios

From dan.berindei at gmail.com  Mon Mar  2 09:38:20 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 2 Mar 2015 16:38:20 +0200
Subject: [infinispan-dev] Status update
Message-ID: <CA+nfvwQ+zQR7UbkN1pVLZD5jh79W+Qjm-339iB5640rWpOHj+Q@mail.gmail.com>

Hi guys

I'll be missing the meeting today, so here's my update.

I've fixed some small issues:

* ISPN-5246 Re-enable the queue for the JGroups internal thread pool
in the default configuration
* ISPN-5223 NPE in InfinispanCollections.containsAny - partition handling
* ISPN-5141 SingleFileStore.process() should use sequential access

I also looked into the status of the test suite in CI and I created
some new JIRAs:
* JGRP-1916 ConcurrentModificationException in FD_ALL
* ISPN-5254 Server not always stopped properly with the IBM JDK

This week:
* ISPN-5174 Transaction cannot be recommitted after ownership changes
* ISPN-5044 ClusterTopologyManagerTest.testClusterRecoveryAfterSplitAndCoordLeave
* Review Will's LRU/LIRS changes

Cheers
Dan

From galder at redhat.com  Tue Mar  3 10:23:31 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 3 Mar 2015 16:23:31 +0100
Subject: [infinispan-dev]
	org.infinispan.notifications.cachelistener.filter vs
	org.infinispan.filter
In-Reply-To: <CA+YCuUMwFe9AvBbmcO7SQY-NrPq58Pgf6ogTahkgj5=dj1h9Cw@mail.gmail.com>
References: <A8598238-AA8E-4B97-9271-C02180AB704E@redhat.com>
	<CA+YCuUMwFe9AvBbmcO7SQY-NrPq58Pgf6ogTahkgj5=dj1h9Cw@mail.gmail.com>
Message-ID: <73E0896C-7713-42EB-9E39-2C792EE2E8B4@redhat.com>

Hmm, ok. It's true that down the line the remote events might morph into more specialised DSL-based remote event filter/conversion.

I just wished the naming would have been a bit more representative.

Ideally, the filter/converter classes used for remote events should have lived in a remote jar and be named accordingly, but they had to live in core, which has made it a little akward.

Some users might want to check old value even for cluster listeners, in that case it's a little akward the way it's now. I guess time will tell. I also wonder whether Java 8's default methods might enable more selective implementation, but using it to encompass both types of filters might be a misused. Just some thoughts.

Will, thanks for the clear response :)

> On 26 Feb 2015, at 15:12, William Burns <mudokonman at gmail.com> wrote:
> 
> On Thu, Feb 26, 2015 at 3:57 AM, Galder Zamarre?o <galder at redhat.com> wrote:
>> Hey Will,
>> 
>> I wanted to ask you about the classes in org.infinispan.filter package.
>> 
>> I thought we agreed to get rid of these duplicate classes before 7.0.0.Final:
>> - org.infinispan.filter.Converter
>> - org.infinispan.filter.KeyValueFilter
>> - ... and related classes
>> 
>> The reason being that we pretty much had the same classes in org.infinispan.notifications.cachelistener.filter package.
> 
> I agree they are very similar.
> 
>> 
>> Both set of classes were added in 7.0, so there's no reason why we should have left both sets around :|
> 
> Each one fulfills a different purpose.  Originally only the 1 set of
> classes was used, but event filters evolved to have new requirements
> (oldValue, oldMetadata, retry, event type) that don't make sense for
> normal filtering.  The regular KeyValueFilter, KeyFilter, Converter
> classes are still used when filtering existing entries (CacheLoader,
> EntryIterator).  Also the simpler filter & converter will be able to
> be used as stepping stones for Streams when we move to Java 8
> (although a tweak to the interface(s) will probably be required).
> 
>> 
>> These latter classes provide more information, but it can confuse users to decide which filters/converters to use where and how.
> 
> The CacheEventFilter/CacheEventFilterConverter are only used when
> using events.  The other simpler filters are used in all other cases..
> 
>> 
>> Am I missing something here?
> 
> The classes we talked about removing were the *Factory classes for the
> various filters.  We can discuss converging the ones you mentioned
> here, but IMO the provided functionality has diverged quite a bit,
> which would make having a consistent API very ugly for one or both of
> the use cases.
> 
>> 
>> We would not be able to remove anything now, but we should deprecate what should not be used.
>> 
>> From a remote even perspective, only the org.infinispan.notifications.cachelistener.filter ones are relevant.
> 
> I would say from a remote event perspective you are correct.  However
> once we add entry iteration for remote, we would want to use the more
> simplified interfaces.
> 
>> 
>> Cheers,
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From galder at redhat.com  Tue Mar  3 10:28:46 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 3 Mar 2015 16:28:46 +0100
Subject: [infinispan-dev] HotRodClient SocketTimeout behavior during
	Continuous Queries
In-Reply-To: <1877694297.10910580.1425303837766.JavaMail.zimbra@redhat.com>
References: <534506269.9991279.1425025831773.JavaMail.zimbra@redhat.com>
	<1877694297.10910580.1425303837766.JavaMail.zimbra@redhat.com>
Message-ID: <D6F9FE43-2058-46FF-8741-A1EF46A89AF1@redhat.com>

A similar issue was reported recently in JIRA, but where the cause was long GCs. 

I'll be adding a bit of safety to deal with these situations.

The reason this happens is cos the existing Java Hot Rod client is based around sync TCP communications. A true async client would make this a lot easier. I have some ideas for it for after Java 8 is integrated :)

Cheers,

[1] https://issues.jboss.org/browse/ISPN-5221

> On 2 Mar 2015, at 14:43, Stelios Koussouris <stkousso at redhat.com> wrote:
> 
> Looking at Remote Listeners and Continuous Queries last week I noticed that after the default 60secs of inactivity the socket maintained open for server to client push of grid events was closed with java.net.SocketTimeoutException and the client had no way of recovering.
> 
> I am not sure if this was intentional feature so that inactive connections are closed but the way to overcome this was to set sockettimeout(0).
> 
> However, even this solution is not perfect as in an environment where client/server are separated by a firewall the "inactive" connection could be closed by the firewall.
> 
> It be useful to "refresh" the connection at SOCKET_TIMEOUT-x Time to ensure this doesn't occur.
> 
> Stelios
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From emmanuel at hibernate.org  Tue Mar  3 10:32:11 2015
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Tue, 3 Mar 2015 16:32:11 +0100
Subject: [infinispan-dev]
	org.infinispan.notifications.cachelistener.filter vs
	org.infinispan.filter
In-Reply-To: <73E0896C-7713-42EB-9E39-2C792EE2E8B4@redhat.com>
References: <A8598238-AA8E-4B97-9271-C02180AB704E@redhat.com>
	<CA+YCuUMwFe9AvBbmcO7SQY-NrPq58Pgf6ogTahkgj5=dj1h9Cw@mail.gmail.com>
	<73E0896C-7713-42EB-9E39-2C792EE2E8B4@redhat.com>
Message-ID: <88769462-CE4A-464F-8E4B-BB6CA837C1F8@hibernate.org>

Should it be morphed into a FAQ?

> On 03 Mar 2015, at 16:23, Galder Zamarre?o <galder at redhat.com> wrote:
> 
> Hmm, ok. It's true that down the line the remote events might morph into more specialised DSL-based remote event filter/conversion.
> 
> I just wished the naming would have been a bit more representative.
> 
> Ideally, the filter/converter classes used for remote events should have lived in a remote jar and be named accordingly, but they had to live in core, which has made it a little akward.
> 
> Some users might want to check old value even for cluster listeners, in that case it's a little akward the way it's now. I guess time will tell. I also wonder whether Java 8's default methods might enable more selective implementation, but using it to encompass both types of filters might be a misused. Just some thoughts.
> 
> Will, thanks for the clear response :)
> 
>> On 26 Feb 2015, at 15:12, William Burns <mudokonman at gmail.com> wrote:
>> 
>> On Thu, Feb 26, 2015 at 3:57 AM, Galder Zamarre?o <galder at redhat.com> wrote:
>>> Hey Will,
>>> 
>>> I wanted to ask you about the classes in org.infinispan.filter package.
>>> 
>>> I thought we agreed to get rid of these duplicate classes before 7.0.0.Final:
>>> - org.infinispan.filter.Converter
>>> - org.infinispan.filter.KeyValueFilter
>>> - ... and related classes
>>> 
>>> The reason being that we pretty much had the same classes in org.infinispan.notifications.cachelistener.filter package.
>> 
>> I agree they are very similar.
>> 
>>> 
>>> Both set of classes were added in 7.0, so there's no reason why we should have left both sets around :|
>> 
>> Each one fulfills a different purpose.  Originally only the 1 set of
>> classes was used, but event filters evolved to have new requirements
>> (oldValue, oldMetadata, retry, event type) that don't make sense for
>> normal filtering.  The regular KeyValueFilter, KeyFilter, Converter
>> classes are still used when filtering existing entries (CacheLoader,
>> EntryIterator).  Also the simpler filter & converter will be able to
>> be used as stepping stones for Streams when we move to Java 8
>> (although a tweak to the interface(s) will probably be required).
>> 
>>> 
>>> These latter classes provide more information, but it can confuse users to decide which filters/converters to use where and how.
>> 
>> The CacheEventFilter/CacheEventFilterConverter are only used when
>> using events.  The other simpler filters are used in all other cases..
>> 
>>> 
>>> Am I missing something here?
>> 
>> The classes we talked about removing were the *Factory classes for the
>> various filters.  We can discuss converging the ones you mentioned
>> here, but IMO the provided functionality has diverged quite a bit,
>> which would make having a consistent API very ugly for one or both of
>> the use cases.
>> 
>>> 
>>> We would not be able to remove anything now, but we should deprecate what should not be used.
>>> 
>>> From a remote even perspective, only the org.infinispan.notifications.cachelistener.filter ones are relevant.
>> 
>> I would say from a remote event perspective you are correct.  However
>> once we add entry iteration for remote, we would want to use the more
>> simplified interfaces.
>> 
>>> 
>>> Cheers,
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>> 
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> --
> Galder Zamarre?o
> galder at redhat.com
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From dan.berindei at gmail.com  Thu Mar  5 06:04:49 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 5 Mar 2015 13:04:49 +0200
Subject: [infinispan-dev] state transfer timed out, where to configure?
In-Reply-To: <54D8C9A5.4000101@nexustelecom.com>
References: <54CA6D02.1030009@nexustelecom.com>
	<CA+nfvwS5M1kEQ4+fphYZZfd3-Fpkb0HzHS_=21ozUKFgHXdxkw@mail.gmail.com>
	<54D89072.3010403@nexustelecom.com>
	<CA+nfvwT1aceM1jti+A0w+jW962=AW0r1rzHOohFJKyXHvaiRfg@mail.gmail.com>
	<54D8C9A5.4000101@nexustelecom.com>
Message-ID: <CA+nfvwRhNUDL48BHOCGOSXMSL8dmzRGxRerGkBDLf5CP4Lwp2w@mail.gmail.com>

The fix for ISPN-5141 is in master now, if you want to give it a try.

You may also want to try using SoftIndexFileStore instead of
SingleFileStore. SingleFileStore needs to keep all the keys in memory,
so the memory overhead is huge when the values are small.
SoftIndexFileStore doesn't have this problem, and state transfer might
be even faster than with the ISPN-5141 fix for SingleFileStore.

Cheers
Dan


On Mon, Feb 9, 2015 at 4:52 PM, Andreas Kruthoff
<andreas.kruthoff at nexustelecom.com> wrote:
> I've added my vote.
>
> I've increased the chunk size-from 512 entries (default) to 10'000
> entries, as from the xsd: "The number of cache entries to batch in each
> transfer." I thought that an increased chunk-size helps to populate the
> cache faster? Or is it the opposite? One entry is very small, just a
> Long for key and value.
>
>
> On 02/09/2015 02:31 PM, Dan Berindei wrote:
>> I'm not 100% sure, but this sounds like you're hitting ISPN-5141 [1].
>> Please add your vote, it might make it into 7.2.0.
>>
>> Lowering the state transfer chunkSize may also help, especially if your
>> values are large. The chunkSize is the number of entries in a single
>> chunk, you should aim at making the average chunk <= 500KB.
>>
>> Cheers
>> Dan
>>
>> [1] https://issues.jboss.org/browse/ISPN-5141
>>
>>
>> On Mon, Feb 9, 2015 at 11:48 AM, Andreas Kruthoff
>> <andreas.kruthoff at nexustelecom.com
>> <mailto:andreas.kruthoff at nexustelecom.com>> wrote:
>>
>>     Thanks. The cache is a distributed cache with 3 nodes and replication
>>     factor 2. It contains ~40 million entries. As soon as a node has loaded
>>     all entries from disk, new entries are added to this node with high
>>     frequency (several thousand/s).
>>
>>     The other nodes might not be up yet. It takes 'forever' to bring them up
>>     then...?
>>
>>     On 02/03/2015 12:32 PM, Dan Berindei wrote:
>>     > Hi Andreas
>>     >
>>     > Yes, that's the one. 1200s sounds like a huge timeout, though, how big
>>     > is your cache?
>>     >
>>     > Cheers
>>     > Dan
>>     >
>>     >
>>     > On Thu, Jan 29, 2015 at 7:25 PM, Andreas Kruthoff
>>     > <andreas.kruthoff at nexustelecom.com
>>     <mailto:andreas.kruthoff at nexustelecom.com>
>>      > <mailto:andreas.kruthoff at nexustelecom.com
>>     <mailto:andreas.kruthoff at nexustelecom.com>>> wrote:
>>      >
>>      >     Hi dev
>>      >
>>      >
>>      >     I'm running into the following exception on the 3rd node out
>>     of 2.
>>      >     Distributed cluster, file store with a few millions of entries.
>>      >
>>      >     The 3rd node times out during startup, I think. "Initial
>>     state transfer
>>      >     timed out". How can I configure/increase the timeout in my
>>      >     infinispan.xml?
>>      >
>>      >     Is it <state-transfer timeout="1200000"/> within <distributed
>>     cache/> ?
>>      >
>>      >
>>      >     thanks for help
>>      >
>>      >     -andreas
>>      >
>>      >
>>      >     Exception in thread "main"
>>     org.infinispan.commons.CacheException: Unable
>>      >     to invoke method public void
>>      >
>>       org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
>>      >     th
>>      >     rows java.lang.InterruptedException on object of type
>>      >     StateTransferManagerImpl
>>      >               at
>>      >
>>       org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
>>      >               at
>>      >
>>       org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
>>      >               at
>>      >
>>       org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
>>      >               at
>>      >
>>       org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
>>      >               at
>>      >
>>       org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
>>      >               at
>>      >
>>       org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
>>      >               at
>>      >     org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:813)
>>      >               at
>>      >
>>       org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
>>      >               at
>>      >
>>       org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
>>      >               at
>>      >
>>       org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
>>      >               at
>>      >     ch.nexustelecom.lbd.engine.ImeiCache.init(ImeiCache.java:49)
>>      >               at
>>      >
>>       ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
>>      >               at
>>      >
>>       ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
>>      >               at
>>      >
>>       ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
>>      >               at
>>      >
>>       ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
>>      >     Caused by: org.infinispan.commons.CacheException: Initial
>>     state transfer
>>      >     timed out for cache infinicache-lbd-imei on m4sxhpsrm672-11986
>>      >               at
>>      >
>>       org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
>>      >               at
>>     sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>      >               at
>>      >
>>       sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>      >               at
>>      >
>>       sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>      >               at java.lang.reflect.Method.invoke(Method.java:606)
>>      >               at
>>      >
>>       org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
>>      >               ... 14 more
>>      >
>>      >     This email and any attachment may contain confidential
>>     information
>>      >     which is intended for use only by the addressee(s) named
>>     above. If
>>      >     you received this email by mistake, please notify the sender
>>      >     immediately, and delete the email from your system. You are
>>      >     prohibited from copying, disseminating or otherwise using the
>>     email
>>      >     or any attachment.
>>      >
>>      >     _______________________________________________
>>      >     infinispan-dev mailing list
>>      > infinispan-dev at lists.jboss.org
>>     <mailto:infinispan-dev at lists.jboss.org>
>>     <mailto:infinispan-dev at lists.jboss.org
>>     <mailto:infinispan-dev at lists.jboss.org>>
>>      > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>      >
>>      >
>>
>>     This email and any attachment may contain confidential information
>>     which is intended for use only by the addressee(s) named above. If
>>     you received this email by mistake, please notify the sender
>>     immediately, and delete the email from your system. You are
>>     prohibited from copying, disseminating or otherwise using the email
>>     or any attachment.
>>
>>     _______________________________________________
>>     infinispan-dev mailing list
>>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>
> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From andreas.kruthoff at nexustelecom.com  Thu Mar  5 08:19:38 2015
From: andreas.kruthoff at nexustelecom.com (Andreas Kruthoff)
Date: Thu, 5 Mar 2015 14:19:38 +0100
Subject: [infinispan-dev] state transfer timed out, where to configure?
In-Reply-To: <CA+nfvwRhNUDL48BHOCGOSXMSL8dmzRGxRerGkBDLf5CP4Lwp2w@mail.gmail.com>
References: <54CA6D02.1030009@nexustelecom.com>	<CA+nfvwS5M1kEQ4+fphYZZfd3-Fpkb0HzHS_=21ozUKFgHXdxkw@mail.gmail.com>	<54D89072.3010403@nexustelecom.com>	<CA+nfvwT1aceM1jti+A0w+jW962=AW0r1rzHOohFJKyXHvaiRfg@mail.gmail.com>	<54D8C9A5.4000101@nexustelecom.com>
	<CA+nfvwRhNUDL48BHOCGOSXMSL8dmzRGxRerGkBDLf5CP4Lwp2w@mail.gmail.com>
Message-ID: <54F857EA.1030002@nexustelecom.com>

Thanks for the information. I'd like to try the SoftIndexFileStore, but
no luck with the configuration so far... I'm using version 7.1.1.

Exception in thread "main"
org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
find a parser for element 'soft-index-file-store' in namespace
'urn:infinispan:config:soft-index:7.1'. Check that your configuration is
up-to date for this version of Infinispan.

I was trying this:
http://infinispan.org/docs/7.1.x/user_guide/user_guide.html#_soft_index_file_store

<infinispan
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="urn:infinispan:config:7.1
http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
     xmlns="urn:infinispan:config:7.1">

   <jgroups>
     <stack-file name="fast" path="jgroups-fast.xml"/>
   </jgroups>

   <cache-container name="DataCacheManager" statistics="false"
default-cache="data-cache" shutdown-hook="DEFAULT">

     <transport stack="fast" cluster="data"/>

     <local-cache name="with_sifs">
       <persistence>
         <soft-index-file-store
xmlns="urn:infinispan:config:soft-index:7.1">
           <index path="/tmp/sifs/testCache/index" />
           <data path="/tmp/sifs/testCache/data" />
         </soft-index-file-store>
       </persistence>
     </local-cache>

   </cache-container>

</infinispan>


On 03/05/2015 12:04 PM, Dan Berindei wrote:
> The fix for ISPN-5141 is in master now, if you want to give it a try.
>
> You may also want to try using SoftIndexFileStore instead of
> SingleFileStore. SingleFileStore needs to keep all the keys in memory,
> so the memory overhead is huge when the values are small.
> SoftIndexFileStore doesn't have this problem, and state transfer might
> be even faster than with the ISPN-5141 fix for SingleFileStore.
>
> Cheers
> Dan
>
>
> On Mon, Feb 9, 2015 at 4:52 PM, Andreas Kruthoff
> <andreas.kruthoff at nexustelecom.com> wrote:
>> I've added my vote.
>>
>> I've increased the chunk size-from 512 entries (default) to 10'000
>> entries, as from the xsd: "The number of cache entries to batch in each
>> transfer." I thought that an increased chunk-size helps to populate the
>> cache faster? Or is it the opposite? One entry is very small, just a
>> Long for key and value.
>>
>>
>> On 02/09/2015 02:31 PM, Dan Berindei wrote:
>>> I'm not 100% sure, but this sounds like you're hitting ISPN-5141 [1].
>>> Please add your vote, it might make it into 7.2.0.
>>>
>>> Lowering the state transfer chunkSize may also help, especially if your
>>> values are large. The chunkSize is the number of entries in a single
>>> chunk, you should aim at making the average chunk <= 500KB.
>>>
>>> Cheers
>>> Dan
>>>
>>> [1] https://issues.jboss.org/browse/ISPN-5141
>>>
>>>
>>> On Mon, Feb 9, 2015 at 11:48 AM, Andreas Kruthoff
>>> <andreas.kruthoff at nexustelecom.com
>>> <mailto:andreas.kruthoff at nexustelecom.com>> wrote:
>>>
>>>      Thanks. The cache is a distributed cache with 3 nodes and replication
>>>      factor 2. It contains ~40 million entries. As soon as a node has loaded
>>>      all entries from disk, new entries are added to this node with high
>>>      frequency (several thousand/s).
>>>
>>>      The other nodes might not be up yet. It takes 'forever' to bring them up
>>>      then...?
>>>
>>>      On 02/03/2015 12:32 PM, Dan Berindei wrote:
>>>      > Hi Andreas
>>>      >
>>>      > Yes, that's the one. 1200s sounds like a huge timeout, though, how big
>>>      > is your cache?
>>>      >
>>>      > Cheers
>>>      > Dan
>>>      >
>>>      >
>>>      > On Thu, Jan 29, 2015 at 7:25 PM, Andreas Kruthoff
>>>      > <andreas.kruthoff at nexustelecom.com
>>>      <mailto:andreas.kruthoff at nexustelecom.com>
>>>       > <mailto:andreas.kruthoff at nexustelecom.com
>>>      <mailto:andreas.kruthoff at nexustelecom.com>>> wrote:
>>>       >
>>>       >     Hi dev
>>>       >
>>>       >
>>>       >     I'm running into the following exception on the 3rd node out
>>>      of 2.
>>>       >     Distributed cluster, file store with a few millions of entries.
>>>       >
>>>       >     The 3rd node times out during startup, I think. "Initial
>>>      state transfer
>>>       >     timed out". How can I configure/increase the timeout in my
>>>       >     infinispan.xml?
>>>       >
>>>       >     Is it <state-transfer timeout="1200000"/> within <distributed
>>>      cache/> ?
>>>       >
>>>       >
>>>       >     thanks for help
>>>       >
>>>       >     -andreas
>>>       >
>>>       >
>>>       >     Exception in thread "main"
>>>      org.infinispan.commons.CacheException: Unable
>>>       >     to invoke method public void
>>>       >
>>>        org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
>>>       >     th
>>>       >     rows java.lang.InterruptedException on object of type
>>>       >     StateTransferManagerImpl
>>>       >               at
>>>       >
>>>        org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
>>>       >               at
>>>       >
>>>        org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
>>>       >               at
>>>       >
>>>        org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
>>>       >               at
>>>       >
>>>        org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
>>>       >               at
>>>       >
>>>        org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
>>>       >               at
>>>       >
>>>        org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
>>>       >               at
>>>       >     org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:813)
>>>       >               at
>>>       >
>>>        org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
>>>       >               at
>>>       >
>>>        org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
>>>       >               at
>>>       >
>>>        org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
>>>       >               at
>>>       >     ch.nexustelecom.lbd.engine.ImeiCache.init(ImeiCache.java:49)
>>>       >               at
>>>       >
>>>        ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
>>>       >               at
>>>       >
>>>        ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
>>>       >               at
>>>       >
>>>        ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
>>>       >               at
>>>       >
>>>        ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
>>>       >     Caused by: org.infinispan.commons.CacheException: Initial
>>>      state transfer
>>>       >     timed out for cache infinicache-lbd-imei on m4sxhpsrm672-11986
>>>       >               at
>>>       >
>>>        org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
>>>       >               at
>>>      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>       >               at
>>>       >
>>>        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>       >               at
>>>       >
>>>        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>       >               at java.lang.reflect.Method.invoke(Method.java:606)
>>>       >               at
>>>       >
>>>        org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
>>>       >               ... 14 more
>>>       >
>>>       >     This email and any attachment may contain confidential
>>>      information
>>>       >     which is intended for use only by the addressee(s) named
>>>      above. If
>>>       >     you received this email by mistake, please notify the sender
>>>       >     immediately, and delete the email from your system. You are
>>>       >     prohibited from copying, disseminating or otherwise using the
>>>      email
>>>       >     or any attachment.
>>>       >
>>>       >     _______________________________________________
>>>       >     infinispan-dev mailing list
>>>       > infinispan-dev at lists.jboss.org
>>>      <mailto:infinispan-dev at lists.jboss.org>
>>>      <mailto:infinispan-dev at lists.jboss.org
>>>      <mailto:infinispan-dev at lists.jboss.org>>
>>>       > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>       >
>>>       >
>>>
>>>      This email and any attachment may contain confidential information
>>>      which is intended for use only by the addressee(s) named above. If
>>>      you received this email by mistake, please notify the sender
>>>      immediately, and delete the email from your system. You are
>>>      prohibited from copying, disseminating or otherwise using the email
>>>      or any attachment.
>>>
>>>      _______________________________________________
>>>      infinispan-dev mailing list
>>>      infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>
>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.


From paul.ferraro at redhat.com  Thu Mar  5 08:48:30 2015
From: paul.ferraro at redhat.com (Paul Ferraro)
Date: Thu, 5 Mar 2015 08:48:30 -0500 (EST)
Subject: [infinispan-dev] Is it time to reduce the complexity of
	the	CacheManager?
In-Reply-To: <CAFm4XO3Y75+_+7nxHQvQ-A+550p+LZQmpAZjhwUUVkfr7K_3Kw@mail.gmail.com>
References: <CAFm4XO3Y75+_+7nxHQvQ-A+550p+LZQmpAZjhwUUVkfr7K_3Kw@mail.gmail.com>
Message-ID: <412500443.24059145.1425563310443.JavaMail.zimbra@redhat.com>

----- Original Message -----
> From: "Sanne Grinovero" <sanne at infinispan.org>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Thursday, February 19, 2015 11:46:11 AM
> Subject: [infinispan-dev] Is it time to reduce the complexity of the	CacheManager?
> 
> All,
> at  the beginning of time, the expectation was that an application
> server (aka WildFly) would have a single CacheManager, and different
> applications would define their different Cache configuration on this
> app-server singleton.
> 
> In that primitive world that sounded reasonable, as system
> administrators wouldn't want to manage firewalls and port assignments
> for a new Transport for each deployed application.
> 
> Then the complexities came:
>  - deployments are asymmetric compared to the application server
>  - each deployment has its own ClassLoader
>  - deployments start/stop independently from each other
> 
> At that point a considerable investment was made to get lazily
> starting Caches, per-Cache sets of Externalizer(s) to isolate
> classloaders, ClassLoader-aware Cache decorators, up to the recently
> introduced Cache-dependency rules for stopping dependant Caches last.
> 
> Not to mention we have now a complex per-Cache View handling, which
> results in performance complexities such as ISPN-4842.
> 
> There are some more complexities coming:
> Hibernate OGM wishes to control the context of deserialization - this
> is actually an important optimisation to keep garbage production under
> control, but also applications might want to register custom RPC
> commands; this has been a long standing problem for Search (among
> others).
> Infinispan Query does have custom RPC commands, and this just happens
> to work because the Infinispan core module has an explicit dependency
> to query.. but that's a twisted dependency scheme, as the module would
> need to list each possible extension point: it's not something you can
> do for all projects using it.
> 
> Interestingly enough, there is a very simple solution which wipes out
> all of the above complexity, and also resolves some pain points:
> today the app server supports the FORK protocol from JGroups, so we
> can get rid of the idea of a single CacheManager per appserver, and
> create one per classloader and *within* the classloader.
> 
> By doing so, we can delete all code about per-Cache classloaders,

+1
This was something that was originally needed in AS6, but was made obsolete by the use of ModularClassResolver in AS7 and later.

> remove the CacheView concept, 

-1
So long as there is no such thing as a FORK view, I don't think we can get rid of Cache views - all you would succeed in doing is replacing it with a "cache manager" view (which would be equivalent to the FORK view).  However, I think the argument can be made that a cache manager vieww

> and also allow the deployment (the
> application) which is needing caching services to register whatever it
> wants.
> It could register custom interceptors, commands, externalizers,
> CacheStore(s), etc.. without pain.
> 
> Finally, we could get rid of the concept that Caches start lazily. I'd
> change to a simplified lifecycle which expects the CacheManager to
> initialize, then allows Cache configurations to be defined, and then
> it all starts atomically.
> At that point, you'd not even be responsible anymore for complex
> dependency resolutions across caches.

This makes sense - WildFly has its own mechanism for handling the lifecycle of dependent caches.

> I'd hope this would allow OGM to get the features it needs, and also
> significantly simplify the definition of boot process for any user,
> not least it should simplify quite some code which is now being
> maintained.
> 
> A nice follow-up task would be that WildFly would need to be able to
> "pick up" some configuration file from the deployment and inject a
> matching CacheManager, so this requires a bit of cooperation with the
> app server team, and an agreement on a conventional configuration
> name.
> This should be done by WildFly (and not the app), so that the user
> deployment can lookup the CacheManager by JNDI without needing to
> understand how to wire things up in the FORK channel.

I have to ask, if the proposal is to create a 1:1 relationship between cache and cache manager, and to also link their lifecycle - why are these separate objects/concepts at all?

> I also believe this is a winner from usability point of view, as many
> of the troubles I see in forums and customers are about "how do I
> start this up?". Remember our guides essentially teach you to either
> take the AS CacheManager, or to start your own. Neither of those are
> the optimal solution, and people get in trouble.
> 
> WDYT?
> 
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From rvansa at redhat.com  Thu Mar  5 10:12:26 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 05 Mar 2015 16:12:26 +0100
Subject: [infinispan-dev] state transfer timed out, where to configure?
In-Reply-To: <54F857EA.1030002@nexustelecom.com>
References: <54CA6D02.1030009@nexustelecom.com>	<CA+nfvwS5M1kEQ4+fphYZZfd3-Fpkb0HzHS_=21ozUKFgHXdxkw@mail.gmail.com>	<54D89072.3010403@nexustelecom.com>	<CA+nfvwT1aceM1jti+A0w+jW962=AW0r1rzHOohFJKyXHvaiRfg@mail.gmail.com>	<54D8C9A5.4000101@nexustelecom.com>	<CA+nfvwRhNUDL48BHOCGOSXMSL8dmzRGxRerGkBDLf5CP4Lwp2w@mail.gmail.com>
	<54F857EA.1030002@nexustelecom.com>
Message-ID: <54F8725A.5000400@redhat.com>

You have a typo in your xmlns - should be 
urn:infinispan:config:store:soft-index:7.1
(you were missing the store part) - this was missing in 7.0 (for 
symmetry with other stores) so it was added for 7.1 but documentation 
was not updated.

Btw., Gustavo has mentioned that he was disappointed with the 
performance of store boot; the whole index is dropped and rebuilt again 
(by design). Therefore, I am afraid that if you want to use it in order 
to speed up the boot, it won't work without further fixes (boot time was 
not considered when I was designing it, the focus was on operation 
throughput).

Radim


On 03/05/2015 02:19 PM, Andreas Kruthoff wrote:
> Thanks for the information. I'd like to try the SoftIndexFileStore, but
> no luck with the configuration so far... I'm using version 7.1.1.
>
> Exception in thread "main"
> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
> find a parser for element 'soft-index-file-store' in namespace
> 'urn:infinispan:config:soft-index:7.1'. Check that your configuration is
> up-to date for this version of Infinispan.
>
> I was trying this:
> http://infinispan.org/docs/7.1.x/user_guide/user_guide.html#_soft_index_file_store
>
> <infinispan
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xsi:schemaLocation="urn:infinispan:config:7.1
> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
>       xmlns="urn:infinispan:config:7.1">
>
>     <jgroups>
>       <stack-file name="fast" path="jgroups-fast.xml"/>
>     </jgroups>
>
>     <cache-container name="DataCacheManager" statistics="false"
> default-cache="data-cache" shutdown-hook="DEFAULT">
>
>       <transport stack="fast" cluster="data"/>
>
>       <local-cache name="with_sifs">
>         <persistence>
>           <soft-index-file-store
> xmlns="urn:infinispan:config:soft-index:7.1">
>             <index path="/tmp/sifs/testCache/index" />
>             <data path="/tmp/sifs/testCache/data" />
>           </soft-index-file-store>
>         </persistence>
>       </local-cache>
>
>     </cache-container>
>
> </infinispan>
>
>
> On 03/05/2015 12:04 PM, Dan Berindei wrote:
>> The fix for ISPN-5141 is in master now, if you want to give it a try.
>>
>> You may also want to try using SoftIndexFileStore instead of
>> SingleFileStore. SingleFileStore needs to keep all the keys in memory,
>> so the memory overhead is huge when the values are small.
>> SoftIndexFileStore doesn't have this problem, and state transfer might
>> be even faster than with the ISPN-5141 fix for SingleFileStore.
>>
>> Cheers
>> Dan
>>
>>
>> On Mon, Feb 9, 2015 at 4:52 PM, Andreas Kruthoff
>> <andreas.kruthoff at nexustelecom.com> wrote:
>>> I've added my vote.
>>>
>>> I've increased the chunk size-from 512 entries (default) to 10'000
>>> entries, as from the xsd: "The number of cache entries to batch in each
>>> transfer." I thought that an increased chunk-size helps to populate the
>>> cache faster? Or is it the opposite? One entry is very small, just a
>>> Long for key and value.
>>>
>>>
>>> On 02/09/2015 02:31 PM, Dan Berindei wrote:
>>>> I'm not 100% sure, but this sounds like you're hitting ISPN-5141 [1].
>>>> Please add your vote, it might make it into 7.2.0.
>>>>
>>>> Lowering the state transfer chunkSize may also help, especially if your
>>>> values are large. The chunkSize is the number of entries in a single
>>>> chunk, you should aim at making the average chunk <= 500KB.
>>>>
>>>> Cheers
>>>> Dan
>>>>
>>>> [1] https://issues.jboss.org/browse/ISPN-5141
>>>>
>>>>
>>>> On Mon, Feb 9, 2015 at 11:48 AM, Andreas Kruthoff
>>>> <andreas.kruthoff at nexustelecom.com
>>>> <mailto:andreas.kruthoff at nexustelecom.com>> wrote:
>>>>
>>>>       Thanks. The cache is a distributed cache with 3 nodes and replication
>>>>       factor 2. It contains ~40 million entries. As soon as a node has loaded
>>>>       all entries from disk, new entries are added to this node with high
>>>>       frequency (several thousand/s).
>>>>
>>>>       The other nodes might not be up yet. It takes 'forever' to bring them up
>>>>       then...?
>>>>
>>>>       On 02/03/2015 12:32 PM, Dan Berindei wrote:
>>>>       > Hi Andreas
>>>>       >
>>>>       > Yes, that's the one. 1200s sounds like a huge timeout, though, how big
>>>>       > is your cache?
>>>>       >
>>>>       > Cheers
>>>>       > Dan
>>>>       >
>>>>       >
>>>>       > On Thu, Jan 29, 2015 at 7:25 PM, Andreas Kruthoff
>>>>       > <andreas.kruthoff at nexustelecom.com
>>>>       <mailto:andreas.kruthoff at nexustelecom.com>
>>>>        > <mailto:andreas.kruthoff at nexustelecom.com
>>>>       <mailto:andreas.kruthoff at nexustelecom.com>>> wrote:
>>>>        >
>>>>        >     Hi dev
>>>>        >
>>>>        >
>>>>        >     I'm running into the following exception on the 3rd node out
>>>>       of 2.
>>>>        >     Distributed cluster, file store with a few millions of entries.
>>>>        >
>>>>        >     The 3rd node times out during startup, I think. "Initial
>>>>       state transfer
>>>>        >     timed out". How can I configure/increase the timeout in my
>>>>        >     infinispan.xml?
>>>>        >
>>>>        >     Is it <state-transfer timeout="1200000"/> within <distributed
>>>>       cache/> ?
>>>>        >
>>>>        >
>>>>        >     thanks for help
>>>>        >
>>>>        >     -andreas
>>>>        >
>>>>        >
>>>>        >     Exception in thread "main"
>>>>       org.infinispan.commons.CacheException: Unable
>>>>        >     to invoke method public void
>>>>        >
>>>>         org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
>>>>        >     th
>>>>        >     rows java.lang.InterruptedException on object of type
>>>>        >     StateTransferManagerImpl
>>>>        >               at
>>>>        >
>>>>         org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
>>>>        >               at
>>>>        >     org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:813)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
>>>>        >               at
>>>>        >     ch.nexustelecom.lbd.engine.ImeiCache.init(ImeiCache.java:49)
>>>>        >               at
>>>>        >
>>>>         ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
>>>>        >               at
>>>>        >
>>>>         ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
>>>>        >               at
>>>>        >
>>>>         ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
>>>>        >               at
>>>>        >
>>>>         ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
>>>>        >     Caused by: org.infinispan.commons.CacheException: Initial
>>>>       state transfer
>>>>        >     timed out for cache infinicache-lbd-imei on m4sxhpsrm672-11986
>>>>        >               at
>>>>        >
>>>>         org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
>>>>        >               at
>>>>       sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>        >               at
>>>>        >
>>>>         sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>        >               at
>>>>        >
>>>>         sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>        >               at java.lang.reflect.Method.invoke(Method.java:606)
>>>>        >               at
>>>>        >
>>>>         org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
>>>>        >               ... 14 more
>>>>        >
>>>>        >     This email and any attachment may contain confidential
>>>>       information
>>>>        >     which is intended for use only by the addressee(s) named
>>>>       above. If
>>>>        >     you received this email by mistake, please notify the sender
>>>>        >     immediately, and delete the email from your system. You are
>>>>        >     prohibited from copying, disseminating or otherwise using the
>>>>       email
>>>>        >     or any attachment.
>>>>        >
>>>>        >     _______________________________________________
>>>>        >     infinispan-dev mailing list
>>>>        > infinispan-dev at lists.jboss.org
>>>>       <mailto:infinispan-dev at lists.jboss.org>
>>>>       <mailto:infinispan-dev at lists.jboss.org
>>>>       <mailto:infinispan-dev at lists.jboss.org>>
>>>>        > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>        >
>>>>        >
>>>>
>>>>       This email and any attachment may contain confidential information
>>>>       which is intended for use only by the addressee(s) named above. If
>>>>       you received this email by mistake, please notify the sender
>>>>       immediately, and delete the email from your system. You are
>>>>       prohibited from copying, disseminating or otherwise using the email
>>>>       or any attachment.
>>>>
>>>>       _______________________________________________
>>>>       infinispan-dev mailing list
>>>>       infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>>       https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>>
>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From dan.berindei at gmail.com  Thu Mar  5 10:17:08 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 5 Mar 2015 17:17:08 +0200
Subject: [infinispan-dev] state transfer timed out, where to configure?
In-Reply-To: <54F857EA.1030002@nexustelecom.com>
References: <54CA6D02.1030009@nexustelecom.com>
	<CA+nfvwS5M1kEQ4+fphYZZfd3-Fpkb0HzHS_=21ozUKFgHXdxkw@mail.gmail.com>
	<54D89072.3010403@nexustelecom.com>
	<CA+nfvwT1aceM1jti+A0w+jW962=AW0r1rzHOohFJKyXHvaiRfg@mail.gmail.com>
	<54D8C9A5.4000101@nexustelecom.com>
	<CA+nfvwRhNUDL48BHOCGOSXMSL8dmzRGxRerGkBDLf5CP4Lwp2w@mail.gmail.com>
	<54F857EA.1030002@nexustelecom.com>
Message-ID: <CA+nfvwSuQp4dQUL8sF84Ok16tTgb3=ADTn+NBaXMoEEFCeJGCA@mail.gmail.com>

I think the namespace should be
"urn:infinispan:config:store:soft-index:7.1" (the extra "store" was
added in 7.1).

On Thu, Mar 5, 2015 at 3:19 PM, Andreas Kruthoff
<andreas.kruthoff at nexustelecom.com> wrote:
> Thanks for the information. I'd like to try the SoftIndexFileStore, but
> no luck with the configuration so far... I'm using version 7.1.1.
>
> Exception in thread "main"
> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
> find a parser for element 'soft-index-file-store' in namespace
> 'urn:infinispan:config:soft-index:7.1'. Check that your configuration is
> up-to date for this version of Infinispan.
>
> I was trying this:
> http://infinispan.org/docs/7.1.x/user_guide/user_guide.html#_soft_index_file_store
>
> <infinispan
>      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>      xsi:schemaLocation="urn:infinispan:config:7.1
> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
>      xmlns="urn:infinispan:config:7.1">
>
>    <jgroups>
>      <stack-file name="fast" path="jgroups-fast.xml"/>
>    </jgroups>
>
>    <cache-container name="DataCacheManager" statistics="false"
> default-cache="data-cache" shutdown-hook="DEFAULT">
>
>      <transport stack="fast" cluster="data"/>
>
>      <local-cache name="with_sifs">
>        <persistence>
>          <soft-index-file-store
> xmlns="urn:infinispan:config:soft-index:7.1">
>            <index path="/tmp/sifs/testCache/index" />
>            <data path="/tmp/sifs/testCache/data" />
>          </soft-index-file-store>
>        </persistence>
>      </local-cache>
>
>    </cache-container>
>
> </infinispan>
>
>
> On 03/05/2015 12:04 PM, Dan Berindei wrote:
>> The fix for ISPN-5141 is in master now, if you want to give it a try.
>>
>> You may also want to try using SoftIndexFileStore instead of
>> SingleFileStore. SingleFileStore needs to keep all the keys in memory,
>> so the memory overhead is huge when the values are small.
>> SoftIndexFileStore doesn't have this problem, and state transfer might
>> be even faster than with the ISPN-5141 fix for SingleFileStore.
>>
>> Cheers
>> Dan
>>
>>
>> On Mon, Feb 9, 2015 at 4:52 PM, Andreas Kruthoff
>> <andreas.kruthoff at nexustelecom.com> wrote:
>>> I've added my vote.
>>>
>>> I've increased the chunk size-from 512 entries (default) to 10'000
>>> entries, as from the xsd: "The number of cache entries to batch in each
>>> transfer." I thought that an increased chunk-size helps to populate the
>>> cache faster? Or is it the opposite? One entry is very small, just a
>>> Long for key and value.
>>>
>>>
>>> On 02/09/2015 02:31 PM, Dan Berindei wrote:
>>>> I'm not 100% sure, but this sounds like you're hitting ISPN-5141 [1].
>>>> Please add your vote, it might make it into 7.2.0.
>>>>
>>>> Lowering the state transfer chunkSize may also help, especially if your
>>>> values are large. The chunkSize is the number of entries in a single
>>>> chunk, you should aim at making the average chunk <= 500KB.
>>>>
>>>> Cheers
>>>> Dan
>>>>
>>>> [1] https://issues.jboss.org/browse/ISPN-5141
>>>>
>>>>
>>>> On Mon, Feb 9, 2015 at 11:48 AM, Andreas Kruthoff
>>>> <andreas.kruthoff at nexustelecom.com
>>>> <mailto:andreas.kruthoff at nexustelecom.com>> wrote:
>>>>
>>>>      Thanks. The cache is a distributed cache with 3 nodes and replication
>>>>      factor 2. It contains ~40 million entries. As soon as a node has loaded
>>>>      all entries from disk, new entries are added to this node with high
>>>>      frequency (several thousand/s).
>>>>
>>>>      The other nodes might not be up yet. It takes 'forever' to bring them up
>>>>      then...?
>>>>
>>>>      On 02/03/2015 12:32 PM, Dan Berindei wrote:
>>>>      > Hi Andreas
>>>>      >
>>>>      > Yes, that's the one. 1200s sounds like a huge timeout, though, how big
>>>>      > is your cache?
>>>>      >
>>>>      > Cheers
>>>>      > Dan
>>>>      >
>>>>      >
>>>>      > On Thu, Jan 29, 2015 at 7:25 PM, Andreas Kruthoff
>>>>      > <andreas.kruthoff at nexustelecom.com
>>>>      <mailto:andreas.kruthoff at nexustelecom.com>
>>>>       > <mailto:andreas.kruthoff at nexustelecom.com
>>>>      <mailto:andreas.kruthoff at nexustelecom.com>>> wrote:
>>>>       >
>>>>       >     Hi dev
>>>>       >
>>>>       >
>>>>       >     I'm running into the following exception on the 3rd node out
>>>>      of 2.
>>>>       >     Distributed cluster, file store with a few millions of entries.
>>>>       >
>>>>       >     The 3rd node times out during startup, I think. "Initial
>>>>      state transfer
>>>>       >     timed out". How can I configure/increase the timeout in my
>>>>       >     infinispan.xml?
>>>>       >
>>>>       >     Is it <state-transfer timeout="1200000"/> within <distributed
>>>>      cache/> ?
>>>>       >
>>>>       >
>>>>       >     thanks for help
>>>>       >
>>>>       >     -andreas
>>>>       >
>>>>       >
>>>>       >     Exception in thread "main"
>>>>      org.infinispan.commons.CacheException: Unable
>>>>       >     to invoke method public void
>>>>       >
>>>>        org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
>>>>       >     th
>>>>       >     rows java.lang.InterruptedException on object of type
>>>>       >     StateTransferManagerImpl
>>>>       >               at
>>>>       >
>>>>        org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
>>>>       >               at
>>>>       >     org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:813)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
>>>>       >               at
>>>>       >     ch.nexustelecom.lbd.engine.ImeiCache.init(ImeiCache.java:49)
>>>>       >               at
>>>>       >
>>>>        ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
>>>>       >               at
>>>>       >
>>>>        ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
>>>>       >               at
>>>>       >
>>>>        ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
>>>>       >               at
>>>>       >
>>>>        ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
>>>>       >     Caused by: org.infinispan.commons.CacheException: Initial
>>>>      state transfer
>>>>       >     timed out for cache infinicache-lbd-imei on m4sxhpsrm672-11986
>>>>       >               at
>>>>       >
>>>>        org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
>>>>       >               at
>>>>      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>       >               at
>>>>       >
>>>>        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>       >               at
>>>>       >
>>>>        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>       >               at java.lang.reflect.Method.invoke(Method.java:606)
>>>>       >               at
>>>>       >
>>>>        org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
>>>>       >               ... 14 more
>>>>       >
>>>>       >     This email and any attachment may contain confidential
>>>>      information
>>>>       >     which is intended for use only by the addressee(s) named
>>>>      above. If
>>>>       >     you received this email by mistake, please notify the sender
>>>>       >     immediately, and delete the email from your system. You are
>>>>       >     prohibited from copying, disseminating or otherwise using the
>>>>      email
>>>>       >     or any attachment.
>>>>       >
>>>>       >     _______________________________________________
>>>>       >     infinispan-dev mailing list
>>>>       > infinispan-dev at lists.jboss.org
>>>>      <mailto:infinispan-dev at lists.jboss.org>
>>>>      <mailto:infinispan-dev at lists.jboss.org
>>>>      <mailto:infinispan-dev at lists.jboss.org>>
>>>>       > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>       >
>>>>       >
>>>>
>>>>      This email and any attachment may contain confidential information
>>>>      which is intended for use only by the addressee(s) named above. If
>>>>      you received this email by mistake, please notify the sender
>>>>      immediately, and delete the email from your system. You are
>>>>      prohibited from copying, disseminating or otherwise using the email
>>>>      or any attachment.
>>>>
>>>>      _______________________________________________
>>>>      infinispan-dev mailing list
>>>>      infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>>      https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>>
>>>
>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From andreas.kruthoff at nexustelecom.com  Thu Mar  5 10:32:30 2015
From: andreas.kruthoff at nexustelecom.com (Andreas Kruthoff)
Date: Thu, 5 Mar 2015 16:32:30 +0100
Subject: [infinispan-dev] state transfer timed out, where to configure?
In-Reply-To: <CA+nfvwSuQp4dQUL8sF84Ok16tTgb3=ADTn+NBaXMoEEFCeJGCA@mail.gmail.com>
References: <54CA6D02.1030009@nexustelecom.com>	<CA+nfvwS5M1kEQ4+fphYZZfd3-Fpkb0HzHS_=21ozUKFgHXdxkw@mail.gmail.com>	<54D89072.3010403@nexustelecom.com>	<CA+nfvwT1aceM1jti+A0w+jW962=AW0r1rzHOohFJKyXHvaiRfg@mail.gmail.com>	<54D8C9A5.4000101@nexustelecom.com>	<CA+nfvwRhNUDL48BHOCGOSXMSL8dmzRGxRerGkBDLf5CP4Lwp2w@mail.gmail.com>	<54F857EA.1030002@nexustelecom.com>
	<CA+nfvwSuQp4dQUL8sF84Ok16tTgb3=ADTn+NBaXMoEEFCeJGCA@mail.gmail.com>
Message-ID: <54F8770E.7080204@nexustelecom.com>

Ok, I've corrected it, but I don't know what else I'm doing wrong, the
error is similar:

Exception in thread "main"
org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
find a parser for element 'soft-index-file-store' in namespace
'urn:infinispan:config:store:soft-index:7.1'. Check that your
configuration is up-to date for this version of Infinispan.

Do I need to change the <infinispan xmlns:... > part, too? How? I'm not
familiar with those schema namespace references, unfortunately.

<infinispan
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="urn:infinispan:config:7.1
http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
     xmlns="urn:infinispan:config:7.1"
 >


On 03/05/2015 04:17 PM, Dan Berindei wrote:
> I think the namespace should be
> "urn:infinispan:config:store:soft-index:7.1" (the extra "store" was
> added in 7.1).
>
> On Thu, Mar 5, 2015 at 3:19 PM, Andreas Kruthoff
> <andreas.kruthoff at nexustelecom.com> wrote:
>> Thanks for the information. I'd like to try the SoftIndexFileStore, but
>> no luck with the configuration so far... I'm using version 7.1.1.
>>
>> Exception in thread "main"
>> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
>> find a parser for element 'soft-index-file-store' in namespace
>> 'urn:infinispan:config:soft-index:7.1'. Check that your configuration is
>> up-to date for this version of Infinispan.
>>
>> I was trying this:
>> http://infinispan.org/docs/7.1.x/user_guide/user_guide.html#_soft_index_file_store
>>
>> <infinispan
>>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>       xsi:schemaLocation="urn:infinispan:config:7.1
>> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
>>       xmlns="urn:infinispan:config:7.1">
>>
>>     <jgroups>
>>       <stack-file name="fast" path="jgroups-fast.xml"/>
>>     </jgroups>
>>
>>     <cache-container name="DataCacheManager" statistics="false"
>> default-cache="data-cache" shutdown-hook="DEFAULT">
>>
>>       <transport stack="fast" cluster="data"/>
>>
>>       <local-cache name="with_sifs">
>>         <persistence>
>>           <soft-index-file-store
>> xmlns="urn:infinispan:config:soft-index:7.1">
>>             <index path="/tmp/sifs/testCache/index" />
>>             <data path="/tmp/sifs/testCache/data" />
>>           </soft-index-file-store>
>>         </persistence>
>>       </local-cache>
>>
>>     </cache-container>
>>
>> </infinispan>
>>
>>
>> On 03/05/2015 12:04 PM, Dan Berindei wrote:
>>> The fix for ISPN-5141 is in master now, if you want to give it a try.
>>>
>>> You may also want to try using SoftIndexFileStore instead of
>>> SingleFileStore. SingleFileStore needs to keep all the keys in memory,
>>> so the memory overhead is huge when the values are small.
>>> SoftIndexFileStore doesn't have this problem, and state transfer might
>>> be even faster than with the ISPN-5141 fix for SingleFileStore.
>>>
>>> Cheers
>>> Dan
>>>
>>>
>>> On Mon, Feb 9, 2015 at 4:52 PM, Andreas Kruthoff
>>> <andreas.kruthoff at nexustelecom.com> wrote:
>>>> I've added my vote.
>>>>
>>>> I've increased the chunk size-from 512 entries (default) to 10'000
>>>> entries, as from the xsd: "The number of cache entries to batch in each
>>>> transfer." I thought that an increased chunk-size helps to populate the
>>>> cache faster? Or is it the opposite? One entry is very small, just a
>>>> Long for key and value.
>>>>
>>>>
>>>> On 02/09/2015 02:31 PM, Dan Berindei wrote:
>>>>> I'm not 100% sure, but this sounds like you're hitting ISPN-5141 [1].
>>>>> Please add your vote, it might make it into 7.2.0.
>>>>>
>>>>> Lowering the state transfer chunkSize may also help, especially if your
>>>>> values are large. The chunkSize is the number of entries in a single
>>>>> chunk, you should aim at making the average chunk <= 500KB.
>>>>>
>>>>> Cheers
>>>>> Dan
>>>>>
>>>>> [1] https://issues.jboss.org/browse/ISPN-5141
>>>>>
>>>>>
>>>>> On Mon, Feb 9, 2015 at 11:48 AM, Andreas Kruthoff
>>>>> <andreas.kruthoff at nexustelecom.com
>>>>> <mailto:andreas.kruthoff at nexustelecom.com>> wrote:
>>>>>
>>>>>       Thanks. The cache is a distributed cache with 3 nodes and replication
>>>>>       factor 2. It contains ~40 million entries. As soon as a node has loaded
>>>>>       all entries from disk, new entries are added to this node with high
>>>>>       frequency (several thousand/s).
>>>>>
>>>>>       The other nodes might not be up yet. It takes 'forever' to bring them up
>>>>>       then...?
>>>>>
>>>>>       On 02/03/2015 12:32 PM, Dan Berindei wrote:
>>>>>       > Hi Andreas
>>>>>       >
>>>>>       > Yes, that's the one. 1200s sounds like a huge timeout, though, how big
>>>>>       > is your cache?
>>>>>       >
>>>>>       > Cheers
>>>>>       > Dan
>>>>>       >
>>>>>       >
>>>>>       > On Thu, Jan 29, 2015 at 7:25 PM, Andreas Kruthoff
>>>>>       > <andreas.kruthoff at nexustelecom.com
>>>>>       <mailto:andreas.kruthoff at nexustelecom.com>
>>>>>        > <mailto:andreas.kruthoff at nexustelecom.com
>>>>>       <mailto:andreas.kruthoff at nexustelecom.com>>> wrote:
>>>>>        >
>>>>>        >     Hi dev
>>>>>        >
>>>>>        >
>>>>>        >     I'm running into the following exception on the 3rd node out
>>>>>       of 2.
>>>>>        >     Distributed cluster, file store with a few millions of entries.
>>>>>        >
>>>>>        >     The 3rd node times out during startup, I think. "Initial
>>>>>       state transfer
>>>>>        >     timed out". How can I configure/increase the timeout in my
>>>>>        >     infinispan.xml?
>>>>>        >
>>>>>        >     Is it <state-transfer timeout="1200000"/> within <distributed
>>>>>       cache/> ?
>>>>>        >
>>>>>        >
>>>>>        >     thanks for help
>>>>>        >
>>>>>        >     -andreas
>>>>>        >
>>>>>        >
>>>>>        >     Exception in thread "main"
>>>>>       org.infinispan.commons.CacheException: Unable
>>>>>        >     to invoke method public void
>>>>>        >
>>>>>         org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
>>>>>        >     th
>>>>>        >     rows java.lang.InterruptedException on object of type
>>>>>        >     StateTransferManagerImpl
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
>>>>>        >               at
>>>>>        >     org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:813)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
>>>>>        >               at
>>>>>        >     ch.nexustelecom.lbd.engine.ImeiCache.init(ImeiCache.java:49)
>>>>>        >               at
>>>>>        >
>>>>>         ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
>>>>>        >               at
>>>>>        >
>>>>>         ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
>>>>>        >               at
>>>>>        >
>>>>>         ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
>>>>>        >               at
>>>>>        >
>>>>>         ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
>>>>>        >     Caused by: org.infinispan.commons.CacheException: Initial
>>>>>       state transfer
>>>>>        >     timed out for cache infinicache-lbd-imei on m4sxhpsrm672-11986
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
>>>>>        >               at
>>>>>       sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>        >               at
>>>>>        >
>>>>>         sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>        >               at
>>>>>        >
>>>>>         sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>        >               at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>        >               at
>>>>>        >
>>>>>         org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
>>>>>        >               ... 14 more
>>>>>        >
>>>>>        >     This email and any attachment may contain confidential
>>>>>       information
>>>>>        >     which is intended for use only by the addressee(s) named
>>>>>       above. If
>>>>>        >     you received this email by mistake, please notify the sender
>>>>>        >     immediately, and delete the email from your system. You are
>>>>>        >     prohibited from copying, disseminating or otherwise using the
>>>>>       email
>>>>>        >     or any attachment.
>>>>>        >
>>>>>        >     _______________________________________________
>>>>>        >     infinispan-dev mailing list
>>>>>        > infinispan-dev at lists.jboss.org
>>>>>       <mailto:infinispan-dev at lists.jboss.org>
>>>>>       <mailto:infinispan-dev at lists.jboss.org
>>>>>       <mailto:infinispan-dev at lists.jboss.org>>
>>>>>        > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>        >
>>>>>        >
>>>>>
>>>>>       This email and any attachment may contain confidential information
>>>>>       which is intended for use only by the addressee(s) named above. If
>>>>>       you received this email by mistake, please notify the sender
>>>>>       immediately, and delete the email from your system. You are
>>>>>       prohibited from copying, disseminating or otherwise using the email
>>>>>       or any attachment.
>>>>>
>>>>>       _______________________________________________
>>>>>       infinispan-dev mailing list
>>>>>       infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>>>       https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>>>
>>>>
>>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>
>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.


From rvansa at redhat.com  Thu Mar  5 11:51:53 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 05 Mar 2015 17:51:53 +0100
Subject: [infinispan-dev] state transfer timed out, where to configure?
In-Reply-To: <54F8770E.7080204@nexustelecom.com>
References: <54CA6D02.1030009@nexustelecom.com>	<CA+nfvwS5M1kEQ4+fphYZZfd3-Fpkb0HzHS_=21ozUKFgHXdxkw@mail.gmail.com>	<54D89072.3010403@nexustelecom.com>	<CA+nfvwT1aceM1jti+A0w+jW962=AW0r1rzHOohFJKyXHvaiRfg@mail.gmail.com>	<54D8C9A5.4000101@nexustelecom.com>	<CA+nfvwRhNUDL48BHOCGOSXMSL8dmzRGxRerGkBDLf5CP4Lwp2w@mail.gmail.com>	<54F857EA.1030002@nexustelecom.com>	<CA+nfvwSuQp4dQUL8sF84Ok16tTgb3=ADTn+NBaXMoEEFCeJGCA@mail.gmail.com>
	<54F8770E.7080204@nexustelecom.com>
Message-ID: <54F889A9.4020202@redhat.com>

It seems this discussion is getting long and should be moved to forums 
instead of mailing list.

Anyway, make sure that you're including soft-index file store in your 
dependencies and that it's on the classpath.

Radim

On 03/05/2015 04:32 PM, Andreas Kruthoff wrote:
> Ok, I've corrected it, but I don't know what else I'm doing wrong, the
> error is similar:
>
> Exception in thread "main"
> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
> find a parser for element 'soft-index-file-store' in namespace
> 'urn:infinispan:config:store:soft-index:7.1'. Check that your
> configuration is up-to date for this version of Infinispan.
>
> Do I need to change the <infinispan xmlns:... > part, too? How? I'm not
> familiar with those schema namespace references, unfortunately.
>
> <infinispan
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xsi:schemaLocation="urn:infinispan:config:7.1
> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
>       xmlns="urn:infinispan:config:7.1"
>   >
>
>
> On 03/05/2015 04:17 PM, Dan Berindei wrote:
>> I think the namespace should be
>> "urn:infinispan:config:store:soft-index:7.1" (the extra "store" was
>> added in 7.1).
>>
>> On Thu, Mar 5, 2015 at 3:19 PM, Andreas Kruthoff
>> <andreas.kruthoff at nexustelecom.com> wrote:
>>> Thanks for the information. I'd like to try the SoftIndexFileStore, but
>>> no luck with the configuration so far... I'm using version 7.1.1.
>>>
>>> Exception in thread "main"
>>> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
>>> find a parser for element 'soft-index-file-store' in namespace
>>> 'urn:infinispan:config:soft-index:7.1'. Check that your configuration is
>>> up-to date for this version of Infinispan.
>>>
>>> I was trying this:
>>> http://infinispan.org/docs/7.1.x/user_guide/user_guide.html#_soft_index_file_store
>>>
>>> <infinispan
>>>        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>        xsi:schemaLocation="urn:infinispan:config:7.1
>>> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
>>>        xmlns="urn:infinispan:config:7.1">
>>>
>>>      <jgroups>
>>>        <stack-file name="fast" path="jgroups-fast.xml"/>
>>>      </jgroups>
>>>
>>>      <cache-container name="DataCacheManager" statistics="false"
>>> default-cache="data-cache" shutdown-hook="DEFAULT">
>>>
>>>        <transport stack="fast" cluster="data"/>
>>>
>>>        <local-cache name="with_sifs">
>>>          <persistence>
>>>            <soft-index-file-store
>>> xmlns="urn:infinispan:config:soft-index:7.1">
>>>              <index path="/tmp/sifs/testCache/index" />
>>>              <data path="/tmp/sifs/testCache/data" />
>>>            </soft-index-file-store>
>>>          </persistence>
>>>        </local-cache>
>>>
>>>      </cache-container>
>>>
>>> </infinispan>
>>>
>>>
>>> On 03/05/2015 12:04 PM, Dan Berindei wrote:
>>>> The fix for ISPN-5141 is in master now, if you want to give it a try.
>>>>
>>>> You may also want to try using SoftIndexFileStore instead of
>>>> SingleFileStore. SingleFileStore needs to keep all the keys in memory,
>>>> so the memory overhead is huge when the values are small.
>>>> SoftIndexFileStore doesn't have this problem, and state transfer might
>>>> be even faster than with the ISPN-5141 fix for SingleFileStore.
>>>>
>>>> Cheers
>>>> Dan
>>>>
>>>>
>>>> On Mon, Feb 9, 2015 at 4:52 PM, Andreas Kruthoff
>>>> <andreas.kruthoff at nexustelecom.com> wrote:
>>>>> I've added my vote.
>>>>>
>>>>> I've increased the chunk size-from 512 entries (default) to 10'000
>>>>> entries, as from the xsd: "The number of cache entries to batch in each
>>>>> transfer." I thought that an increased chunk-size helps to populate the
>>>>> cache faster? Or is it the opposite? One entry is very small, just a
>>>>> Long for key and value.
>>>>>
>>>>>
>>>>> On 02/09/2015 02:31 PM, Dan Berindei wrote:
>>>>>> I'm not 100% sure, but this sounds like you're hitting ISPN-5141 [1].
>>>>>> Please add your vote, it might make it into 7.2.0.
>>>>>>
>>>>>> Lowering the state transfer chunkSize may also help, especially if your
>>>>>> values are large. The chunkSize is the number of entries in a single
>>>>>> chunk, you should aim at making the average chunk <= 500KB.
>>>>>>
>>>>>> Cheers
>>>>>> Dan
>>>>>>
>>>>>> [1] https://issues.jboss.org/browse/ISPN-5141
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 9, 2015 at 11:48 AM, Andreas Kruthoff
>>>>>> <andreas.kruthoff at nexustelecom.com
>>>>>> <mailto:andreas.kruthoff at nexustelecom.com>> wrote:
>>>>>>
>>>>>>        Thanks. The cache is a distributed cache with 3 nodes and replication
>>>>>>        factor 2. It contains ~40 million entries. As soon as a node has loaded
>>>>>>        all entries from disk, new entries are added to this node with high
>>>>>>        frequency (several thousand/s).
>>>>>>
>>>>>>        The other nodes might not be up yet. It takes 'forever' to bring them up
>>>>>>        then...?
>>>>>>
>>>>>>        On 02/03/2015 12:32 PM, Dan Berindei wrote:
>>>>>>        > Hi Andreas
>>>>>>        >
>>>>>>        > Yes, that's the one. 1200s sounds like a huge timeout, though, how big
>>>>>>        > is your cache?
>>>>>>        >
>>>>>>        > Cheers
>>>>>>        > Dan
>>>>>>        >
>>>>>>        >
>>>>>>        > On Thu, Jan 29, 2015 at 7:25 PM, Andreas Kruthoff
>>>>>>        > <andreas.kruthoff at nexustelecom.com
>>>>>>        <mailto:andreas.kruthoff at nexustelecom.com>
>>>>>>         > <mailto:andreas.kruthoff at nexustelecom.com
>>>>>>        <mailto:andreas.kruthoff at nexustelecom.com>>> wrote:
>>>>>>         >
>>>>>>         >     Hi dev
>>>>>>         >
>>>>>>         >
>>>>>>         >     I'm running into the following exception on the 3rd node out
>>>>>>        of 2.
>>>>>>         >     Distributed cluster, file store with a few millions of entries.
>>>>>>         >
>>>>>>         >     The 3rd node times out during startup, I think. "Initial
>>>>>>        state transfer
>>>>>>         >     timed out". How can I configure/increase the timeout in my
>>>>>>         >     infinispan.xml?
>>>>>>         >
>>>>>>         >     Is it <state-transfer timeout="1200000"/> within <distributed
>>>>>>        cache/> ?
>>>>>>         >
>>>>>>         >
>>>>>>         >     thanks for help
>>>>>>         >
>>>>>>         >     -andreas
>>>>>>         >
>>>>>>         >
>>>>>>         >     Exception in thread "main"
>>>>>>        org.infinispan.commons.CacheException: Unable
>>>>>>         >     to invoke method public void
>>>>>>         >
>>>>>>          org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
>>>>>>         >     th
>>>>>>         >     rows java.lang.InterruptedException on object of type
>>>>>>         >     StateTransferManagerImpl
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
>>>>>>         >               at
>>>>>>         >     org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:813)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
>>>>>>         >               at
>>>>>>         >     ch.nexustelecom.lbd.engine.ImeiCache.init(ImeiCache.java:49)
>>>>>>         >               at
>>>>>>         >
>>>>>>          ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
>>>>>>         >               at
>>>>>>         >
>>>>>>          ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
>>>>>>         >               at
>>>>>>         >
>>>>>>          ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
>>>>>>         >               at
>>>>>>         >
>>>>>>          ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
>>>>>>         >     Caused by: org.infinispan.commons.CacheException: Initial
>>>>>>        state transfer
>>>>>>         >     timed out for cache infinicache-lbd-imei on m4sxhpsrm672-11986
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
>>>>>>         >               at
>>>>>>        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>         >               at
>>>>>>         >
>>>>>>          sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>         >               at
>>>>>>         >
>>>>>>          sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>         >               at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>         >               at
>>>>>>         >
>>>>>>          org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
>>>>>>         >               ... 14 more
>>>>>>         >
>>>>>>         >     This email and any attachment may contain confidential
>>>>>>        information
>>>>>>         >     which is intended for use only by the addressee(s) named
>>>>>>        above. If
>>>>>>         >     you received this email by mistake, please notify the sender
>>>>>>         >     immediately, and delete the email from your system. You are
>>>>>>         >     prohibited from copying, disseminating or otherwise using the
>>>>>>        email
>>>>>>         >     or any attachment.
>>>>>>         >
>>>>>>         >     _______________________________________________
>>>>>>         >     infinispan-dev mailing list
>>>>>>         > infinispan-dev at lists.jboss.org
>>>>>>        <mailto:infinispan-dev at lists.jboss.org>
>>>>>>        <mailto:infinispan-dev at lists.jboss.org
>>>>>>        <mailto:infinispan-dev at lists.jboss.org>>
>>>>>>         > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>         >
>>>>>>         >
>>>>>>
>>>>>>        This email and any attachment may contain confidential information
>>>>>>        which is intended for use only by the addressee(s) named above. If
>>>>>>        you received this email by mistake, please notify the sender
>>>>>>        immediately, and delete the email from your system. You are
>>>>>>        prohibited from copying, disseminating or otherwise using the email
>>>>>>        or any attachment.
>>>>>>
>>>>>>        _______________________________________________
>>>>>>        infinispan-dev mailing list
>>>>>>        infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>>>>        https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>>
>>>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From andreas.kruthoff at nexustelecom.com  Fri Mar  6 09:53:41 2015
From: andreas.kruthoff at nexustelecom.com (Andreas Kruthoff)
Date: Fri, 6 Mar 2015 15:53:41 +0100
Subject: [infinispan-dev] state transfer timed out, where to configure?
In-Reply-To: <54F889A9.4020202@redhat.com>
References: <54CA6D02.1030009@nexustelecom.com>	<CA+nfvwS5M1kEQ4+fphYZZfd3-Fpkb0HzHS_=21ozUKFgHXdxkw@mail.gmail.com>	<54D89072.3010403@nexustelecom.com>	<CA+nfvwT1aceM1jti+A0w+jW962=AW0r1rzHOohFJKyXHvaiRfg@mail.gmail.com>	<54D8C9A5.4000101@nexustelecom.com>	<CA+nfvwRhNUDL48BHOCGOSXMSL8dmzRGxRerGkBDLf5CP4Lwp2w@mail.gmail.com>	<54F857EA.1030002@nexustelecom.com>	<CA+nfvwSuQp4dQUL8sF84Ok16tTgb3=ADTn+NBaXMoEEFCeJGCA@mail.gmail.com>	<54F8770E.7080204@nexustelecom.com>
	<54F889A9.4020202@redhat.com>
Message-ID: <54F9BF75.8080201@nexustelecom.com>

Thanks, problem was resolved (CLASSPATH issue).
https://developer.jboss.org/message/920753#920753


On 03/05/2015 05:51 PM, Radim Vansa wrote:
> It seems this discussion is getting long and should be moved to forums
> instead of mailing list.
>
> Anyway, make sure that you're including soft-index file store in your
> dependencies and that it's on the classpath.
>
> Radim
>
> On 03/05/2015 04:32 PM, Andreas Kruthoff wrote:
>> Ok, I've corrected it, but I don't know what else I'm doing wrong, the
>> error is similar:
>>
>> Exception in thread "main"
>> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
>> find a parser for element 'soft-index-file-store' in namespace
>> 'urn:infinispan:config:store:soft-index:7.1'. Check that your
>> configuration is up-to date for this version of Infinispan.
>>
>> Do I need to change the <infinispan xmlns:... > part, too? How? I'm not
>> familiar with those schema namespace references, unfortunately.
>>
>> <infinispan
>>        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>        xsi:schemaLocation="urn:infinispan:config:7.1
>> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
>>        xmlns="urn:infinispan:config:7.1"
>>    >
>>
>>
>> On 03/05/2015 04:17 PM, Dan Berindei wrote:
>>> I think the namespace should be
>>> "urn:infinispan:config:store:soft-index:7.1" (the extra "store" was
>>> added in 7.1).
>>>
>>> On Thu, Mar 5, 2015 at 3:19 PM, Andreas Kruthoff
>>> <andreas.kruthoff at nexustelecom.com> wrote:
>>>> Thanks for the information. I'd like to try the SoftIndexFileStore, but
>>>> no luck with the configuration so far... I'm using version 7.1.1.
>>>>
>>>> Exception in thread "main"
>>>> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
>>>> find a parser for element 'soft-index-file-store' in namespace
>>>> 'urn:infinispan:config:soft-index:7.1'. Check that your configuration is
>>>> up-to date for this version of Infinispan.
>>>>
>>>> I was trying this:
>>>> http://infinispan.org/docs/7.1.x/user_guide/user_guide.html#_soft_index_file_store
>>>>
>>>> <infinispan
>>>>         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>         xsi:schemaLocation="urn:infinispan:config:7.1
>>>> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
>>>>         xmlns="urn:infinispan:config:7.1">
>>>>
>>>>       <jgroups>
>>>>         <stack-file name="fast" path="jgroups-fast.xml"/>
>>>>       </jgroups>
>>>>
>>>>       <cache-container name="DataCacheManager" statistics="false"
>>>> default-cache="data-cache" shutdown-hook="DEFAULT">
>>>>
>>>>         <transport stack="fast" cluster="data"/>
>>>>
>>>>         <local-cache name="with_sifs">
>>>>           <persistence>
>>>>             <soft-index-file-store
>>>> xmlns="urn:infinispan:config:soft-index:7.1">
>>>>               <index path="/tmp/sifs/testCache/index" />
>>>>               <data path="/tmp/sifs/testCache/data" />
>>>>             </soft-index-file-store>
>>>>           </persistence>
>>>>         </local-cache>
>>>>
>>>>       </cache-container>
>>>>
>>>> </infinispan>
>>>>
>>>>
>>>> On 03/05/2015 12:04 PM, Dan Berindei wrote:
>>>>> The fix for ISPN-5141 is in master now, if you want to give it a try.
>>>>>
>>>>> You may also want to try using SoftIndexFileStore instead of
>>>>> SingleFileStore. SingleFileStore needs to keep all the keys in memory,
>>>>> so the memory overhead is huge when the values are small.
>>>>> SoftIndexFileStore doesn't have this problem, and state transfer might
>>>>> be even faster than with the ISPN-5141 fix for SingleFileStore.
>>>>>
>>>>> Cheers
>>>>> Dan
>>>>>
>>>>>
>>>>> On Mon, Feb 9, 2015 at 4:52 PM, Andreas Kruthoff
>>>>> <andreas.kruthoff at nexustelecom.com> wrote:
>>>>>> I've added my vote.
>>>>>>
>>>>>> I've increased the chunk size-from 512 entries (default) to 10'000
>>>>>> entries, as from the xsd: "The number of cache entries to batch in each
>>>>>> transfer." I thought that an increased chunk-size helps to populate the
>>>>>> cache faster? Or is it the opposite? One entry is very small, just a
>>>>>> Long for key and value.
>>>>>>
>>>>>>
>>>>>> On 02/09/2015 02:31 PM, Dan Berindei wrote:
>>>>>>> I'm not 100% sure, but this sounds like you're hitting ISPN-5141 [1].
>>>>>>> Please add your vote, it might make it into 7.2.0.
>>>>>>>
>>>>>>> Lowering the state transfer chunkSize may also help, especially if your
>>>>>>> values are large. The chunkSize is the number of entries in a single
>>>>>>> chunk, you should aim at making the average chunk <= 500KB.
>>>>>>>
>>>>>>> Cheers
>>>>>>> Dan
>>>>>>>
>>>>>>> [1] https://issues.jboss.org/browse/ISPN-5141
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Feb 9, 2015 at 11:48 AM, Andreas Kruthoff
>>>>>>> <andreas.kruthoff at nexustelecom.com
>>>>>>> <mailto:andreas.kruthoff at nexustelecom.com>> wrote:
>>>>>>>
>>>>>>>         Thanks. The cache is a distributed cache with 3 nodes and replication
>>>>>>>         factor 2. It contains ~40 million entries. As soon as a node has loaded
>>>>>>>         all entries from disk, new entries are added to this node with high
>>>>>>>         frequency (several thousand/s).
>>>>>>>
>>>>>>>         The other nodes might not be up yet. It takes 'forever' to bring them up
>>>>>>>         then...?
>>>>>>>
>>>>>>>         On 02/03/2015 12:32 PM, Dan Berindei wrote:
>>>>>>>         > Hi Andreas
>>>>>>>         >
>>>>>>>         > Yes, that's the one. 1200s sounds like a huge timeout, though, how big
>>>>>>>         > is your cache?
>>>>>>>         >
>>>>>>>         > Cheers
>>>>>>>         > Dan
>>>>>>>         >
>>>>>>>         >
>>>>>>>         > On Thu, Jan 29, 2015 at 7:25 PM, Andreas Kruthoff
>>>>>>>         > <andreas.kruthoff at nexustelecom.com
>>>>>>>         <mailto:andreas.kruthoff at nexustelecom.com>
>>>>>>>          > <mailto:andreas.kruthoff at nexustelecom.com
>>>>>>>         <mailto:andreas.kruthoff at nexustelecom.com>>> wrote:
>>>>>>>          >
>>>>>>>          >     Hi dev
>>>>>>>          >
>>>>>>>          >
>>>>>>>          >     I'm running into the following exception on the 3rd node out
>>>>>>>         of 2.
>>>>>>>          >     Distributed cluster, file store with a few millions of entries.
>>>>>>>          >
>>>>>>>          >     The 3rd node times out during startup, I think. "Initial
>>>>>>>         state transfer
>>>>>>>          >     timed out". How can I configure/increase the timeout in my
>>>>>>>          >     infinispan.xml?
>>>>>>>          >
>>>>>>>          >     Is it <state-transfer timeout="1200000"/> within <distributed
>>>>>>>         cache/> ?
>>>>>>>          >
>>>>>>>          >
>>>>>>>          >     thanks for help
>>>>>>>          >
>>>>>>>          >     -andreas
>>>>>>>          >
>>>>>>>          >
>>>>>>>          >     Exception in thread "main"
>>>>>>>         org.infinispan.commons.CacheException: Unable
>>>>>>>          >     to invoke method public void
>>>>>>>          >
>>>>>>>           org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
>>>>>>>          >     th
>>>>>>>          >     rows java.lang.InterruptedException on object of type
>>>>>>>          >     StateTransferManagerImpl
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
>>>>>>>          >               at
>>>>>>>          >     org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:813)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
>>>>>>>          >               at
>>>>>>>          >     ch.nexustelecom.lbd.engine.ImeiCache.init(ImeiCache.java:49)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
>>>>>>>          >     Caused by: org.infinispan.commons.CacheException: Initial
>>>>>>>         state transfer
>>>>>>>          >     timed out for cache infinicache-lbd-imei on m4sxhpsrm672-11986
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
>>>>>>>          >               at
>>>>>>>         sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>          >               at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>          >               at
>>>>>>>          >
>>>>>>>           org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
>>>>>>>          >               ... 14 more
>>>>>>>          >
>>>>>>>          >     This email and any attachment may contain confidential
>>>>>>>         information
>>>>>>>          >     which is intended for use only by the addressee(s) named
>>>>>>>         above. If
>>>>>>>          >     you received this email by mistake, please notify the sender
>>>>>>>          >     immediately, and delete the email from your system. You are
>>>>>>>          >     prohibited from copying, disseminating or otherwise using the
>>>>>>>         email
>>>>>>>          >     or any attachment.
>>>>>>>          >
>>>>>>>          >     _______________________________________________
>>>>>>>          >     infinispan-dev mailing list
>>>>>>>          > infinispan-dev at lists.jboss.org
>>>>>>>         <mailto:infinispan-dev at lists.jboss.org>
>>>>>>>         <mailto:infinispan-dev at lists.jboss.org
>>>>>>>         <mailto:infinispan-dev at lists.jboss.org>>
>>>>>>>          > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>          >
>>>>>>>          >
>>>>>>>
>>>>>>>         This email and any attachment may contain confidential information
>>>>>>>         which is intended for use only by the addressee(s) named above. If
>>>>>>>         you received this email by mistake, please notify the sender
>>>>>>>         immediately, and delete the email from your system. You are
>>>>>>>         prohibited from copying, disseminating or otherwise using the email
>>>>>>>         or any attachment.
>>>>>>>
>>>>>>>         _______________________________________________
>>>>>>>         infinispan-dev mailing list
>>>>>>>         infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>>>>>         https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>
>>>>>>>
>>>>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>
>>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.


From rory.odonnell at oracle.com  Tue Mar 10 07:03:10 2015
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Tue, 10 Mar 2015 11:03:10 +0000
Subject: [infinispan-dev] Early Access builds for JDK 9 b53 and JDK 8u60 b05
 are available on java.net
Message-ID: <54FECF6E.1060305@oracle.com>


Hi Galder,

Early Access build for JDK 9 b53 <https://jdk9.java.net/download/> 
available on java.net, summary of  changes are listed here 
<http://www.java.net/download/jdk9/changes/jdk9-b53.html>

Early Access build for JDK 8u60 b05 <http://jdk8.java.net/download.html> 
is available on java.net, summary of changes are listed here. 
<http://www.java.net/download/jdk8u60/changes/jdk8u60-b05.html>

I'd also like to use this opportunity to point you to JEP 238: 
Multi-Version JAR Files [0],
which is currently a Candidate JEP for JDK 9.

It's goal is to extend the JAR file format to allow multiple, JDK 
release-specific versions of class
files to coexist in a single file. An additional goal is to backport the 
run-time changes to
JDK 8u60, thereby enabling JDK 8 to consume multi-version JARs. For a 
detailed discussion,
please see the corresponding thread on the core-libs-dev mailing list. [1]

Please keep in mind that a JEP in the Candidate state is merely an idea 
worthy of consideration
by JDK Release Projects and related efforts; there is no commitment that 
it will be delivered in
any particular release.

Comments, questions, and suggestions are welcome on the corelibs-dev 
mailing list. (If you
haven?t already subscribed to that list then please do so first, 
otherwise your message will be
discarded as spam.)

Rgds,Rory

[0] http://openjdk.java.net/jeps/238
[1] 
http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-February/031461.html 



-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20150310/1fd8c6bb/attachment.html 

From rvansa at redhat.com  Thu Mar 12 11:07:22 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 12 Mar 2015 16:07:22 +0100
Subject: [infinispan-dev] CacheEntry vs. Metadata
In-Reply-To: <54F083FF.9000802@redhat.com>
References: <54F083FF.9000802@redhat.com>
Message-ID: <5501ABAA.9020802@redhat.com>

Asking this on Friday afternoon was probably not the best time, so I'll 
try to draw some attention on Thursday :)

Radim

On 02/27/2015 03:49 PM, Radim Vansa wrote:
> Hi,
>
> I was already several times looking on the class hierarchy of CacheEntry
> and its descendants. Since the documentation of those interfaces is
> usually a one liner, I'd like to ask for the big picture:
>
> So we have CacheEntry, which implements MetadataAware - therefore, it
> contains a metadata, which define lifespan, maxIdle time and version.
> However, even the CacheEntry interface itself contains getters for
> lifespan and idle time and MortalCacheEntry hosts the fields - so I see
> that there's some duplication with the Metadata object.
> Beyond the expiration-related stuff (and common key - value getters),
> CacheEntry has several methods querying and manipulating its state -
> isChanged, isValid, isRemoved etc. It's a bit confusing that this is
> presented not as a single state but rather a matrix of boolean states.
> When I've tried to implement EntryProcessor several weeks ago (I've
> stopped the attempts since this should be implemented in Infinispan 8),
> I had quite a hard time figuring out which should be set and how in case
> I want to update/remove the entry. undelete() and skipLookup() are not
> obvious, either.
>
> Is the reason for having Immortal/Mortal/Transient/TransientMortal
> entries + Metadata* versions + *Values to optimally use memory?
>
> Then there are the ReadCommitted and RepeatableRead entries - are these
> ever stored in data container, or just in context? What's the exact
> relation between those implementing InternalCacheEntry and MVCCEntry?
>
> Then there's the DeltaAwareCacheEntry - this does not fit to the image
> for me at all.
>
> I am also not sure about the relation of EmbeddedMetadata and
> InternalMetadataImpl
>
> Thanks for your insight!
>
> Radim
>
>


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From galder at redhat.com  Fri Mar 13 04:10:09 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 13 Mar 2015 09:10:09 +0100
Subject: [infinispan-dev] Cache Java 8 API proposal - First draft
Message-ID: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>

Hi all,

We're starting to work on some prototypes for the Java 8 API coming up in Infinispan 8.

I've written a design wiki for a replacement of our main embedded Cache interface that takes advantage of Java 8 improvements:
https://github.com/infinispan/infinispan/wiki/Java-8-API-proposal

It'd be great to hear your thoughts.

Proposals for Java 8 versions for cache manager, remote cache and other external APIs are yet TBD.

Regards,
--
Galder Zamarre?o
galder at redhat.com






From rvansa at redhat.com  Fri Mar 13 05:20:53 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Fri, 13 Mar 2015 10:20:53 +0100
Subject: [infinispan-dev] Cache Java 8 API proposal - First draft
In-Reply-To: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>
References: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>
Message-ID: <5502ABF5.40703@redhat.com>

Q1: What about versions? I really like the approach taken for 
conditional operations in RemoteCache, because versions (as opposed to 
replace(key, old, new)) get rid of ABA problems, wouldn't have problems 
in return values after operation retry during topology change, reduce 
overhead of sending big old values etc... The functional API luckily 
allows 'any' operation, but this seems to me as such an useful 
functionality that we should not force users implementing versions in 
their values (unless they want to); versions should be part of Metadata 
(at least I hope so - see CacheEntry vs. Metadata thread). Since this 
API should be the most comprehensive one on which we'll build even more 
complicated data structures than atomic counters, I somehow lack the 
access to Metadata there (though I understand that you didn't want to 
bloat it).

Q2: For performance, I would expect that a (replicated?) cache for 
well-known/registered lambdas could be useful. Then, you'd need the 
functional API with direct lambda IDs - you don't want to do the map 
lookup every time. Or any other ideas? I think you said that you 
benchmarked the serialization overhead, can you present the numbers?

Q3: Do you expect that in non-tx mode the primary will replicate to 
backup the lambda, or full value? With non-trivial lambdas, after node 
crash when the backups have different values, this can lead to forever 
diverging values. But yes, this could be chosen with a flag.

Radim

On 03/13/2015 09:10 AM, Galder Zamarre?o wrote:
> Hi all,
>
> We're starting to work on some prototypes for the Java 8 API coming up in Infinispan 8.
>
> I've written a design wiki for a replacement of our main embedded Cache interface that takes advantage of Java 8 improvements:
> https://github.com/infinispan/infinispan/wiki/Java-8-API-proposal
>
> It'd be great to hear your thoughts.
>
> Proposals for Java 8 versions for cache manager, remote cache and other external APIs are yet TBD.
>
> Regards,
> --
> Galder Zamarre?o
> galder at redhat.com
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From bban at redhat.com  Sat Mar 14 04:39:34 2015
From: bban at redhat.com (Bela Ban)
Date: Sat, 14 Mar 2015 09:39:34 +0100
Subject: [infinispan-dev] Cache Java 8 API proposal - First draft
In-Reply-To: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>
References: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>
Message-ID: <5503F3C6.9080503@redhat.com>

Hi Galder,

apologies for the long reply !

imo the value of Java 8 is:

#1 Reduction of implementation code (streams and internal iteration)

#2 Making some of the APIs simpler, e.g. distributed executors, map-reduce

#3 Reactive programming, e.g. interceptor chain (discussed in Berlin), 
non-blocking behavior; the goal being to reduce the number of threads 
required.

However, changing the entire Cache API is going too far atm. Now that 
we've finally provided a Cache/JSR107 API, people are familiar with it 
and I would not change it, at least not right now.

If we're able to use reactive programming (e.g. your eval() function) 
*under the cover*, then all the better.

If your proposal is about providing FunctionalCache and implementing 
Cache and JSR 107 Cache with it, then you have my +1. But IMO 
FunctionalCache should not be the main API.

Compare FunctionalCache.eval() to ioctl() for a moment: of course, every 
access to a file system can be implemented via ioctl(), but people like 
the simplicity of write(), read(), stat() etc.

So providing eval() *only* shifts the burden of implementation to the 
user. But I agree that perhaps it's a good thing to expose 
FunctionalCache so power users (e.g. Sanne :-)) can use it directly to 
implement functionality not provided by Cache.

Being able to switch to Java 8 doesn't imply we have to use all of its 
new features.

Let me address your proposal step by step.



"What's wrong with the Infinispan Cache API?"
--------------------------------------
- Nothing !
- I can't follow your argument as to why Cache is bad. This is the 
premise to your proposal and it needs to be very clear !
- JSR 107 came up with a similar API. So if we have to change Cache, 
then I'd +1 a change to JSR 107 as main API
- I don't see why it's difficult to add new operations: distributed 
queues should be a separate API anyway, and not extend Cache ! We should 
use CacheManager to get a new DistributedQueue, as we do to get a new 
Cache...


"How does Java 8 improve on this?"
------------------------------
- Passing serialized lambdas around sounds awfully familiar: it's the 
old 'agent-based programming' paradigm fashionable before the turn of 
the millenium. Instead of invoking RPCs which carry data on P, code 
(also carrying data) is shipped to P and executed there and the result 
sent back to the invoker. (Disclaimer: I wrote such a system myself with 
GOMScript... :-().
I'm not saying this is bad, but we have to make sure that
(a) Infinispan benefits from this either in code reduction / 
maintainability / code simplification
(b) performance doesn't decrease



"Proposal"
---------
- Returning a CompletableFuture is a good idea: +1
- I think this is where the main value of your proposal lies: being able 
to chain functions (lambdas) together, that are invoked turn-by-turn, 
possibly by different threads, without a caller having to block
- The F.invoke(G).andThenInvoke(H).andThenInvoke(I) pattern (where F is 
a future and G,H,I are lambdas) is very valuable to implement 
non-blocking behavior: whenever a function (lambda) has to block, it 
registers a function with the blocking resource that's called upon 
unblocking and the thread terminates and is added back to the thread pool
- Blocking can be waiting to read from the file system or network, or 
trying to acquire a lock etc
- As was discussed in Berlin, the interceptor chain could be rewritten 
using CompletableFutures (or sth similar).


Summary
--------

I'd not make FunctionalCache the main API and I'd definitely not 
deprecate Cache.

+1 on providing FunctionalCache as a (parent) interface to Cache, so 
power users can use FunctionalCache if needed.

+1 on implementing Cache and JSR 107 Cache with FunctionalCache.

This should only be done if:
- Performance remains at least as good as it is now [mandatory]
- Maintainability increases [nice]
- Code reduction [nice]

My 5 cents


On 13/03/15 09:10, Galder Zamarre?o wrote:
> Hi all,
>
> We're starting to work on some prototypes for the Java 8 API coming up in Infinispan 8.
>
> I've written a design wiki for a replacement of our main embedded Cache interface that takes advantage of Java 8 improvements:
> https://github.com/infinispan/infinispan/wiki/Java-8-API-proposal
>
> It'd be great to hear your thoughts.
>
> Proposals for Java 8 versions for cache manager, remote cache and other external APIs are yet TBD.
>
> Regards,
> --
> Galder Zamarre?o
> galder at redhat.com

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)

From ttarrant at redhat.com  Mon Mar 16 11:47:01 2015
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 16 Mar 2015 16:47:01 +0100
Subject: [infinispan-dev] Weekly IRC Meeting Logs 2015-03-16
Message-ID: <5506FAF5.6060008@redhat.com>

Here they are:

http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2015/infinispan.2015-03-16-15.01.log.html

Tristan

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat


From galder at redhat.com  Tue Mar 17 05:45:03 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 17 Mar 2015 10:45:03 +0100
Subject: [infinispan-dev] CacheEntry vs. Metadata
In-Reply-To: <54F083FF.9000802@redhat.com>
References: <54F083FF.9000802@redhat.com>
Message-ID: <B53F3C3B-E35D-444C-93BB-133DD24008BA@redhat.com>

Hi Radim,

See responses inline.

> On 27 Feb 2015, at 15:49, Radim Vansa <rvansa at redhat.com> wrote:
> 
> Hi,
> 
> I was already several times looking on the class hierarchy of CacheEntry 
> and its descendants. Since the documentation of those interfaces is 
> usually a one liner, I'd like to ask for the big picture:
> 
> So we have CacheEntry, which implements MetadataAware - therefore, it 
> contains a metadata, which define lifespan, maxIdle time and version. 
> However, even the CacheEntry interface itself contains getters for 
> lifespan and idle time and MortalCacheEntry hosts the fields - so I see 
> that there's some duplication with the Metadata object.

Originally, there was only CacheEntry, we added MetadataAware later on.

To avoid causing too much havoc, we decided to leave existing CacheEntry methods in there.

> Beyond the expiration-related stuff (and common key - value getters), 
> CacheEntry has several methods querying and manipulating its state - 
> isChanged, isValid, isRemoved etc. It's a bit confusing that this is 
> presented not as a single state but rather a matrix of boolean states. 

^ I agree.

> When I've tried to implement EntryProcessor several weeks ago (I've 
> stopped the attempts since this should be implemented in Infinispan 8), 
> I had quite a hard time figuring out which should be set and how in case 
> I want to update/remove the entry. undelete() and skipLookup() are not 
> obvious, either.

^ Right, we've not made any changes to it because with Java 8, we'd want a way to manipulate the entry in a much saner way.

> Is the reason for having Immortal/Mortal/Transient/TransientMortal 
> entries + Metadata* versions + *Values to optimally use memory?

Hmmm, the different subclasses were created to try to make sure only the data relevant to the use case is maintained as much as possible and avoid empty fields.

> Then there are the ReadCommitted and RepeatableRead entries - are these 
> ever stored in data container, or just in context? What's the exact 
> relation between those implementing InternalCacheEntry and MVCCEntry?

RC and RR are wrappers that provide RC and RR guaranteees :)

No, they're not stored as such in the DC. They're only use for guarantees.

> Then there's the DeltaAwareCacheEntry - this does not fit to the image 
> for me at all.

^ Not sure about that one, I guess it's for handling delta aware entries?

> I am also not sure about the relation of EmbeddedMetadata and 
> InternalMetadataImpl

When metadata layer was added, I created EmbeddedMetadata as implementation.

InternalMetadataImpl appears to have been created by Mircea when reworking persistence layer, so he possibly created it to have it persisted. Nor its name nor its intention seems clear either.

NOTE:

Cache entry and metadata will be even more important in Java 8 API because this will be what we expose to the eval functions and users can manipulate it to do execute all sorts of functions. Cache entry and/or metadata will most likely be revamped into new interfaces (as already hinted by the prototype), so I'd not worry too much about changing what we have now, but make sure that we'll have fits properly. 

Thanks for the questions and feedback :)

Cheers,

> 
> Thanks for your insight!
> 
> Radim
> 
> 
> -- 
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From dan.berindei at gmail.com  Tue Mar 17 17:49:49 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Tue, 17 Mar 2015 23:49:49 +0200
Subject: [infinispan-dev] Infinispan 7.2.0.Beta1 released
Message-ID: <CA+nfvwTq6P5W=zrGs7iWUcvg-2y3KtS7DU7PtAmK8Fm7dn7U4A@mail.gmail.com>

Dear Infinispan community,

Infinispan 7.2.0.Beta1 is now available!

Along the usual assortment of bug fixes, this release includes a few
exciting new features:

* Server-side scripting with JSR-223 (ISPN-5013)
* Initial support for the JCache API over HotRod (ISPN-4955)
* Improved size-based eviction, implemented on top of Doug Lea's
ConcurrentHashMapV8 (ISPN-3023)

The blog post has more details (and links):
http://blog.infinispan.org/2015/03/infinispan-720beta1-released.html


Cheers
Dan

From galder at redhat.com  Wed Mar 18 12:27:00 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 18 Mar 2015 17:27:00 +0100
Subject: [infinispan-dev] Cache Java 8 API proposal - First draft
In-Reply-To: <5502ABF5.40703@redhat.com>
References: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>
	<5502ABF5.40703@redhat.com>
Message-ID: <5545A6A1-30F2-44FE-8E6A-51F76021EE6C@redhat.com>

Thanks Radim for the great feedback, comments inline:

> On 13 Mar 2015, at 10:20, Radim Vansa <rvansa at redhat.com> wrote:
> 
> Q1: What about versions? I really like the approach taken for 
> conditional operations in RemoteCache, because versions (as opposed to 
> replace(key, old, new)) get rid of ABA problems, wouldn't have problems 
> in return values after operation retry during topology change, reduce 
> overhead of sending big old values etc... The functional API luckily 
> allows 'any' operation, but this seems to me as such an useful 
> functionality that we should not force users implementing versions in 
> their values (unless they want to); versions should be part of Metadata 
> (at least I hope so - see CacheEntry vs. Metadata thread). Since this 
> API should be the most comprehensive one on which we'll build even more 
> complicated data structures than atomic counters, I somehow lack the 
> access to Metadata there (though I understand that you didn't want to 
> bloat it).

^ `Value` class will expose metadata and versioning information somehow, but as said in the proposal, there are still some things that need to be added.

Exposing version of the existing entry to the functions is crucial because one of the problems I'm trying to find a good solution for is [1] (see also slides in [2] that I prepared for Berlin). The problem there is that existing replace() is only atomic when comparing objects by equality, and the `eval` method design will allow you to customize that comparison the way you want, and in fact for Hot Rod, we'd compare using version information. E.g. here's how it could work (TBD!):

CompletableFuture<Boolean> replaced = functionalCache.eval("e", v -> {
   if (v.version().equals(userProvidedVersion)) return true;
   else return false;
});

[1] https://issues.jboss.org/browse/ISPN-4972
[2] https://dl.dropboxusercontent.com/u/6148072/berlin-2015-galder-versionedreplace.pdf

> Q2: For performance, I would expect that a (replicated?) cache for 
> well-known/registered lambdas could be useful. Then, you'd need the 
> functional API with direct lambda IDs - you don't want to do the map 
> lookup every time. Or any other ideas? I think you said that you 
> benchmarked the serialization overhead, can you present the numbers?

I've pinged David, JBoss Marshalling creator, to get some tips about this and I also plan to check with other Java 8 gurus such as Mario, but from what I can understand, there's three parts to a lambda: 

- the lambda function class instance itself
- the parameters you pass in to the lambda
- any objects captured/referenced by the lambda

I think the lambda functions can be cached but of course, the parameters nor captured objects can't. When we create Map/ConcurrentMap/JCache decorators, we'll control the decorator contents and we'd have the chance to do whatever caching we need. I don't think we even need to replicate them at all since the classes themselves will cache the functions.

So, that would leave parameters and captured/referenced objects to be serialized since that's the variable part. Not sure if these two can be serialized independently. Parameters maybe but captured objects might be tough.

I've not done benchmarking yet.

> Q3: Do you expect that in non-tx mode the primary will replicate to 
> backup the lambda, or full value? With non-trivial lambdas, after node 
> crash when the backups have different values, this can lead to forever 
> diverging values. But yes, this could be chosen with a flag.

You'd replicate the lambda (or if you can cache it, then somehow avoid sending all its info), along with parameters, and any capture values. So, for `Map.put` implementation function you are replicating, it includes the full value.

If you're trying to store a data structure and and non-tx, there's indeed a chance of having partially applied updates. We'd need some merging here possibly...

Cheers,

> 
> Radim
> 
> On 03/13/2015 09:10 AM, Galder Zamarre?o wrote:
>> Hi all,
>> 
>> We're starting to work on some prototypes for the Java 8 API coming up in Infinispan 8.
>> 
>> I've written a design wiki for a replacement of our main embedded Cache interface that takes advantage of Java 8 improvements:
>> https://github.com/infinispan/infinispan/wiki/Java-8-API-proposal
>> 
>> It'd be great to hear your thoughts.
>> 
>> Proposals for Java 8 versions for cache manager, remote cache and other external APIs are yet TBD.
>> 
>> Regards,
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> -- 
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From rvansa at redhat.com  Wed Mar 18 13:23:16 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 18 Mar 2015 18:23:16 +0100
Subject: [infinispan-dev] Cache Java 8 API proposal - First draft
In-Reply-To: <5545A6A1-30F2-44FE-8E6A-51F76021EE6C@redhat.com>
References: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>	<5502ABF5.40703@redhat.com>
	<5545A6A1-30F2-44FE-8E6A-51F76021EE6C@redhat.com>
Message-ID: <5509B484.4000405@redhat.com>

On 03/18/2015 05:27 PM, Galder Zamarre?o wrote:
> Thanks Radim for the great feedback, comments inline:
>
>> On 13 Mar 2015, at 10:20, Radim Vansa <rvansa at redhat.com> wrote:
>>
>> Q1: What about versions? I really like the approach taken for
>> conditional operations in RemoteCache, because versions (as opposed to
>> replace(key, old, new)) get rid of ABA problems, wouldn't have problems
>> in return values after operation retry during topology change, reduce
>> overhead of sending big old values etc... The functional API luckily
>> allows 'any' operation, but this seems to me as such an useful
>> functionality that we should not force users implementing versions in
>> their values (unless they want to); versions should be part of Metadata
>> (at least I hope so - see CacheEntry vs. Metadata thread). Since this
>> API should be the most comprehensive one on which we'll build even more
>> complicated data structures than atomic counters, I somehow lack the
>> access to Metadata there (though I understand that you didn't want to
>> bloat it).
> ^ `Value` class will expose metadata and versioning information somehow, but as said in the proposal, there are still some things that need to be added.
>
> Exposing version of the existing entry to the functions is crucial because one of the problems I'm trying to find a good solution for is [1] (see also slides in [2] that I prepared for Berlin). The problem there is that existing replace() is only atomic when comparing objects by equality, and the `eval` method design will allow you to customize that comparison the way you want, and in fact for Hot Rod, we'd compare using version information. E.g. here's how it could work (TBD!):
>
> CompletableFuture<Boolean> replaced = functionalCache.eval("e", v -> {
>     if (v.version().equals(userProvidedVersion)) return true;
>     else return false;
> });
>
> [1] https://issues.jboss.org/browse/ISPN-4972
> [2] https://dl.dropboxusercontent.com/u/6148072/berlin-2015-galder-versionedreplace.pdf

I didn't expect that you forgot about versions, I just think that if 
this is not in the suggested API now, it's too early to review it. So 
you expect that version/metadata getters and setters will be part of the 
Value interface (as opposed to second argument to the lambda)? I am 
looking forward to updated wiki.

>
>> Q2: For performance, I would expect that a (replicated?) cache for
>> well-known/registered lambdas could be useful. Then, you'd need the
>> functional API with direct lambda IDs - you don't want to do the map
>> lookup every time. Or any other ideas? I think you said that you
>> benchmarked the serialization overhead, can you present the numbers?
> I've pinged David, JBoss Marshalling creator, to get some tips about this and I also plan to check with other Java 8 gurus such as Mario, but from what I can understand, there's three parts to a lambda:
>
> - the lambda function class instance itself
> - the parameters you pass in to the lambda
> - any objects captured/referenced by the lambda
>
> I think the lambda functions can be cached but of course, the parameters nor captured objects can't. When we create Map/ConcurrentMap/JCache decorators, we'll control the decorator contents and we'd have the chance to do whatever caching we need. I don't think we even need to replicate them at all since the classes themselves will cache the functions.

I don't really understand the last sentence. Let's consider only the 
function class instance; how that would be cached in the decorators? 
Because even if you send the same instance twice, IMO, it will be sent 
over the wire twice, Marshalling won't do any caching magic on its own. 
Or am I mistaken? And will the FunctionalCache API directly support this 
caching anyhow (such as version of methods with ID instead of the lambda)?

>
> So, that would leave parameters and captured/referenced objects to be serialized since that's the variable part. Not sure if these two can be serialized independently. Parameters maybe but captured objects might be tough.
>
> I've not done benchmarking yet.
>
>> Q3: Do you expect that in non-tx mode the primary will replicate to
>> backup the lambda, or full value? With non-trivial lambdas, after node
>> crash when the backups have different values, this can lead to forever
>> diverging values. But yes, this could be chosen with a flag.
> You'd replicate the lambda (or if you can cache it, then somehow avoid sending all its info), along with parameters, and any capture values. So, for `Map.put` implementation function you are replicating, it includes the full value.
>
> If you're trying to store a data structure and and non-tx, there's indeed a chance of having partially applied updates. We'd need some merging here possibly...

Let's have the increment() lambda; during topology change, this could 
arrive to the backup two times. Merging does not help much if you want 
to implement atomic counters in cache.

Radim

>
> Cheers,
>
>> Radim
>>
>> On 03/13/2015 09:10 AM, Galder Zamarre?o wrote:
>>> Hi all,
>>>
>>> We're starting to work on some prototypes for the Java 8 API coming up in Infinispan 8.
>>>
>>> I've written a design wiki for a replacement of our main embedded Cache interface that takes advantage of Java 8 improvements:
>>> https://github.com/infinispan/infinispan/wiki/Java-8-API-proposal
>>>
>>> It'd be great to hear your thoughts.
>>>
>>> Proposals for Java 8 versions for cache manager, remote cache and other external APIs are yet TBD.
>>>
>>> Regards,
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> -- 
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Galder Zamarre?o
> galder at redhat.com
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From jmarkos at redhat.com  Thu Mar 19 12:16:12 2015
From: jmarkos at redhat.com (Jakub Markos)
Date: Thu, 19 Mar 2015 12:16:12 -0400 (EDT)
Subject: [infinispan-dev] FindBugs on infinispan
In-Reply-To: <1624798272.9697831.1426781681369.JavaMail.zimbra@redhat.com>
Message-ID: <2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>

Hi,

out of curiosity, I ran FindBugs [1] (java static analysis tool) on infinispan. Attached are 2 output files (compressed, because the mailing list accepts only <0.5MB attachments):
1. for the release-jars-analysis.html, only the 3 main jars from 7.2.0.Alpha1 were analyzed
2. for the project-jars-analysis.html, I took all jars that were present in the infinispan project directory after a build (except the test jars) - this one has therefore more findings

In both cases, FindBugs complained that it couldn't find some imported classes, so the analysis may not be 100% complete.

I didn't look through the actual results much, but for example it detected an infinite loop at [2], or a self-assignment at [3].

If you want to run it yourself, you can use [4]. There is also a GUI, but I wasn't able to save the results into a html, and a maven
plugin [5], but it only creates the results in an xml format for each module separately, so it's not very useful.

Jakub

[1] http://findbugs.sourceforge.net/
[2] https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/core/src/main/java/org/infinispan/context/impl/ImmutableContext.java#L95
[3] https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/transport/tcp/TcpTransportFactory.java#L90
[4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress -release infinispan -html -output infinispan-findbugs-analysis -onlyAnalyze org.infinispan.- infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar
[5] http://mojo.codehaus.org/findbugs-maven-plugin/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: analysis.zip
Type: application/zip
Size: 126933 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20150319/7993f254/attachment-0001.zip 

From galder at redhat.com  Fri Mar 20 04:31:42 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 20 Mar 2015 09:31:42 +0100
Subject: [infinispan-dev]
	org.infinispan.notifications.cachelistener.filter vs
	org.infinispan.filter
In-Reply-To: <88769462-CE4A-464F-8E4B-BB6CA837C1F8@hibernate.org>
References: <A8598238-AA8E-4B97-9271-C02180AB704E@redhat.com>
	<CA+YCuUMwFe9AvBbmcO7SQY-NrPq58Pgf6ogTahkgj5=dj1h9Cw@mail.gmail.com>
	<73E0896C-7713-42EB-9E39-2C792EE2E8B4@redhat.com>
	<88769462-CE4A-464F-8E4B-BB6CA837C1F8@hibernate.org>
Message-ID: <60902DB2-DFB8-4EF4-9888-9CCA9C0F2DDD@redhat.com>

Yeah, should do: https://github.com/infinispan/infinispan/pull/3323 :)

> On 3 Mar 2015, at 16:32, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
> 
> Should it be morphed into a FAQ?
> 
>> On 03 Mar 2015, at 16:23, Galder Zamarre?o <galder at redhat.com> wrote:
>> 
>> Hmm, ok. It's true that down the line the remote events might morph into more specialised DSL-based remote event filter/conversion.
>> 
>> I just wished the naming would have been a bit more representative.
>> 
>> Ideally, the filter/converter classes used for remote events should have lived in a remote jar and be named accordingly, but they had to live in core, which has made it a little akward.
>> 
>> Some users might want to check old value even for cluster listeners, in that case it's a little akward the way it's now. I guess time will tell. I also wonder whether Java 8's default methods might enable more selective implementation, but using it to encompass both types of filters might be a misused. Just some thoughts.
>> 
>> Will, thanks for the clear response :)
>> 
>>> On 26 Feb 2015, at 15:12, William Burns <mudokonman at gmail.com> wrote:
>>> 
>>> On Thu, Feb 26, 2015 at 3:57 AM, Galder Zamarre?o <galder at redhat.com> wrote:
>>>> Hey Will,
>>>> 
>>>> I wanted to ask you about the classes in org.infinispan.filter package.
>>>> 
>>>> I thought we agreed to get rid of these duplicate classes before 7.0.0.Final:
>>>> - org.infinispan.filter.Converter
>>>> - org.infinispan.filter.KeyValueFilter
>>>> - ... and related classes
>>>> 
>>>> The reason being that we pretty much had the same classes in org.infinispan.notifications.cachelistener.filter package.
>>> 
>>> I agree they are very similar.
>>> 
>>>> 
>>>> Both set of classes were added in 7.0, so there's no reason why we should have left both sets around :|
>>> 
>>> Each one fulfills a different purpose.  Originally only the 1 set of
>>> classes was used, but event filters evolved to have new requirements
>>> (oldValue, oldMetadata, retry, event type) that don't make sense for
>>> normal filtering.  The regular KeyValueFilter, KeyFilter, Converter
>>> classes are still used when filtering existing entries (CacheLoader,
>>> EntryIterator).  Also the simpler filter & converter will be able to
>>> be used as stepping stones for Streams when we move to Java 8
>>> (although a tweak to the interface(s) will probably be required).
>>> 
>>>> 
>>>> These latter classes provide more information, but it can confuse users to decide which filters/converters to use where and how.
>>> 
>>> The CacheEventFilter/CacheEventFilterConverter are only used when
>>> using events.  The other simpler filters are used in all other cases..
>>> 
>>>> 
>>>> Am I missing something here?
>>> 
>>> The classes we talked about removing were the *Factory classes for the
>>> various filters.  We can discuss converging the ones you mentioned
>>> here, but IMO the provided functionality has diverged quite a bit,
>>> which would make having a consistent API very ugly for one or both of
>>> the use cases.
>>> 
>>>> 
>>>> We would not be able to remove anything now, but we should deprecate what should not be used.
>>>> 
>>>> From a remote even perspective, only the org.infinispan.notifications.cachelistener.filter ones are relevant.
>>> 
>>> I would say from a remote event perspective you are correct.  However
>>> once we add entry iteration for remote, we would want to use the more
>>> simplified interfaces.
>>> 
>>>> 
>>>> Cheers,
>>>> --
>>>> Galder Zamarre?o
>>>> galder at redhat.com
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From galder at redhat.com  Fri Mar 20 05:18:48 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 20 Mar 2015 10:18:48 +0100
Subject: [infinispan-dev] Cache Java 8 API proposal - First draft
In-Reply-To: <5503F3C6.9080503@redhat.com>
References: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>
	<5503F3C6.9080503@redhat.com>
Message-ID: <4B4824C2-B361-4545-A6DD-5D40852DC784@redhat.com>


> On 14 Mar 2015, at 09:39, Bela Ban <bban at redhat.com> wrote:
> 
> Hi Galder,
> 
> apologies for the long reply !

Haha, it's still sorter than my proposal, so not too bad :)

> imo the value of Java 8 is:
> 
> #1 Reduction of implementation code (streams and internal iteration)
> 
> #2 Making some of the APIs simpler, e.g. distributed executors, map-reduce
> 
> #3 Reactive programming, e.g. interceptor chain (discussed in Berlin), 
> non-blocking behavior; the goal being to reduce the number of threads 
> required.

I've heard a lot about ^, looking forward to seeing design doc.

I was discussing Java 8 API with others while you were having this discussion.

> However, changing the entire Cache API is going too far atm. Now that 
> we've finally provided a Cache/JSR107 API, people are familiar with it 
> and I would not change it, at least not right now.

You are missing essence of the proposal:

There's no unique way in which people want to access a Cache. Some people want to use ConcurrentMap, others want to use JSR-107 Cache API,  others might want to use Infinispan 7 Cache API, others just want to store data and run map/reduce jobs... 

I'm not suggesting that we change the entire Cache API. What I'm suggesting is this: Java 8 allows us the vast majority of caching use cases to be distilled into a reduced, flexible API, which can be the foundation for the facade's that we offer. 

IOW, we're not forcing anyone to use the new Cache API. If you want, you can, but the vast majority of users will use APIs they know (JSR-107, java.util.concurrent). 

It's about choice, not about having a new main API. IOW, use whatever API you want, but if youw want flexibility, we have something for you that will allow you to build more interesting data structures on top of this functional cache API.

> If we're able to use reactive programming (e.g. your eval() function) 
> *under the cover*, then all the better.
> 
> If your proposal is about providing FunctionalCache and implementing 
> Cache and JSR 107 Cache with it, then you have my +1. But IMO 
> FunctionalCache should not be the main API.

^ You're getting main point but you are wrongly believing that I want Functional Cache to be the main API.

Functional Cache will offer the basis to provide whatever API you want, but it should never be the first choice for users to use. 

IMO, the users of Functional Cache would be advanced users that say, want to build a distributed queue on top of Infinispan.

There should not be a main API going forward, but choice: JSR-107 API, java.util.concurrent...,etc

> Compare FunctionalCache.eval() to ioctl() for a moment: of course, every 
> access to a file system can be implemented via ioctl(), but people like 
> the simplicity of write(), read(), stat() etc.
> 
> So providing eval() *only* shifts the burden of implementation to the 
> user. But I agree that perhaps it's a good thing to expose 
> FunctionalCache so power users (e.g. Sanne :-)) can use it directly to 
> implement functionality not provided by Cache.

+1000, that's precisely the vision I have, but again Functional Cache is not forced on them.

> Being able to switch to Java 8 doesn't imply we have to use all of its 
> new features.

Totally.

> 
> Let me address your proposal step by step.
> 
> 
> 
> "What's wrong with the Infinispan Cache API?"
> --------------------------------------
> - Nothing !
> - I can't follow your argument as to why Cache is bad. This is the 
> premise to your proposal and it needs to be very clear !
> - JSR 107 came up with a similar API. So if we have to change Cache, 
> then I'd +1 a change to JSR 107 as main API
> - I don't see why it's difficult to add new operations: distributed 
> queues should be a separate API anyway, and not extend Cache ! We should 
> use CacheManager to get a new DistributedQueue, as we do to get a new 
> Cache...

^ Right, I think all this section needs to be rewritten to avoid confusion.

> 
> 
> "How does Java 8 improve on this?"
> ------------------------------
> - Passing serialized lambdas around sounds awfully familiar: it's the 
> old 'agent-based programming' paradigm fashionable before the turn of 
> the millenium. Instead of invoking RPCs which carry data on P, code 
> (also carrying data) is shipped to P and executed there and the result 
> sent back to the invoker. (Disclaimer: I wrote such a system myself with 
> GOMScript... :-().
> I'm not saying this is bad, but we have to make sure that
> (a) Infinispan benefits from this either in code reduction / 
> maintainability / code simplification
> (b) performance doesn't decrease

^ +1, performance at least for the most critical operations, e.g. put/get, needs to be at least on par.

> 
> 
> 
> "Proposal"
> ---------
> - Returning a CompletableFuture is a good idea: +1
> - I think this is where the main value of your proposal lies: being able 
> to chain functions (lambdas) together, that are invoked turn-by-turn, 
> possibly by different threads, without a caller having to block
> - The F.invoke(G).andThenInvoke(H).andThenInvoke(I) pattern (where F is 
> a future and G,H,I are lambdas) is very valuable to implement 
> non-blocking behavior: whenever a function (lambda) has to block, it 
> registers a function with the blocking resource that's called upon 
> unblocking and the thread terminates and is added back to the thread pool
> - Blocking can be waiting to read from the file system or network, or 
> trying to acquire a lock etc
> - As was discussed in Berlin, the interceptor chain could be rewritten 
> using CompletableFutures (or sth similar).

^ Yeah, there's a lot of value to be added in this area. 

I was very pleased to hear about the CompletableFuture based interceptor chain discussion because I thought it'd fit perfectly with the direction of the functional cache API.

> 
> 
> Summary
> --------
> 
> I'd not make FunctionalCache the main API and I'd definitely not 
> deprecate Cache.

Ok, maybe we won't deprecate Infinispan Cache but we'll definitely won't add anything extra there. It's way too ovebloated IMO.

> 
> +1 on providing FunctionalCache as a (parent) interface to Cache, so 
> power users can use FunctionalCache if needed.
> 
> +1 on implementing Cache and JSR 107 Cache with FunctionalCache.
> 
> This should only be done if:
> - Performance remains at least as good as it is now [mandatory]
> - Maintainability increases [nice]
> - Code reduction [nice]
> 
> My 5 cents

+1 to all those.

Thanks Bela for the feedback!

> 
> 
> On 13/03/15 09:10, Galder Zamarre?o wrote:
>> Hi all,
>> 
>> We're starting to work on some prototypes for the Java 8 API coming up in Infinispan 8.
>> 
>> I've written a design wiki for a replacement of our main embedded Cache interface that takes advantage of Java 8 improvements:
>> https://github.com/infinispan/infinispan/wiki/Java-8-API-proposal
>> 
>> It'd be great to hear your thoughts.
>> 
>> Proposals for Java 8 versions for cache manager, remote cache and other external APIs are yet TBD.
>> 
>> Regards,
>> --
>> Galder Zamarre?o
>> galder at redhat.com
> 
> -- 
> Bela Ban, JGroups lead (http://www.jgroups.org)
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From sebastian.laskawiec at gmail.com  Fri Mar 20 05:48:17 2015
From: sebastian.laskawiec at gmail.com (=?UTF-8?Q?Sebastian_=C5=81askawiec?=)
Date: Fri, 20 Mar 2015 09:48:17 +0000
Subject: [infinispan-dev] FindBugs on infinispan
In-Reply-To: <2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
References: <1624798272.9697831.1426781681369.JavaMail.zimbra@redhat.com>
	<2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
Message-ID: <CADP5VTWBUQaFx=QznFw66Nfa57NhR92On-ryyvSFuBHiBszeoQ@mail.gmail.com>

Hey!

I'm thinking about some other approach for static code analysis - SonarQube
[1]. Apart from FindBugs it performs also PMD and visualizes test coverage
(typically measured by Jacoco).

Sonar analysis could be performed once day in separate TeamCity job. This
way we would have an insight how the number of violations changed over the
time.

As the matter of fact I did the first step and installed Sonar on Openshift
[2]. The installation was performed using this tutorial [3].

All we need to do now is to create a TeamCity job and introduce Sonar
plugin into the code. Would you like to help me with this?

Thanks
Sebastian

[1] http://www.sonarqube.org/
[2] http://sonar-infinispansonar.rhcloud.com/
[3] https://forums.openshift.com/run-sonar-on-openshift

czw., 19 mar 2015 o 17:16 u?ytkownik Jakub Markos <jmarkos at redhat.com>
napisa?:

> Hi,
>
> out of curiosity, I ran FindBugs [1] (java static analysis tool) on
> infinispan. Attached are 2 output files (compressed, because the mailing
> list accepts only <0.5MB attachments):
> 1. for the release-jars-analysis.html, only the 3 main jars from
> 7.2.0.Alpha1 were analyzed
> 2. for the project-jars-analysis.html, I took all jars that were present
> in the infinispan project directory after a build (except the test jars) -
> this one has therefore more findings
>
> In both cases, FindBugs complained that it couldn't find some imported
> classes, so the analysis may not be 100% complete.
>
> I didn't look through the actual results much, but for example it detected
> an infinite loop at [2], or a self-assignment at [3].
>
> If you want to run it yourself, you can use [4]. There is also a GUI, but
> I wasn't able to save the results into a html, and a maven
> plugin [5], but it only creates the results in an xml format for each
> module separately, so it's not very useful.
>
> Jakub
>
> [1] http://findbugs.sourceforge.net/
> [2] https://github.com/infinispan/infinispan/blob/
> 841c789a866745b8d48475f98acd51fa74b16f13/core/src/main/java/
> org/infinispan/context/impl/ImmutableContext.java#L95
> [3] https://github.com/infinispan/infinispan/blob/
> 841c789a866745b8d48475f98acd51fa74b16f13/client/hotrod-
> client/src/main/java/org/infinispan/client/hotrod/impl/transport/tcp/
> TcpTransportFactory.java#L90
> [4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress -release
> infinispan -html -output infinispan-findbugs-analysis -onlyAnalyze
> org.infinispan.- infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar
> infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar
> infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar
> [5] http://mojo.codehaus.org/findbugs-maven-plugin/
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20150320/66ddda5c/attachment.html 

From rvansa at redhat.com  Fri Mar 20 06:54:47 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Fri, 20 Mar 2015 11:54:47 +0100
Subject: [infinispan-dev] FindBugs on infinispan
In-Reply-To: <CADP5VTWBUQaFx=QznFw66Nfa57NhR92On-ryyvSFuBHiBszeoQ@mail.gmail.com>
References: <1624798272.9697831.1426781681369.JavaMail.zimbra@redhat.com>	<2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
	<CADP5VTWBUQaFx=QznFw66Nfa57NhR92On-ryyvSFuBHiBszeoQ@mail.gmail.com>
Message-ID: <550BFC77.404@redhat.com>

Setting up analysis tool is rather easy, the hard part is to get people 
look into the results and fix what's wrong. And if noone does it, even 
setting it up is wasted time.

my 2c

Radim

On 03/20/2015 10:48 AM, Sebastian ?askawiec wrote:
> Hey!
>
> I'm thinking about some other approach for static code analysis - 
> SonarQube [1]. Apart from FindBugs it performs also PMD and visualizes 
> test coverage (typically measured by Jacoco).
>
> Sonar analysis could be performed once day in separate TeamCity job. 
> This way we would have an insight how the number of violations changed 
> over the time.
>
> As the matter of fact I did the first step and installed Sonar on 
> Openshift [2]. The installation was performed using this tutorial [3].
>
> All we need to do now is to create a TeamCity job and introduce Sonar 
> plugin into the code. Would you like to help me with this?
>
> Thanks
> Sebastian
>
> [1] http://www.sonarqube.org/
> [2] http://sonar-infinispansonar.rhcloud.com/
> [3] https://forums.openshift.com/run-sonar-on-openshift
>
> czw., 19 mar 2015 o 17:16 u?ytkownik Jakub Markos <jmarkos at redhat.com 
> <mailto:jmarkos at redhat.com>> napisa?:
>
>     Hi,
>
>     out of curiosity, I ran FindBugs [1] (java static analysis tool)
>     on infinispan. Attached are 2 output files (compressed, because
>     the mailing list accepts only <0.5MB attachments):
>     1. for the release-jars-analysis.html, only the 3 main jars from
>     7.2.0.Alpha1 were analyzed
>     2. for the project-jars-analysis.html, I took all jars that were
>     present in the infinispan project directory after a build (except
>     the test jars) - this one has therefore more findings
>
>     In both cases, FindBugs complained that it couldn't find some
>     imported classes, so the analysis may not be 100% complete.
>
>     I didn't look through the actual results much, but for example it
>     detected an infinite loop at [2], or a self-assignment at [3].
>
>     If you want to run it yourself, you can use [4]. There is also a
>     GUI, but I wasn't able to save the results into a html, and a maven
>     plugin [5], but it only creates the results in an xml format for
>     each module separately, so it's not very useful.
>
>     Jakub
>
>     [1] http://findbugs.sourceforge.net/
>     [2]
>     https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/core/src/main/java/org/infinispan/context/impl/ImmutableContext.java#L95
>     [3]
>     https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/transport/tcp/TcpTransportFactory.java#L90
>     [4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress
>     -release infinispan -html -output infinispan-findbugs-analysis
>     -onlyAnalyze org.infinispan.-
>     infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar
>     infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar
>     infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar
>     [5]
>     http://mojo.codehaus.org/findbugs-maven-plugin/_______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From pedro at infinispan.org  Fri Mar 20 07:39:23 2015
From: pedro at infinispan.org (Pedro Ruivo)
Date: Fri, 20 Mar 2015 11:39:23 +0000
Subject: [infinispan-dev] FindBugs on infinispan
In-Reply-To: <2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
References: <2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
Message-ID: <550C06EB.7090803@infinispan.org>

Hi,

There is a findbugs pluging for IntelliJ.

I run it against the classes I modified in the PR and solve the existing 
problems and avoid creating new ones.

Pedro

On 03/19/2015 04:16 PM, Jakub Markos wrote:
> Hi,
>
> out of curiosity, I ran FindBugs [1] (java static analysis tool) on infinispan. Attached are 2 output files (compressed, because the mailing list accepts only <0.5MB attachments):
> 1. for the release-jars-analysis.html, only the 3 main jars from 7.2.0.Alpha1 were analyzed
> 2. for the project-jars-analysis.html, I took all jars that were present in the infinispan project directory after a build (except the test jars) - this one has therefore more findings
>
> In both cases, FindBugs complained that it couldn't find some imported classes, so the analysis may not be 100% complete.
>
> I didn't look through the actual results much, but for example it detected an infinite loop at [2], or a self-assignment at [3].
>
> If you want to run it yourself, you can use [4]. There is also a GUI, but I wasn't able to save the results into a html, and a maven
> plugin [5], but it only creates the results in an xml format for each module separately, so it's not very useful.
>
> Jakub
>
> [1] http://findbugs.sourceforge.net/
> [2] https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/core/src/main/java/org/infinispan/context/impl/ImmutableContext.java#L95
> [3] https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/transport/tcp/TcpTransportFactory.java#L90
> [4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress -release infinispan -html -output infinispan-findbugs-analysis -onlyAnalyze org.infinispan.- infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar
> [5] http://mojo.codehaus.org/findbugs-maven-plugin/
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

From sanne at infinispan.org  Fri Mar 20 07:57:14 2015
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 20 Mar 2015 11:57:14 +0000
Subject: [infinispan-dev] FindBugs on infinispan
In-Reply-To: <550C06EB.7090803@infinispan.org>
References: <2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
	<550C06EB.7090803@infinispan.org>
Message-ID: <CAFm4XO2kz+nsxmOdZya6h8aTbwTGfXDxAW_0X0QW6UHJJMD7HQ@mail.gmail.com>

Same for Eclipse. I run it occasionally, but only the modules I'm working on.
If everyone could do the same, that helps ;-)

There is an option to get Maven to fail the build on specific
selectable categories; I've played with that in the past but I had the
impression that the annoyance level and maintenance overhead was a bit
too high compared to the benefits.

With Jenkins you can plot graphs, to make sure you have a general
trend of such problems to get lower over time.. and if I remember
correctly there also should be a way to have it reject a PR if the
threshold of code quality degradation is too high.
We do the same for CheckStyle on Hibernate ORM: not forcing an
absolute zero of violations (there's too much to fix!) but forcing a
trend of improvement.

See: http://ci.hibernate.org/view/ORM/job/hibernate-orm-4.3-h2/

-- Sanne

On 20 March 2015 at 11:39, Pedro Ruivo <pedro at infinispan.org> wrote:
> Hi,
>
> There is a findbugs pluging for IntelliJ.
>
> I run it against the classes I modified in the PR and solve the existing
> problems and avoid creating new ones.
>
> Pedro
>
> On 03/19/2015 04:16 PM, Jakub Markos wrote:
>> Hi,
>>
>> out of curiosity, I ran FindBugs [1] (java static analysis tool) on infinispan. Attached are 2 output files (compressed, because the mailing list accepts only <0.5MB attachments):
>> 1. for the release-jars-analysis.html, only the 3 main jars from 7.2.0.Alpha1 were analyzed
>> 2. for the project-jars-analysis.html, I took all jars that were present in the infinispan project directory after a build (except the test jars) - this one has therefore more findings
>>
>> In both cases, FindBugs complained that it couldn't find some imported classes, so the analysis may not be 100% complete.
>>
>> I didn't look through the actual results much, but for example it detected an infinite loop at [2], or a self-assignment at [3].
>>
>> If you want to run it yourself, you can use [4]. There is also a GUI, but I wasn't able to save the results into a html, and a maven
>> plugin [5], but it only creates the results in an xml format for each module separately, so it's not very useful.
>>
>> Jakub
>>
>> [1] http://findbugs.sourceforge.net/
>> [2] https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/core/src/main/java/org/infinispan/context/impl/ImmutableContext.java#L95
>> [3] https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/transport/tcp/TcpTransportFactory.java#L90
>> [4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress -release infinispan -html -output infinispan-findbugs-analysis -onlyAnalyze org.infinispan.- infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar
>> [5] http://mojo.codehaus.org/findbugs-maven-plugin/
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From afield at redhat.com  Fri Mar 20 09:03:13 2015
From: afield at redhat.com (Alan Field)
Date: Fri, 20 Mar 2015 09:03:13 -0400 (EDT)
Subject: [infinispan-dev] FindBugs on infinispan
In-Reply-To: <CAFm4XO2kz+nsxmOdZya6h8aTbwTGfXDxAW_0X0QW6UHJJMD7HQ@mail.gmail.com>
References: <2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
	<550C06EB.7090803@infinispan.org>
	<CAFm4XO2kz+nsxmOdZya6h8aTbwTGfXDxAW_0X0QW6UHJJMD7HQ@mail.gmail.com>
Message-ID: <492440988.569143.1426856593484.JavaMail.zimbra@redhat.com>

I really like the idea of "forcing a trend of improvement"! :-) I agree with Radim that getting the improvement in the code is the hard part. Running static analysis tools on the files you are modifying is a great way to tackle the problem in a manageable fashion. Looking at all of the output across a project is daunting! I've also always thought source code formatting arguments were a waste of time, so I have always thought that the source code should be formatted by a tool when checked into the repo. That way individuals can apply whatever formatting they prefer in their IDEs, but the source code in the repo is stored in a specific way.

Thanks,
Alan

----- Original Message -----
> From: "Sanne Grinovero" <sanne at infinispan.org>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Friday, March 20, 2015 7:57:14 AM
> Subject: Re: [infinispan-dev] FindBugs on infinispan
> 
> Same for Eclipse. I run it occasionally, but only the modules I'm working on.
> If everyone could do the same, that helps ;-)
> 
> There is an option to get Maven to fail the build on specific
> selectable categories; I've played with that in the past but I had the
> impression that the annoyance level and maintenance overhead was a bit
> too high compared to the benefits.
> 
> With Jenkins you can plot graphs, to make sure you have a general
> trend of such problems to get lower over time.. and if I remember
> correctly there also should be a way to have it reject a PR if the
> threshold of code quality degradation is too high.
> We do the same for CheckStyle on Hibernate ORM: not forcing an
> absolute zero of violations (there's too much to fix!) but forcing a
> trend of improvement.
> 
> See: http://ci.hibernate.org/view/ORM/job/hibernate-orm-4.3-h2/
> 
> -- Sanne
> 
> On 20 March 2015 at 11:39, Pedro Ruivo <pedro at infinispan.org> wrote:
> > Hi,
> >
> > There is a findbugs pluging for IntelliJ.
> >
> > I run it against the classes I modified in the PR and solve the existing
> > problems and avoid creating new ones.
> >
> > Pedro
> >
> > On 03/19/2015 04:16 PM, Jakub Markos wrote:
> >> Hi,
> >>
> >> out of curiosity, I ran FindBugs [1] (java static analysis tool) on
> >> infinispan. Attached are 2 output files (compressed, because the mailing
> >> list accepts only <0.5MB attachments):
> >> 1. for the release-jars-analysis.html, only the 3 main jars from
> >> 7.2.0.Alpha1 were analyzed
> >> 2. for the project-jars-analysis.html, I took all jars that were present
> >> in the infinispan project directory after a build (except the test jars)
> >> - this one has therefore more findings
> >>
> >> In both cases, FindBugs complained that it couldn't find some imported
> >> classes, so the analysis may not be 100% complete.
> >>
> >> I didn't look through the actual results much, but for example it detected
> >> an infinite loop at [2], or a self-assignment at [3].
> >>
> >> If you want to run it yourself, you can use [4]. There is also a GUI, but
> >> I wasn't able to save the results into a html, and a maven
> >> plugin [5], but it only creates the results in an xml format for each
> >> module separately, so it's not very useful.
> >>
> >> Jakub
> >>
> >> [1] http://findbugs.sourceforge.net/
> >> [2]
> >> https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/core/src/main/java/org/infinispan/context/impl/ImmutableContext.java#L95
> >> [3]
> >> https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/transport/tcp/TcpTransportFactory.java#L90
> >> [4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress -release
> >> infinispan -html -output infinispan-findbugs-analysis -onlyAnalyze
> >> org.infinispan.-
> >> infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar
> >> infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar
> >> infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar
> >> [5] http://mojo.codehaus.org/findbugs-maven-plugin/
> >>
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From gustavo at infinispan.org  Fri Mar 20 09:43:07 2015
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Fri, 20 Mar 2015 13:43:07 +0000
Subject: [infinispan-dev] Infinispan Directory Provider moving from
 Hibernate Search, suggestions on names
Message-ID: <CAH8Ud1Eyi==TLYLQLgF1gURH6nrJcCF51EFxv4S2sef8_a43jA@mail.gmail.com>

Hi all,

We are in process of migrating Hibernate Search's Infinispan Directory
Provider to the Infinispan codebase. As you know, the Infinispan Lucene
Directory has always been part of Infinispan, but not the layer (called
'Directory Provider') that glues it to the Hibernate Search core engine.

With this move the goal is to facilitate maintenance and avoid circular
dependencies between both projects.

So what would be the new package name and the jboss module name for this
component? I've created a preview PR [1] that is using
'org.infinispan.hibernate.search' and 'org.infinispan.directory-provider'
for package and module respectively, but would like to hear more opinions.

Thoughts?


[1] https://github.com/infinispan/infinispan/pull/3322
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20150320/f11f9eb4/attachment.html 

From ttarrant at redhat.com  Fri Mar 20 10:09:26 2015
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 20 Mar 2015 15:09:26 +0100
Subject: [infinispan-dev] Infinispan Directory Provider moving from
 Hibernate Search, suggestions on names
In-Reply-To: <CAH8Ud1Eyi==TLYLQLgF1gURH6nrJcCF51EFxv4S2sef8_a43jA@mail.gmail.com>
References: <CAH8Ud1Eyi==TLYLQLgF1gURH6nrJcCF51EFxv4S2sef8_a43jA@mail.gmail.com>
Message-ID: <550C2A16.9040808@redhat.com>

Hi Gustavo,

org.infinispan.directory-provider doesn't actually tell me much, it 
should be org.infinispan.lucene-directory-provider.

As for the package name, that's fine.

Tristan

On 20/03/2015 14:43, Gustavo Fernandes wrote:
> Hi all,
>
> We are in process of migrating Hibernate Search's Infinispan Directory 
> Provider to the Infinispan codebase. As you know, the Infinispan 
> Lucene Directory has always been part of Infinispan, but not the layer 
> (called 'Directory Provider') that glues it to the Hibernate Search 
> core engine.
>
> With this move the goal is to facilitate maintenance and avoid 
> circular dependencies between both projects.
>
> So what would be the new package name and the jboss module name for 
> this component? I've created a preview PR [1] that is using 
> 'org.infinispan.hibernate.search' and 
> 'org.infinispan.directory-provider' for package and module 
> respectively, but would like to hear more opinions.
>
> Thoughts?
>
>
> [1] https://github.com/infinispan/infinispan/pull/3322
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat


From pedro at infinispan.org  Fri Mar 20 10:36:50 2015
From: pedro at infinispan.org (Pedro Ruivo)
Date: Fri, 20 Mar 2015 14:36:50 +0000
Subject: [infinispan-dev] Infinispan Directory Provider moving from
 Hibernate Search, suggestions on names
In-Reply-To: <550C2A16.9040808@redhat.com>
References: <CAH8Ud1Eyi==TLYLQLgF1gURH6nrJcCF51EFxv4S2sef8_a43jA@mail.gmail.com>
	<550C2A16.9040808@redhat.com>
Message-ID: <550C3082.20607@infinispan.org>



On 03/20/2015 02:09 PM, Tristan Tarrant wrote:
> Hi Gustavo,
>
> org.infinispan.directory-provider doesn't actually tell me much, it
> should be org.infinispan.lucene-directory-provider.

I thought that the package name shouldn't have '-', but +1 for the 
package name

>
> As for the package name, that's fine.
>
> Tristan
>
> On 20/03/2015 14:43, Gustavo Fernandes wrote:
>> Hi all,
>>
>> We are in process of migrating Hibernate Search's Infinispan Directory
>> Provider to the Infinispan codebase. As you know, the Infinispan
>> Lucene Directory has always been part of Infinispan, but not the layer
>> (called 'Directory Provider') that glues it to the Hibernate Search
>> core engine.
>>
>> With this move the goal is to facilitate maintenance and avoid
>> circular dependencies between both projects.
>>
>> So what would be the new package name and the jboss module name for
>> this component? I've created a preview PR [1] that is using
>> 'org.infinispan.hibernate.search' and
>> 'org.infinispan.directory-provider' for package and module
>> respectively, but would like to hear more opinions.
>>
>> Thoughts?
>>
>>
>> [1] https://github.com/infinispan/infinispan/pull/3322
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>

From galder at redhat.com  Fri Mar 20 11:22:33 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 20 Mar 2015 16:22:33 +0100
Subject: [infinispan-dev] Random SocketTimeoutExceptions demonstrate we're
	not using Netty the right way
Message-ID: <FECA2706-C717-412A-BD62-F4F7C10557E2@redhat.com>

Hi all,

Summary: I've been debugging [1] and found that an unexpected operation throwing a SocketTE was being thrown due to the worker thread pool being too small, but the real problem is the fact that we can block within Netty's worker thread, something we're hoping to fix with Gustavo's work to implement [2].

The test in [1] was failing randomly as a result of this sequence of events:

1. Server configured with worker thread pool being 2 (in code, this is 2 * Runtime.getRuntime().availableProcessors())
2. We emulate a server side operation hanging with a sleep injected by an interceptor.
3. Client gets SocketTimeoutException and retries the operation.
4. The retried operation hangs because it cannot acquire the lock.
5. Client gets SocketTimeoutException again, runs out of retries.
6. Test now executes an operation that should not throw a SocketTE but it gets it because the 2 worker threads are in use. The first is in the sleep, and the retried one waiting for the lock, so it cannot make any progress and hence ends up getting a SocketTE.

You might think this can be solved by increasing the default worker thread pool size but you're just moving the problem around. You could have N operations that hang at the same time, e.g. if there's a long GC pause.

So really, we need to stop blocking within the worker thread and implement [2].

I think we might have seen this issue already back in Infinispan 5, because we were already getting SocketTEs back then in another SocketTE test [3]. I wasn't able to replicate it locally at the time and solved it by ignoring it :|

As far as ISPN-5314 is concerned, it can easily be solved by increasing the worker thread pool for that particular test, but we should revert that back when [2] is implemented.

Cheers,

[1] https://issues.jboss.org/browse/ISPN-5314
[2] https://issues.jboss.org/browse/ISPN-5083
[3] https://issues.jboss.org/browse/ISPN-2110
--
Galder Zamarre?o
galder at redhat.com






From dan.berindei at gmail.com  Fri Mar 20 12:59:56 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Fri, 20 Mar 2015 18:59:56 +0200
Subject: [infinispan-dev] FindBugs on infinispan
In-Reply-To: <492440988.569143.1426856593484.JavaMail.zimbra@redhat.com>
References: <2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
	<550C06EB.7090803@infinispan.org>
	<CAFm4XO2kz+nsxmOdZya6h8aTbwTGfXDxAW_0X0QW6UHJJMD7HQ@mail.gmail.com>
	<492440988.569143.1426856593484.JavaMail.zimbra@redhat.com>
Message-ID: <CA+nfvwR85kaeSJPYzT0e39Pntos18_c-gdp0h4WUF6ec_oP6SA@mail.gmail.com>

Keeping off the thread topic for a bit: I'm all for using a standard
formatting and enforcing that on commit, but I don't see how that
would make it easy for people to use their preferred code style in
their IDEs. Would they reformat everything so it's easier for them to
read, and never ever use git diff or annotate? Or would they use their
preferred code style just in the lines they modify, just to prove to
themselves that their style is better and ignoring the fact that the
mix of styles makes it much harder to read?

Back on topic: TC also allows you to produce custom metrics for a
build and put them in a chart, or fail the build if the value of the
metric increases too much (like we have now for the test duration). It
does sound interesting, but I'm not sure where you guys got the idea
that a failed build will prevent a PR from being integrated ;)

Cheers
Dan


On Fri, Mar 20, 2015 at 3:03 PM, Alan Field <afield at redhat.com> wrote:
> I really like the idea of "forcing a trend of improvement"! :-) I agree with Radim that getting the improvement in the code is the hard part. Running static analysis tools on the files you are modifying is a great way to tackle the problem in a manageable fashion. Looking at all of the output across a project is daunting! I've also always thought source code formatting arguments were a waste of time, so I have always thought that the source code should be formatted by a tool when checked into the repo. That way individuals can apply whatever formatting they prefer in their IDEs, but the source code in the repo is stored in a specific way.
>
> Thanks,
> Alan
>
> ----- Original Message -----
>> From: "Sanne Grinovero" <sanne at infinispan.org>
>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>> Sent: Friday, March 20, 2015 7:57:14 AM
>> Subject: Re: [infinispan-dev] FindBugs on infinispan
>>
>> Same for Eclipse. I run it occasionally, but only the modules I'm working on.
>> If everyone could do the same, that helps ;-)
>>
>> There is an option to get Maven to fail the build on specific
>> selectable categories; I've played with that in the past but I had the
>> impression that the annoyance level and maintenance overhead was a bit
>> too high compared to the benefits.
>>
>> With Jenkins you can plot graphs, to make sure you have a general
>> trend of such problems to get lower over time.. and if I remember
>> correctly there also should be a way to have it reject a PR if the
>> threshold of code quality degradation is too high.
>> We do the same for CheckStyle on Hibernate ORM: not forcing an
>> absolute zero of violations (there's too much to fix!) but forcing a
>> trend of improvement.
>>
>> See: http://ci.hibernate.org/view/ORM/job/hibernate-orm-4.3-h2/
>>
>> -- Sanne
>>
>> On 20 March 2015 at 11:39, Pedro Ruivo <pedro at infinispan.org> wrote:
>> > Hi,
>> >
>> > There is a findbugs pluging for IntelliJ.
>> >
>> > I run it against the classes I modified in the PR and solve the existing
>> > problems and avoid creating new ones.
>> >
>> > Pedro
>> >
>> > On 03/19/2015 04:16 PM, Jakub Markos wrote:
>> >> Hi,
>> >>
>> >> out of curiosity, I ran FindBugs [1] (java static analysis tool) on
>> >> infinispan. Attached are 2 output files (compressed, because the mailing
>> >> list accepts only <0.5MB attachments):
>> >> 1. for the release-jars-analysis.html, only the 3 main jars from
>> >> 7.2.0.Alpha1 were analyzed
>> >> 2. for the project-jars-analysis.html, I took all jars that were present
>> >> in the infinispan project directory after a build (except the test jars)
>> >> - this one has therefore more findings
>> >>
>> >> In both cases, FindBugs complained that it couldn't find some imported
>> >> classes, so the analysis may not be 100% complete.
>> >>
>> >> I didn't look through the actual results much, but for example it detected
>> >> an infinite loop at [2], or a self-assignment at [3].
>> >>
>> >> If you want to run it yourself, you can use [4]. There is also a GUI, but
>> >> I wasn't able to save the results into a html, and a maven
>> >> plugin [5], but it only creates the results in an xml format for each
>> >> module separately, so it's not very useful.
>> >>
>> >> Jakub
>> >>
>> >> [1] http://findbugs.sourceforge.net/
>> >> [2]
>> >> https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/core/src/main/java/org/infinispan/context/impl/ImmutableContext.java#L95
>> >> [3]
>> >> https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/transport/tcp/TcpTransportFactory.java#L90
>> >> [4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress -release
>> >> infinispan -html -output infinispan-findbugs-analysis -onlyAnalyze
>> >> org.infinispan.-
>> >> infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar
>> >> infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar
>> >> infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar
>> >> [5] http://mojo.codehaus.org/findbugs-maven-plugin/
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> infinispan-dev mailing list
>> >> infinispan-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From dan.berindei at gmail.com  Fri Mar 20 13:32:11 2015
From: dan.berindei at gmail.com (Dan Berindei)
Date: Fri, 20 Mar 2015 19:32:11 +0200
Subject: [infinispan-dev] Random SocketTimeoutExceptions demonstrate
 we're not using Netty the right way
In-Reply-To: <FECA2706-C717-412A-BD62-F4F7C10557E2@redhat.com>
References: <FECA2706-C717-412A-BD62-F4F7C10557E2@redhat.com>
Message-ID: <CA+nfvwRVY9wnca=Z4MuDEiRzAOCSr=3i5gu70ZA-URs_H72jiQ@mail.gmail.com>

Hi Galder

Using cache.putAsync on the server will use a thread from the "async"
transport executor. While the default number of threads in the
transport executor is higher (25), it's still going to be exhausted at
some point as you're generating more and more duplicate operations
from the client.

Using async operations will be a better solution once the blocked
operations don't need to keep a thread busy in core. But ATM I believe
a better solution would be to increase the socket timeout on the
client so that we get a TimeoutException from the server before we get
the SocketTimeoutException on the client.

Cheers
Dan


On Fri, Mar 20, 2015 at 5:22 PM, Galder Zamarre?o <galder at redhat.com> wrote:
> Hi all,
>
> Summary: I've been debugging [1] and found that an unexpected operation throwing a SocketTE was being thrown due to the worker thread pool being too small, but the real problem is the fact that we can block within Netty's worker thread, something we're hoping to fix with Gustavo's work to implement [2].
>
> The test in [1] was failing randomly as a result of this sequence of events:
>
> 1. Server configured with worker thread pool being 2 (in code, this is 2 * Runtime.getRuntime().availableProcessors())
> 2. We emulate a server side operation hanging with a sleep injected by an interceptor.
> 3. Client gets SocketTimeoutException and retries the operation.
> 4. The retried operation hangs because it cannot acquire the lock.
> 5. Client gets SocketTimeoutException again, runs out of retries.
> 6. Test now executes an operation that should not throw a SocketTE but it gets it because the 2 worker threads are in use. The first is in the sleep, and the retried one waiting for the lock, so it cannot make any progress and hence ends up getting a SocketTE.
>
> You might think this can be solved by increasing the default worker thread pool size but you're just moving the problem around. You could have N operations that hang at the same time, e.g. if there's a long GC pause.
>
> So really, we need to stop blocking within the worker thread and implement [2].
>
> I think we might have seen this issue already back in Infinispan 5, because we were already getting SocketTEs back then in another SocketTE test [3]. I wasn't able to replicate it locally at the time and solved it by ignoring it :|
>
> As far as ISPN-5314 is concerned, it can easily be solved by increasing the worker thread pool for that particular test, but we should revert that back when [2] is implemented.
>
> Cheers,
>
> [1] https://issues.jboss.org/browse/ISPN-5314
> [2] https://issues.jboss.org/browse/ISPN-5083
> [3] https://issues.jboss.org/browse/ISPN-2110
> --
> Galder Zamarre?o
> galder at redhat.com
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From afield at redhat.com  Fri Mar 20 13:46:31 2015
From: afield at redhat.com (Alan Field)
Date: Fri, 20 Mar 2015 13:46:31 -0400 (EDT)
Subject: [infinispan-dev] FindBugs on infinispan
In-Reply-To: <CA+nfvwR85kaeSJPYzT0e39Pntos18_c-gdp0h4WUF6ec_oP6SA@mail.gmail.com>
References: <2711197.9698303.1426781772838.JavaMail.zimbra@redhat.com>
	<550C06EB.7090803@infinispan.org>
	<CAFm4XO2kz+nsxmOdZya6h8aTbwTGfXDxAW_0X0QW6UHJJMD7HQ@mail.gmail.com>
	<492440988.569143.1426856593484.JavaMail.zimbra@redhat.com>
	<CA+nfvwR85kaeSJPYzT0e39Pntos18_c-gdp0h4WUF6ec_oP6SA@mail.gmail.com>
Message-ID: <1612972660.752109.1426873591679.JavaMail.zimbra@redhat.com>

Hey Dan,

> Keeping off the thread topic for a bit: I'm all for using a standard
> formatting and enforcing that on commit, but I don't see how that
> would make it easy for people to use their preferred code style in
> their IDEs. Would they reformat everything so it's easier for them to
> read, and never ever use git diff or annotate? Or would they use their
> preferred code style just in the lines they modify, just to prove to
> themselves that their style is better and ignoring the fact that the
> mix of styles makes it much harder to read?

On your off topic thread question: My thought was that you could reformat the code to whatever style you like in your IDE while you are working on it locally, but you make a valid point about the fact that doing comparisons to upstream will be difficult. I guess you would still need a way to reset the format to upstream in this case which means this doesn't solve much. I guess the current strategy of trying to keep the IDE formatting preferences in-sync is still the best right now.

Thanks,
Alan

> Back on topic: TC also allows you to produce custom metrics for a
> build and put them in a chart, or fail the build if the value of the
> metric increases too much (like we have now for the test duration). It
> does sound interesting, but I'm not sure where you guys got the idea
> that a failed build will prevent a PR from being integrated ;)
> 
> Cheers
> Dan
> 
> 
> On Fri, Mar 20, 2015 at 3:03 PM, Alan Field <afield at redhat.com> wrote:
> > I really like the idea of "forcing a trend of improvement"! :-) I agree
> > with Radim that getting the improvement in the code is the hard part.
> > Running static analysis tools on the files you are modifying is a great
> > way to tackle the problem in a manageable fashion. Looking at all of the
> > output across a project is daunting! I've also always thought source code
> > formatting arguments were a waste of time, so I have always thought that
> > the source code should be formatted by a tool when checked into the repo.
> > That way individuals can apply whatever formatting they prefer in their
> > IDEs, but the source code in the repo is stored in a specific way.
> >
> > Thanks,
> > Alan
> >
> > ----- Original Message -----
> >> From: "Sanne Grinovero" <sanne at infinispan.org>
> >> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> >> Sent: Friday, March 20, 2015 7:57:14 AM
> >> Subject: Re: [infinispan-dev] FindBugs on infinispan
> >>
> >> Same for Eclipse. I run it occasionally, but only the modules I'm working
> >> on.
> >> If everyone could do the same, that helps ;-)
> >>
> >> There is an option to get Maven to fail the build on specific
> >> selectable categories; I've played with that in the past but I had the
> >> impression that the annoyance level and maintenance overhead was a bit
> >> too high compared to the benefits.
> >>
> >> With Jenkins you can plot graphs, to make sure you have a general
> >> trend of such problems to get lower over time.. and if I remember
> >> correctly there also should be a way to have it reject a PR if the
> >> threshold of code quality degradation is too high.
> >> We do the same for CheckStyle on Hibernate ORM: not forcing an
> >> absolute zero of violations (there's too much to fix!) but forcing a
> >> trend of improvement.
> >>
> >> See: http://ci.hibernate.org/view/ORM/job/hibernate-orm-4.3-h2/
> >>
> >> -- Sanne
> >>
> >> On 20 March 2015 at 11:39, Pedro Ruivo <pedro at infinispan.org> wrote:
> >> > Hi,
> >> >
> >> > There is a findbugs pluging for IntelliJ.
> >> >
> >> > I run it against the classes I modified in the PR and solve the existing
> >> > problems and avoid creating new ones.
> >> >
> >> > Pedro
> >> >
> >> > On 03/19/2015 04:16 PM, Jakub Markos wrote:
> >> >> Hi,
> >> >>
> >> >> out of curiosity, I ran FindBugs [1] (java static analysis tool) on
> >> >> infinispan. Attached are 2 output files (compressed, because the
> >> >> mailing
> >> >> list accepts only <0.5MB attachments):
> >> >> 1. for the release-jars-analysis.html, only the 3 main jars from
> >> >> 7.2.0.Alpha1 were analyzed
> >> >> 2. for the project-jars-analysis.html, I took all jars that were
> >> >> present
> >> >> in the infinispan project directory after a build (except the test
> >> >> jars)
> >> >> - this one has therefore more findings
> >> >>
> >> >> In both cases, FindBugs complained that it couldn't find some imported
> >> >> classes, so the analysis may not be 100% complete.
> >> >>
> >> >> I didn't look through the actual results much, but for example it
> >> >> detected
> >> >> an infinite loop at [2], or a self-assignment at [3].
> >> >>
> >> >> If you want to run it yourself, you can use [4]. There is also a GUI,
> >> >> but
> >> >> I wasn't able to save the results into a html, and a maven
> >> >> plugin [5], but it only creates the results in an xml format for each
> >> >> module separately, so it's not very useful.
> >> >>
> >> >> Jakub
> >> >>
> >> >> [1] http://findbugs.sourceforge.net/
> >> >> [2]
> >> >> https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/core/src/main/java/org/infinispan/context/impl/ImmutableContext.java#L95
> >> >> [3]
> >> >> https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98acd51fa74b16f13/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/transport/tcp/TcpTransportFactory.java#L90
> >> >> [4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress -release
> >> >> infinispan -html -output infinispan-findbugs-analysis -onlyAnalyze
> >> >> org.infinispan.-
> >> >> infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar
> >> >> infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar
> >> >> infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar
> >> >> [5] http://mojo.codehaus.org/findbugs-maven-plugin/
> >> >>
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> infinispan-dev mailing list
> >> >> infinispan-dev at lists.jboss.org
> >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >> >>
> >> > _______________________________________________
> >> > infinispan-dev mailing list
> >> > infinispan-dev at lists.jboss.org
> >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From bban at redhat.com  Mon Mar 23 04:29:56 2015
From: bban at redhat.com (Bela Ban)
Date: Mon, 23 Mar 2015 09:29:56 +0100
Subject: [infinispan-dev] Cache Java 8 API proposal - First draft
In-Reply-To: <4B4824C2-B361-4545-A6DD-5D40852DC784@redhat.com>
References: <8C3D5176-E2CA-4089-9AF7-B829D0119244@redhat.com>	<5503F3C6.9080503@redhat.com>
	<4B4824C2-B361-4545-A6DD-5D40852DC784@redhat.com>
Message-ID: <550FCF04.1020002@redhat.com>

OK, then we're on the same page, just wanted to verify. Re the design 
doc wrt interceptors, contact Radim
Cheers

On 20/03/15 10:18, Galder Zamarre?o wrote:
>
>> On 14 Mar 2015, at 09:39, Bela Ban <bban at redhat.com> wrote:
>>
>> Hi Galder,
>>
>> apologies for the long reply !
>
> Haha, it's still sorter than my proposal, so not too bad :)
>
>> imo the value of Java 8 is:
>>
>> #1 Reduction of implementation code (streams and internal iteration)
>>
>> #2 Making some of the APIs simpler, e.g. distributed executors, map-reduce
>>
>> #3 Reactive programming, e.g. interceptor chain (discussed in Berlin),
>> non-blocking behavior; the goal being to reduce the number of threads
>> required.
>
> I've heard a lot about ^, looking forward to seeing design doc.
>
> I was discussing Java 8 API with others while you were having this discussion.
>
>> However, changing the entire Cache API is going too far atm. Now that
>> we've finally provided a Cache/JSR107 API, people are familiar with it
>> and I would not change it, at least not right now.
>
> You are missing essence of the proposal:
>
> There's no unique way in which people want to access a Cache. Some people want to use ConcurrentMap, others want to use JSR-107 Cache API,  others might want to use Infinispan 7 Cache API, others just want to store data and run map/reduce jobs...
>
> I'm not suggesting that we change the entire Cache API. What I'm suggesting is this: Java 8 allows us the vast majority of caching use cases to be distilled into a reduced, flexible API, which can be the foundation for the facade's that we offer.
>
> IOW, we're not forcing anyone to use the new Cache API. If you want, you can, but the vast majority of users will use APIs they know (JSR-107, java.util.concurrent).
>
> It's about choice, not about having a new main API. IOW, use whatever API you want, but if youw want flexibility, we have something for you that will allow you to build more interesting data structures on top of this functional cache API.
>
>> If we're able to use reactive programming (e.g. your eval() function)
>> *under the cover*, then all the better.
>>
>> If your proposal is about providing FunctionalCache and implementing
>> Cache and JSR 107 Cache with it, then you have my +1. But IMO
>> FunctionalCache should not be the main API.
>
> ^ You're getting main point but you are wrongly believing that I want Functional Cache to be the main API.
>
> Functional Cache will offer the basis to provide whatever API you want, but it should never be the first choice for users to use.
>
> IMO, the users of Functional Cache would be advanced users that say, want to build a distributed queue on top of Infinispan.
>
> There should not be a main API going forward, but choice: JSR-107 API, java.util.concurrent...,etc
>
>> Compare FunctionalCache.eval() to ioctl() for a moment: of course, every
>> access to a file system can be implemented via ioctl(), but people like
>> the simplicity of write(), read(), stat() etc.
>>
>> So providing eval() *only* shifts the burden of implementation to the
>> user. But I agree that perhaps it's a good thing to expose
>> FunctionalCache so power users (e.g. Sanne :-)) can use it directly to
>> implement functionality not provided by Cache.
>
> +1000, that's precisely the vision I have, but again Functional Cache is not forced on them.
>
>> Being able to switch to Java 8 doesn't imply we have to use all of its
>> new features.
>
> Totally.
>
>>
>> Let me address your proposal step by step.
>>
>>
>>
>> "What's wrong with the Infinispan Cache API?"
>> --------------------------------------
>> - Nothing !
>> - I can't follow your argument as to why Cache is bad. This is the
>> premise to your proposal and it needs to be very clear !
>> - JSR 107 came up with a similar API. So if we have to change Cache,
>> then I'd +1 a change to JSR 107 as main API
>> - I don't see why it's difficult to add new operations: distributed
>> queues should be a separate API anyway, and not extend Cache ! We should
>> use CacheManager to get a new DistributedQueue, as we do to get a new
>> Cache...
>
> ^ Right, I think all this section needs to be rewritten to avoid confusion.
>
>>
>>
>> "How does Java 8 improve on this?"
>> ------------------------------
>> - Passing serialized lambdas around sounds awfully familiar: it's the
>> old 'agent-based programming' paradigm fashionable before the turn of
>> the millenium. Instead of invoking RPCs which carry data on P, code
>> (also carrying data) is shipped to P and executed there and the result
>> sent back to the invoker. (Disclaimer: I wrote such a system myself with
>> GOMScript... :-().
>> I'm not saying this is bad, but we have to make sure that
>> (a) Infinispan benefits from this either in code reduction /
>> maintainability / code simplification
>> (b) performance doesn't decrease
>
> ^ +1, performance at least for the most critical operations, e.g. put/get, needs to be at least on par.
>
>>
>>
>>
>> "Proposal"
>> ---------
>> - Returning a CompletableFuture is a good idea: +1
>> - I think this is where the main value of your proposal lies: being able
>> to chain functions (lambdas) together, that are invoked turn-by-turn,
>> possibly by different threads, without a caller having to block
>> - The F.invoke(G).andThenInvoke(H).andThenInvoke(I) pattern (where F is
>> a future and G,H,I are lambdas) is very valuable to implement
>> non-blocking behavior: whenever a function (lambda) has to block, it
>> registers a function with the blocking resource that's called upon
>> unblocking and the thread terminates and is added back to the thread pool
>> - Blocking can be waiting to read from the file system or network, or
>> trying to acquire a lock etc
>> - As was discussed in Berlin, the interceptor chain could be rewritten
>> using CompletableFutures (or sth similar).
>
> ^ Yeah, there's a lot of value to be added in this area.
>
> I was very pleased to hear about the CompletableFuture based interceptor chain discussion because I thought it'd fit perfectly with the direction of the functional cache API.
>
>>
>>
>> Summary
>> --------
>>
>> I'd not make FunctionalCache the main API and I'd definitely not
>> deprecate Cache.
>
> Ok, maybe we won't deprecate Infinispan Cache but we'll definitely won't add anything extra there. It's way too ovebloated IMO.
>
>>
>> +1 on providing FunctionalCache as a (parent) interface to Cache, so
>> power users can use FunctionalCache if needed.
>>
>> +1 on implementing Cache and JSR 107 Cache with FunctionalCache.
>>
>> This should only be done if:
>> - Performance remains at least as good as it is now [mandatory]
>> - Maintainability increases [nice]
>> - Code reduction [nice]
>>
>> My 5 cents
>
> +1 to all those.
>
> Thanks Bela for the feedback!
>
>>
>>
>> On 13/03/15 09:10, Galder Zamarre?o wrote:
>>> Hi all,
>>>
>>> We're starting to work on some prototypes for the Java 8 API coming up in Infinispan 8.
>>>
>>> I've written a design wiki for a replacement of our main embedded Cache interface that takes advantage of Java 8 improvements:
>>> https://github.com/infinispan/infinispan/wiki/Java-8-API-proposal
>>>
>>> It'd be great to hear your thoughts.
>>>
>>> Proposals for Java 8 versions for cache manager, remote cache and other external APIs are yet TBD.
>>>
>>> Regards,
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>
>> --
>> Bela Ban, JGroups lead (http://www.jgroups.org)
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Galder Zamarre?o
> galder at redhat.com
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)

From gustavo at infinispan.org  Mon Mar 23 05:21:41 2015
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Mon, 23 Mar 2015 09:21:41 +0000
Subject: [infinispan-dev] Infinispan Directory Provider moving from
 Hibernate Search, suggestions on names
In-Reply-To: <550C2A16.9040808@redhat.com>
References: <CAH8Ud1Eyi==TLYLQLgF1gURH6nrJcCF51EFxv4S2sef8_a43jA@mail.gmail.com>
	<550C2A16.9040808@redhat.com>
Message-ID: <CAH8Ud1Hndefw0MtqH1PCj8ESJ0rw790+nfMZdMM4fs3K3-xg+g@mail.gmail.com>

On Fri, Mar 20, 2015 at 2:09 PM, Tristan Tarrant <ttarrant at redhat.com>
wrote:

> Hi Gustavo,
>
> org.infinispan.directory-provider doesn't actually tell me much, it
> should be org.infinispan.lucene-directory-provider.
>

org.infinispan.lucene-directory-provider gives the impression it is about
the Lucene Directory packaging, without the hibernate parts.

What about org.infinispan.hibernate-search.directory-provider?



>
> As for the package name, that's fine.
>
> Tristan
>
> On 20/03/2015 14:43, Gustavo Fernandes wrote:
> > Hi all,
> >
> > We are in process of migrating Hibernate Search's Infinispan Directory
> > Provider to the Infinispan codebase. As you know, the Infinispan
> > Lucene Directory has always been part of Infinispan, but not the layer
> > (called 'Directory Provider') that glues it to the Hibernate Search
> > core engine.
> >
> > With this move the goal is to facilitate maintenance and avoid
> > circular dependencies between both projects.
> >
> > So what would be the new package name and the jboss module name for
> > this component? I've created a preview PR [1] that is using
> > 'org.infinispan.hibernate.search' and
> > 'org.infinispan.directory-provider' for package and module
> > respectively, but would like to hear more opinions.
> >
> > Thoughts?
> >
> >
> > [1] https://github.com/infinispan/infinispan/pull/3322
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20150323/a3c2622c/attachment.html 

From ttarrant at redhat.com  Mon Mar 23 06:21:18 2015
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 23 Mar 2015 11:21:18 +0100
Subject: [infinispan-dev] Infinispan Directory Provider moving from
 Hibernate Search, suggestions on names
In-Reply-To: <CAH8Ud1Hndefw0MtqH1PCj8ESJ0rw790+nfMZdMM4fs3K3-xg+g@mail.gmail.com>
References: <CAH8Ud1Eyi==TLYLQLgF1gURH6nrJcCF51EFxv4S2sef8_a43jA@mail.gmail.com>	<550C2A16.9040808@redhat.com>
	<CAH8Ud1Hndefw0MtqH1PCj8ESJ0rw790+nfMZdMM4fs3K3-xg+g@mail.gmail.com>
Message-ID: <550FE91E.5050903@redhat.com>

On 23/03/2015 10:21, Gustavo Fernandes wrote:
> On Fri, Mar 20, 2015 at 2:09 PM, Tristan Tarrant <ttarrant at redhat.com 
> <mailto:ttarrant at redhat.com>> wrote:
>
>     Hi Gustavo,
>
>     org.infinispan.directory-provider doesn't actually tell me much, it
>     should be org.infinispan.lucene-directory-provider.
>
>
> org.infinispan.lucene-directory-provider gives the impression it is 
> about the Lucene Directory packaging, without the hibernate parts.
>
> What about org.infinispan.hibernate-search.directory-provider?
+1

Tristan

From galder at redhat.com  Tue Mar 24 05:52:02 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 24 Mar 2015 10:52:02 +0100
Subject: [infinispan-dev] Dealing with ownerless segments
Message-ID: <4B1924AD-3D15-4D52-A04E-76E50611BDB1@redhat.com>

Hey Dan,

Re: https://issues.jboss.org/browse/ISPN-5208

I've created a unit test based on Matej's feedback to replicate the issue [1].

Debugging through it, I've discovered that some topology updates are sent with segments that have no owner, hence when the key is hashed to that segment, it finds no owner for it.

Looking at the logs, I see:

2015-03-24 10:29:57,855 4405  TRACE [org.infinispan.topology.CacheTopology] (remote-thread-main-NodeC-p15-t4:) Current consistent hash's routing table: 0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 0, 13: 0, 14: 0, 15: 0, 16: 0, 17: 0, 18: 0, 19: 0, 20: 0, 21: 0, 22: 0, 23: 0, 24: 0, 25: 0, 26: 0, 27: 0, 28: 0, 29: 0, 30: 1, 31: 1, 32: 1, 33: 1, 34: 1, 35: 1, 36: 1, 37: 1, 38: 1, 39: 1, 40: 1, 41: 1, 42: 1, 43: 1, 44: 1, 45: 1, 46: 1, 47: 1, 48: 1, 49: 1, 50: 1, 51: 1, 52: 1, 53: 1, 54: 1, 55: 1, 56: 1, 57: 1, 58: 1, 59: 1

If having a segment that has no owners is valid scenario, what should we do about it? 
- Should the server avoid sending such topology unless all segments have owners? 
- Or should the client use round-robin as fallback when segment for key has no owner? 
- Or is this a bug?

Cheers,

[1] https://gist.github.com/galderz/17f1c4f21b4e7856aa6b
--
Galder Zamarre?o
galder at redhat.com






From mudokonman at gmail.com  Tue Mar 24 09:22:09 2015
From: mudokonman at gmail.com (William Burns)
Date: Tue, 24 Mar 2015 09:22:09 -0400
Subject: [infinispan-dev] Multi get API
Message-ID: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>

I am nearing completion of the new multi get command [1], allowing for
more efficient retrieval of multiple keys at the same time.  Thanks
again to Radim for a lot of the initial work.

In doing so though I want to make sure I get feedback on how we want
the API to actually look, since this is much more difficult to change
down the line.

There are a few things in particular I wanted to discuss.

1. The actual name of the method.  The main suggestions I have seen
are getAll or getMany.  I do like the naming of the former, however it
seems a bit misleading (remember API is getAll(Set) since we are
really getting a subset.  So I am thinking the possibilities at this
point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
the last one (getMany).  I am open to any suggestions though.

2. What should the return value be for this method.  Currently this
returns a Map<K, V> which makes sense when we retain these values in
the local transactional context and is pretty nice and clean for end
users.

The other idea is to use a streaming API possibly returning an
Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
as intuitive to me, but opens us up for more efficient usage
especially when there may be a lot of keys passed in (also results can
be processed concurrently while retrieval is still occurring).

I would lean towards returning the Map<K, V> value, however the next
point could change that.

3. Where this method should live.  Right now this method is on the
BasicCache interface which is a parent of both Cache (embedded) and
RemoteCache (remote).  Unfortunately for remote clients to be as
efficient as possible we really need a Streaming API so that we don't
have multiple copies of data in memory at multiple places at the same
time.  For that reason I suggest we use the streaminig API for both or
have a different API for remote vs embedded.

Let me know what you guys think.

Cheers,

 - Will

[1] https://issues.jboss.org/browse/ISPN-2183

From galder at redhat.com  Tue Mar 24 09:53:14 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 24 Mar 2015 14:53:14 +0100
Subject: [infinispan-dev] Dealing with ownerless segments
In-Reply-To: <4B1924AD-3D15-4D52-A04E-76E50611BDB1@redhat.com>
References: <4B1924AD-3D15-4D52-A04E-76E50611BDB1@redhat.com>
Message-ID: <928EB510-BA4D-40D0-8733-085EEAE09C8B@redhat.com>

False alarm, there are always segment owners. It's an issue with start procedure where the cache manager has members available for segment information before Hot Rod server has added itself to the topology cache. Thx to Will for spotting it :)

> On 24 Mar 2015, at 10:52, Galder Zamarre?o <galder at redhat.com> wrote:
> 
> Hey Dan,
> 
> Re: https://issues.jboss.org/browse/ISPN-5208
> 
> I've created a unit test based on Matej's feedback to replicate the issue [1].
> 
> Debugging through it, I've discovered that some topology updates are sent with segments that have no owner, hence when the key is hashed to that segment, it finds no owner for it.
> 
> Looking at the logs, I see:
> 
> 2015-03-24 10:29:57,855 4405  TRACE [org.infinispan.topology.CacheTopology] (remote-thread-main-NodeC-p15-t4:) Current consistent hash's routing table: 0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 0, 13: 0, 14: 0, 15: 0, 16: 0, 17: 0, 18: 0, 19: 0, 20: 0, 21: 0, 22: 0, 23: 0, 24: 0, 25: 0, 26: 0, 27: 0, 28: 0, 29: 0, 30: 1, 31: 1, 32: 1, 33: 1, 34: 1, 35: 1, 36: 1, 37: 1, 38: 1, 39: 1, 40: 1, 41: 1, 42: 1, 43: 1, 44: 1, 45: 1, 46: 1, 47: 1, 48: 1, 49: 1, 50: 1, 51: 1, 52: 1, 53: 1, 54: 1, 55: 1, 56: 1, 57: 1, 58: 1, 59: 1
> 
> If having a segment that has no owners is valid scenario, what should we do about it? 
> - Should the server avoid sending such topology unless all segments have owners? 
> - Or should the client use round-robin as fallback when segment for key has no owner? 
> - Or is this a bug?
> 
> Cheers,
> 
> [1] https://gist.github.com/galderz/17f1c4f21b4e7856aa6b
> --
> Galder Zamarre?o
> galder at redhat.com
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From rvansa at redhat.com  Tue Mar 24 09:57:03 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 24 Mar 2015 14:57:03 +0100
Subject: [infinispan-dev] Multi get API
In-Reply-To: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
Message-ID: <55116D2F.3090707@redhat.com>

On 03/24/2015 02:22 PM, William Burns wrote:
> I am nearing completion of the new multi get command [1], allowing for
> more efficient retrieval of multiple keys at the same time.  Thanks
> again to Radim for a lot of the initial work.
>
> In doing so though I want to make sure I get feedback on how we want
> the API to actually look, since this is much more difficult to change
> down the line.
>
> There are a few things in particular I wanted to discuss.
>
> 1. The actual name of the method.  The main suggestions I have seen
> are getAll or getMany.  I do like the naming of the former, however it
> seems a bit misleading (remember API is getAll(Set) since we are
> really getting a subset.  So I am thinking the possibilities at this
> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
> the last one (getMany).  I am open to any suggestions though.

Actually Tristan suggested the name multiGet() - I would prefer this one 
in the end, and adding multiPut() that would just do putAll (to have the 
API symmetric) and deprecate the putAll() method. multiRemove and others 
can follow later, and the naming is straightforward.
I would object against getAll() since this sounds like retrieving all 
entries from the cache, not just the specified keys.

>
> 2. What should the return value be for this method.  Currently this
> returns a Map<K, V> which makes sense when we retain these values in
> the local transactional context and is pretty nice and clean for end
> users.
>
> The other idea is to use a streaming API possibly returning an
> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
> as intuitive to me, but opens us up for more efficient usage
> especially when there may be a lot of keys passed in (also results can
> be processed concurrently while retrieval is still occurring).
>
> I would lean towards returning the Map<K, V> value, however the next
> point could change that.

I think that Iterable<CacheEntry<K, V>> is too confusing for the end 
user, I would stick to the Map. If you want lazy loading, the Map (and 
it's EntrySet) could be made lazy by a flag.

>
> 3. Where this method should live.  Right now this method is on the
> BasicCache interface which is a parent of both Cache (embedded) and
> RemoteCache (remote).  Unfortunately for remote clients to be as
> efficient as possible we really need a Streaming API so that we don't
> have multiple copies of data in memory at multiple places at the same
> time.  For that reason I suggest we use the streaminig API for both or
> have a different API for remote vs embedded.
>

Radim

-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From pedro at infinispan.org  Tue Mar 24 10:38:15 2015
From: pedro at infinispan.org (Pedro Ruivo)
Date: Tue, 24 Mar 2015 14:38:15 +0000
Subject: [infinispan-dev] Multi get API
In-Reply-To: <55116D2F.3090707@redhat.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<55116D2F.3090707@redhat.com>
Message-ID: <551176D7.5060306@infinispan.org>

Hi,

comments inline...

On 03/24/2015 01:57 PM, Radim Vansa wrote:
> On 03/24/2015 02:22 PM, William Burns wrote:
>> I am nearing completion of the new multi get command [1], allowing for
>> more efficient retrieval of multiple keys at the same time.  Thanks
>> again to Radim for a lot of the initial work.
>>
>> In doing so though I want to make sure I get feedback on how we want
>> the API to actually look, since this is much more difficult to change
>> down the line.
>>
>> There are a few things in particular I wanted to discuss.
>>
>> 1. The actual name of the method.  The main suggestions I have seen
>> are getAll or getMany.  I do like the naming of the former, however it
>> seems a bit misleading (remember API is getAll(Set) since we are
>> really getting a subset.  So I am thinking the possibilities at this
>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>> the last one (getMany).  I am open to any suggestions though.
>
> Actually Tristan suggested the name multiGet() - I would prefer this one
> in the end, and adding multiPut() that would just do putAll (to have the
> API symmetric) and deprecate the putAll() method. multiRemove and others
> can follow later, and the naming is straightforward.
> I would object against getAll() since this sounds like retrieving all
> entries from the cache, not just the specified keys.
>

Why a Set as parameter?

If we have a Collection as parameter, it would allow a user to specify 
easily the keys he want, like multiGet(Arrays.asList(k1, k2, k3)).

>>
>> 2. What should the return value be for this method.  Currently this
>> returns a Map<K, V> which makes sense when we retain these values in
>> the local transactional context and is pretty nice and clean for end
>> users.
>>
>> The other idea is to use a streaming API possibly returning an
>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>> as intuitive to me, but opens us up for more efficient usage
>> especially when there may be a lot of keys passed in (also results can
>> be processed concurrently while retrieval is still occurring).
>>
>> I would lean towards returning the Map<K, V> value, however the next
>> point could change that.
>
> I think that Iterable<CacheEntry<K, V>> is too confusing for the end
> user, I would stick to the Map. If you want lazy loading, the Map (and
> it's EntrySet) could be made lazy by a flag.

my vote is to use the Map<K,V> interface. The Iterable will force the 
user to fetch the data using the iterator order, while the Map allow him 
to use any order he wants (i.e. fetch particular keys via map.get(()).

BTW, it can be good to wrap the Map in a unmodifiableMap().

>
>>
>> 3. Where this method should live.  Right now this method is on the
>> BasicCache interface which is a parent of both Cache (embedded) and
>> RemoteCache (remote).  Unfortunately for remote clients to be as
>> efficient as possible we really need a Streaming API so that we don't
>> have multiple copies of data in memory at multiple places at the same
>> time.  For that reason I suggest we use the streaminig API for both or
>> have a different API for remote vs embedded.
>>

IMO, we should have different API for embedded and remote.

BasicCache does not seems a good place to put it. It can be placed in 
AdvancedCache.

Cheers,
Pedro

>
> Radim
>

From rvansa at redhat.com  Tue Mar 24 11:06:32 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 24 Mar 2015 16:06:32 +0100
Subject: [infinispan-dev] Multi get API
In-Reply-To: <551176D7.5060306@infinispan.org>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>	<55116D2F.3090707@redhat.com>
	<551176D7.5060306@infinispan.org>
Message-ID: <55117D78.7010101@redhat.com>

On 03/24/2015 03:38 PM, Pedro Ruivo wrote:
> Hi,
>
> comments inline...
>
> On 03/24/2015 01:57 PM, Radim Vansa wrote:
>> On 03/24/2015 02:22 PM, William Burns wrote:
>>> I am nearing completion of the new multi get command [1], allowing for
>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>> again to Radim for a lot of the initial work.
>>>
>>> In doing so though I want to make sure I get feedback on how we want
>>> the API to actually look, since this is much more difficult to change
>>> down the line.
>>>
>>> There are a few things in particular I wanted to discuss.
>>>
>>> 1. The actual name of the method.  The main suggestions I have seen
>>> are getAll or getMany.  I do like the naming of the former, however it
>>> seems a bit misleading (remember API is getAll(Set) since we are
>>> really getting a subset.  So I am thinking the possibilities at this
>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>> the last one (getMany).  I am open to any suggestions though.
>> Actually Tristan suggested the name multiGet() - I would prefer this one
>> in the end, and adding multiPut() that would just do putAll (to have the
>> API symmetric) and deprecate the putAll() method. multiRemove and others
>> can follow later, and the naming is straightforward.
>> I would object against getAll() since this sounds like retrieving all
>> entries from the cache, not just the specified keys.
>>
> Why a Set as parameter?
>
> If we have a Collection as parameter, it would allow a user to specify
> easily the keys he want, like multiGet(Arrays.asList(k1, k2, k3)).

It's Set as the collection should not contain duplicities. However, it's 
a bit questionable whether the collection should be copied for (async) 
internal processing or whether Infinispan takes ownership of this map 
(first option is more safe, second is faster).

Radim

>
>>> 2. What should the return value be for this method.  Currently this
>>> returns a Map<K, V> which makes sense when we retain these values in
>>> the local transactional context and is pretty nice and clean for end
>>> users.
>>>
>>> The other idea is to use a streaming API possibly returning an
>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>> as intuitive to me, but opens us up for more efficient usage
>>> especially when there may be a lot of keys passed in (also results can
>>> be processed concurrently while retrieval is still occurring).
>>>
>>> I would lean towards returning the Map<K, V> value, however the next
>>> point could change that.
>> I think that Iterable<CacheEntry<K, V>> is too confusing for the end
>> user, I would stick to the Map. If you want lazy loading, the Map (and
>> it's EntrySet) could be made lazy by a flag.
> my vote is to use the Map<K,V> interface. The Iterable will force the
> user to fetch the data using the iterator order, while the Map allow him
> to use any order he wants (i.e. fetch particular keys via map.get(()).
>
> BTW, it can be good to wrap the Map in a unmodifiableMap().
>
>>> 3. Where this method should live.  Right now this method is on the
>>> BasicCache interface which is a parent of both Cache (embedded) and
>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>> efficient as possible we really need a Streaming API so that we don't
>>> have multiple copies of data in memory at multiple places at the same
>>> time.  For that reason I suggest we use the streaminig API for both or
>>> have a different API for remote vs embedded.
>>>
> IMO, we should have different API for embedded and remote.
>
> BasicCache does not seems a good place to put it. It can be placed in
> AdvancedCache.
>
> Cheers,
> Pedro
>
>> Radim
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From mudokonman at gmail.com  Tue Mar 24 11:22:52 2015
From: mudokonman at gmail.com (William Burns)
Date: Tue, 24 Mar 2015 11:22:52 -0400
Subject: [infinispan-dev] Multi get API
In-Reply-To: <55117D78.7010101@redhat.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<55116D2F.3090707@redhat.com> <551176D7.5060306@infinispan.org>
	<55117D78.7010101@redhat.com>
Message-ID: <CA+YCuUOt3WPTQ6p_wJvHhSU_YwSMXjOR_GfD6YidbEy1dOQZXA@mail.gmail.com>

On Tue, Mar 24, 2015 at 11:06 AM, Radim Vansa <rvansa at redhat.com> wrote:
> On 03/24/2015 03:38 PM, Pedro Ruivo wrote:
>> Hi,
>>
>> comments inline...
>>
>> On 03/24/2015 01:57 PM, Radim Vansa wrote:
>>> On 03/24/2015 02:22 PM, William Burns wrote:
>>>> I am nearing completion of the new multi get command [1], allowing for
>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>> again to Radim for a lot of the initial work.
>>>>
>>>> In doing so though I want to make sure I get feedback on how we want
>>>> the API to actually look, since this is much more difficult to change
>>>> down the line.
>>>>
>>>> There are a few things in particular I wanted to discuss.
>>>>
>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>> really getting a subset.  So I am thinking the possibilities at this
>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>> the last one (getMany).  I am open to any suggestions though.
>>> Actually Tristan suggested the name multiGet() - I would prefer this one
>>> in the end, and adding multiPut() that would just do putAll (to have the
>>> API symmetric) and deprecate the putAll() method. multiRemove and others
>>> can follow later, and the naming is straightforward.
>>> I would object against getAll() since this sounds like retrieving all
>>> entries from the cache, not just the specified keys.
>>>
>> Why a Set as parameter?
>>
>> If we have a Collection as parameter, it would allow a user to specify
>> easily the keys he want, like multiGet(Arrays.asList(k1, k2, k3)).
>
> It's Set as the collection should not contain duplicities. However, it's
> a bit questionable whether the collection should be copied for (async)
> internal processing or whether Infinispan takes ownership of this map
> (first option is more safe, second is faster).

I would say it shouldn't be copied, but I could put a comment on the
Javadoc if the Set is modified concurrently while the operation is
running the results returned will be nondeterministic.

>
> Radim
>
>>
>>>> 2. What should the return value be for this method.  Currently this
>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>> the local transactional context and is pretty nice and clean for end
>>>> users.
>>>>
>>>> The other idea is to use a streaming API possibly returning an
>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>> as intuitive to me, but opens us up for more efficient usage
>>>> especially when there may be a lot of keys passed in (also results can
>>>> be processed concurrently while retrieval is still occurring).
>>>>
>>>> I would lean towards returning the Map<K, V> value, however the next
>>>> point could change that.
>>> I think that Iterable<CacheEntry<K, V>> is too confusing for the end
>>> user, I would stick to the Map. If you want lazy loading, the Map (and
>>> it's EntrySet) could be made lazy by a flag.

The point wasn't that it was lazy, but rather that we could limit how
many values are maintained on the caller side to prevent excess memory
usage.

>> my vote is to use the Map<K,V> interface. The Iterable will force the
>> user to fetch the data using the iterator order, while the Map allow him
>> to use any order he wants (i.e. fetch particular keys via map.get(()).
>>
>> BTW, it can be good to wrap the Map in a unmodifiableMap().

I wonder if this really matters.  Each invocation would return its own
map, so if they modified it, it shouldn't matter.  I would make sure
to document that the Map is not backing map in that updates to the
cache or this map are not reflected in the other.

>>
>>>> 3. Where this method should live.  Right now this method is on the
>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>> efficient as possible we really need a Streaming API so that we don't
>>>> have multiple copies of data in memory at multiple places at the same
>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>> have a different API for remote vs embedded.
>>>>
>> IMO, we should have different API for embedded and remote.

+1 That is what I was thinking as well that we make the embedded API
nicer to use.  We can try to figure out the specifics of the remote
API later then.

>>
>> BasicCache does not seems a good place to put it. It can be placed in
>> AdvancedCache.

+1 I was thinking this may be the safest for now and we can always
move it to a higher class later if needed, however we can't move it
down easily.

>>
>> Cheers,
>> Pedro
>>
>>> Radim
>>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From ttarrant at redhat.com  Tue Mar 24 12:14:33 2015
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Tue, 24 Mar 2015 17:14:33 +0100
Subject: [infinispan-dev] Multi get API
In-Reply-To: <CA+YCuUOt3WPTQ6p_wJvHhSU_YwSMXjOR_GfD6YidbEy1dOQZXA@mail.gmail.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>	<55116D2F.3090707@redhat.com>
	<551176D7.5060306@infinispan.org>	<55117D78.7010101@redhat.com>
	<CA+YCuUOt3WPTQ6p_wJvHhSU_YwSMXjOR_GfD6YidbEy1dOQZXA@mail.gmail.com>
Message-ID: <55118D69.6030104@redhat.com>

On 24/03/2015 16:22, William Burns wrote:
> On Tue, Mar 24, 2015 at 11:06 AM, Radim Vansa <rvansa at redhat.com> wrote:
>> On 03/24/2015 03:38 PM, Pedro Ruivo wrote:
>>> Hi,
>>>
>>> comments inline...
>>>
>>> On 03/24/2015 01:57 PM, Radim Vansa wrote:
>>>> On 03/24/2015 02:22 PM, William Burns wrote:
>>>>> I am nearing completion of the new multi get command [1], allowing for
>>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>>> again to Radim for a lot of the initial work.
>>>>>
>>>>> In doing so though I want to make sure I get feedback on how we want
>>>>> the API to actually look, since this is much more difficult to change
>>>>> down the line.
>>>>>
>>>>> There are a few things in particular I wanted to discuss.
>>>>>
>>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>>> really getting a subset.  So I am thinking the possibilities at this
>>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>>> the last one (getMany).  I am open to any suggestions though.
>>>> Actually Tristan suggested the name multiGet() - I would prefer this one
>>>> in the end, and adding multiPut() that would just do putAll (to have the
>>>> API symmetric) and deprecate the putAll() method. multiRemove and others
>>>> can follow later, and the naming is straightforward.
>>>> I would object against getAll() since this sounds like retrieving all
>>>> entries from the cache, not just the specified keys.
>>>>
>>> Why a Set as parameter?
>>>
>>> If we have a Collection as parameter, it would allow a user to specify
>>> easily the keys he want, like multiGet(Arrays.asList(k1, k2, k3)).
>> It's Set as the collection should not contain duplicities. However, it's
>> a bit questionable whether the collection should be copied for (async)
>> internal processing or whether Infinispan takes ownership of this map
>> (first option is more safe, second is faster).
> I would say it shouldn't be copied, but I could put a comment on the
> Javadoc if the Set is modified concurrently while the operation is
> running the results returned will be nondeterministic.
The JCache contract for this is:

Map<K, V> getAll(Set<? extends K> keys);

Nothing is said about whether "keys" should be "safe", and I think that 
is absolutely fine for us as well (i.e. no copying, if you change the 
set, you get what you deserve).

>
>> Radim
>>
>>>>> 2. What should the return value be for this method.  Currently this
>>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>>> the local transactional context and is pretty nice and clean for end
>>>>> users.
>>>>>
>>>>> The other idea is to use a streaming API possibly returning an
>>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>>> as intuitive to me, but opens us up for more efficient usage
>>>>> especially when there may be a lot of keys passed in (also results can
>>>>> be processed concurrently while retrieval is still occurring).
>>>>>
>>>>> I would lean towards returning the Map<K, V> value, however the next
>>>>> point could change that.
>>>> I think that Iterable<CacheEntry<K, V>> is too confusing for the end
>>>> user, I would stick to the Map. If you want lazy loading, the Map (and
>>>> it's EntrySet) could be made lazy by a flag.
> The point wasn't that it was lazy, but rather that we could limit how
> many values are maintained on the caller side to prevent excess memory
> usage.
>
>>> my vote is to use the Map<K,V> interface. The Iterable will force the
>>> user to fetch the data using the iterator order, while the Map allow him
>>> to use any order he wants (i.e. fetch particular keys via map.get(()).
>>>
>>> BTW, it can be good to wrap the Map in a unmodifiableMap().
> I wonder if this really matters.  Each invocation would return its own
> map, so if they modified it, it shouldn't matter.  I would make sure
> to document that the Map is not backing map in that updates to the
> cache or this map are not reflected in the other.
Agree on all points. An iterable variant would probably more closely 
resemble the Java 8 streams, so I wouldn't overcomplicated this specific 
API.

>>>>> 3. Where this method should live.  Right now this method is on the
>>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>>> efficient as possible we really need a Streaming API so that we don't
>>>>> have multiple copies of data in memory at multiple places at the same
>>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>>> have a different API for remote vs embedded.
>>>>>
>>> IMO, we should have different API for embedded and remote.
> +1 That is what I was thinking as well that we make the embedded API
> nicer to use.  We can try to figure out the specifics of the remote
> API later then.
>
>>> BasicCache does not seems a good place to put it. It can be placed in
>>> AdvancedCache.
> +1 I was thinking this may be the safest for now and we can always
> move it to a higher class later if needed, however we can't move it
> down easily.
+1 for AdvancedCache for now, for Infinispan 8 Galder and I will discuss 
interfaces next week.

Tristan

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat


From mudokonman at gmail.com  Tue Mar 24 12:45:44 2015
From: mudokonman at gmail.com (William Burns)
Date: Tue, 24 Mar 2015 12:45:44 -0400
Subject: [infinispan-dev] Multi get API
In-Reply-To: <55118D69.6030104@redhat.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<55116D2F.3090707@redhat.com> <551176D7.5060306@infinispan.org>
	<55117D78.7010101@redhat.com>
	<CA+YCuUOt3WPTQ6p_wJvHhSU_YwSMXjOR_GfD6YidbEy1dOQZXA@mail.gmail.com>
	<55118D69.6030104@redhat.com>
Message-ID: <CA+YCuUOvo69SQgBdXcJKVvb3RO7voRZoYGd_Z7A=nnGzZBBUwQ@mail.gmail.com>

On Tue, Mar 24, 2015 at 12:14 PM, Tristan Tarrant <ttarrant at redhat.com> wrote:
> On 24/03/2015 16:22, William Burns wrote:
>> On Tue, Mar 24, 2015 at 11:06 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>> On 03/24/2015 03:38 PM, Pedro Ruivo wrote:
>>>> Hi,
>>>>
>>>> comments inline...
>>>>
>>>> On 03/24/2015 01:57 PM, Radim Vansa wrote:
>>>>> On 03/24/2015 02:22 PM, William Burns wrote:
>>>>>> I am nearing completion of the new multi get command [1], allowing for
>>>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>>>> again to Radim for a lot of the initial work.
>>>>>>
>>>>>> In doing so though I want to make sure I get feedback on how we want
>>>>>> the API to actually look, since this is much more difficult to change
>>>>>> down the line.
>>>>>>
>>>>>> There are a few things in particular I wanted to discuss.
>>>>>>
>>>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>>>> really getting a subset.  So I am thinking the possibilities at this
>>>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>>>> the last one (getMany).  I am open to any suggestions though.
>>>>> Actually Tristan suggested the name multiGet() - I would prefer this one
>>>>> in the end, and adding multiPut() that would just do putAll (to have the
>>>>> API symmetric) and deprecate the putAll() method. multiRemove and others
>>>>> can follow later, and the naming is straightforward.
>>>>> I would object against getAll() since this sounds like retrieving all
>>>>> entries from the cache, not just the specified keys.
>>>>>
>>>> Why a Set as parameter?
>>>>
>>>> If we have a Collection as parameter, it would allow a user to specify
>>>> easily the keys he want, like multiGet(Arrays.asList(k1, k2, k3)).
>>> It's Set as the collection should not contain duplicities. However, it's
>>> a bit questionable whether the collection should be copied for (async)
>>> internal processing or whether Infinispan takes ownership of this map
>>> (first option is more safe, second is faster).
>> I would say it shouldn't be copied, but I could put a comment on the
>> Javadoc if the Set is modified concurrently while the operation is
>> running the results returned will be nondeterministic.
> The JCache contract for this is:
>
> Map<K, V> getAll(Set<? extends K> keys);

Ah, yes I had forgotten this was already on JCache, in that case we
have a clear precedent and full API definition.  I am thinking to be
consistent using this would be best and later we can look into Stream
API as was mentioned.

>
> Nothing is said about whether "keys" should be "safe", and I think that
> is absolutely fine for us as well (i.e. no copying, if you change the
> set, you get what you deserve).
>
>>
>>> Radim
>>>
>>>>>> 2. What should the return value be for this method.  Currently this
>>>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>>>> the local transactional context and is pretty nice and clean for end
>>>>>> users.
>>>>>>
>>>>>> The other idea is to use a streaming API possibly returning an
>>>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>>>> as intuitive to me, but opens us up for more efficient usage
>>>>>> especially when there may be a lot of keys passed in (also results can
>>>>>> be processed concurrently while retrieval is still occurring).
>>>>>>
>>>>>> I would lean towards returning the Map<K, V> value, however the next
>>>>>> point could change that.
>>>>> I think that Iterable<CacheEntry<K, V>> is too confusing for the end
>>>>> user, I would stick to the Map. If you want lazy loading, the Map (and
>>>>> it's EntrySet) could be made lazy by a flag.
>> The point wasn't that it was lazy, but rather that we could limit how
>> many values are maintained on the caller side to prevent excess memory
>> usage.
>>
>>>> my vote is to use the Map<K,V> interface. The Iterable will force the
>>>> user to fetch the data using the iterator order, while the Map allow him
>>>> to use any order he wants (i.e. fetch particular keys via map.get(()).
>>>>
>>>> BTW, it can be good to wrap the Map in a unmodifiableMap().
>> I wonder if this really matters.  Each invocation would return its own
>> map, so if they modified it, it shouldn't matter.  I would make sure
>> to document that the Map is not backing map in that updates to the
>> cache or this map are not reflected in the other.
> Agree on all points. An iterable variant would probably more closely
> resemble the Java 8 streams, so I wouldn't overcomplicated this specific
> API.
>
>>>>>> 3. Where this method should live.  Right now this method is on the
>>>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>>>> efficient as possible we really need a Streaming API so that we don't
>>>>>> have multiple copies of data in memory at multiple places at the same
>>>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>>>> have a different API for remote vs embedded.
>>>>>>
>>>> IMO, we should have different API for embedded and remote.
>> +1 That is what I was thinking as well that we make the embedded API
>> nicer to use.  We can try to figure out the specifics of the remote
>> API later then.
>>
>>>> BasicCache does not seems a good place to put it. It can be placed in
>>>> AdvancedCache.
>> +1 I was thinking this may be the safest for now and we can always
>> move it to a higher class later if needed, however we can't move it
>> down easily.
> +1 for AdvancedCache for now, for Infinispan 8 Galder and I will discuss
> interfaces next week.
>
> Tristan
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From mudokonman at gmail.com  Tue Mar 24 14:29:46 2015
From: mudokonman at gmail.com (William Burns)
Date: Tue, 24 Mar 2015 14:29:46 -0400
Subject: [infinispan-dev] Multi get API
In-Reply-To: <CA+YCuUOvo69SQgBdXcJKVvb3RO7voRZoYGd_Z7A=nnGzZBBUwQ@mail.gmail.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<55116D2F.3090707@redhat.com> <551176D7.5060306@infinispan.org>
	<55117D78.7010101@redhat.com>
	<CA+YCuUOt3WPTQ6p_wJvHhSU_YwSMXjOR_GfD6YidbEy1dOQZXA@mail.gmail.com>
	<55118D69.6030104@redhat.com>
	<CA+YCuUOvo69SQgBdXcJKVvb3RO7voRZoYGd_Z7A=nnGzZBBUwQ@mail.gmail.com>
Message-ID: <CA+YCuUN59+-mrp5=JzOtWpdCqQXi5BGum2r+Rko-EmzWP6b69A@mail.gmail.com>

On Tue, Mar 24, 2015 at 12:45 PM, William Burns <mudokonman at gmail.com> wrote:
> On Tue, Mar 24, 2015 at 12:14 PM, Tristan Tarrant <ttarrant at redhat.com> wrote:
>> On 24/03/2015 16:22, William Burns wrote:
>>> On Tue, Mar 24, 2015 at 11:06 AM, Radim Vansa <rvansa at redhat.com> wrote:
>>>> On 03/24/2015 03:38 PM, Pedro Ruivo wrote:
>>>>> Hi,
>>>>>
>>>>> comments inline...
>>>>>
>>>>> On 03/24/2015 01:57 PM, Radim Vansa wrote:
>>>>>> On 03/24/2015 02:22 PM, William Burns wrote:
>>>>>>> I am nearing completion of the new multi get command [1], allowing for
>>>>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>>>>> again to Radim for a lot of the initial work.
>>>>>>>
>>>>>>> In doing so though I want to make sure I get feedback on how we want
>>>>>>> the API to actually look, since this is much more difficult to change
>>>>>>> down the line.
>>>>>>>
>>>>>>> There are a few things in particular I wanted to discuss.
>>>>>>>
>>>>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>>>>> really getting a subset.  So I am thinking the possibilities at this
>>>>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>>>>> the last one (getMany).  I am open to any suggestions though.
>>>>>> Actually Tristan suggested the name multiGet() - I would prefer this one
>>>>>> in the end, and adding multiPut() that would just do putAll (to have the
>>>>>> API symmetric) and deprecate the putAll() method. multiRemove and others
>>>>>> can follow later, and the naming is straightforward.
>>>>>> I would object against getAll() since this sounds like retrieving all
>>>>>> entries from the cache, not just the specified keys.
>>>>>>
>>>>> Why a Set as parameter?
>>>>>
>>>>> If we have a Collection as parameter, it would allow a user to specify
>>>>> easily the keys he want, like multiGet(Arrays.asList(k1, k2, k3)).
>>>> It's Set as the collection should not contain duplicities. However, it's
>>>> a bit questionable whether the collection should be copied for (async)
>>>> internal processing or whether Infinispan takes ownership of this map
>>>> (first option is more safe, second is faster).
>>> I would say it shouldn't be copied, but I could put a comment on the
>>> Javadoc if the Set is modified concurrently while the operation is
>>> running the results returned will be nondeterministic.
>> The JCache contract for this is:
>>
>> Map<K, V> getAll(Set<? extends K> keys);
>
> Ah, yes I had forgotten this was already on JCache, in that case we
> have a clear precedent and full API definition.  I am thinking to be
> consistent using this would be best and later we can look into Stream
> API as was mentioned.

Actually thinking about this more, the JCache API for getAll seems
wrong.  It seems it should be Map<K, V> getAll(Set<?> keys).  Putting
an upper or lower bounds on the type is too restrictive.  By doing
Set<? extends K> I can't pass in a Set<Object> for example.  Looking
at the various Collection classes they do a similar thing with
containsAll, retainAll, removeAll etc.  I am thinking I am going to
have ours be Set<?> then instead.  We will still have compatibility
with JCache as well.

>
>>
>> Nothing is said about whether "keys" should be "safe", and I think that
>> is absolutely fine for us as well (i.e. no copying, if you change the
>> set, you get what you deserve).
>>
>>>
>>>> Radim
>>>>
>>>>>>> 2. What should the return value be for this method.  Currently this
>>>>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>>>>> the local transactional context and is pretty nice and clean for end
>>>>>>> users.
>>>>>>>
>>>>>>> The other idea is to use a streaming API possibly returning an
>>>>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>>>>> as intuitive to me, but opens us up for more efficient usage
>>>>>>> especially when there may be a lot of keys passed in (also results can
>>>>>>> be processed concurrently while retrieval is still occurring).
>>>>>>>
>>>>>>> I would lean towards returning the Map<K, V> value, however the next
>>>>>>> point could change that.
>>>>>> I think that Iterable<CacheEntry<K, V>> is too confusing for the end
>>>>>> user, I would stick to the Map. If you want lazy loading, the Map (and
>>>>>> it's EntrySet) could be made lazy by a flag.
>>> The point wasn't that it was lazy, but rather that we could limit how
>>> many values are maintained on the caller side to prevent excess memory
>>> usage.
>>>
>>>>> my vote is to use the Map<K,V> interface. The Iterable will force the
>>>>> user to fetch the data using the iterator order, while the Map allow him
>>>>> to use any order he wants (i.e. fetch particular keys via map.get(()).
>>>>>
>>>>> BTW, it can be good to wrap the Map in a unmodifiableMap().
>>> I wonder if this really matters.  Each invocation would return its own
>>> map, so if they modified it, it shouldn't matter.  I would make sure
>>> to document that the Map is not backing map in that updates to the
>>> cache or this map are not reflected in the other.
>> Agree on all points. An iterable variant would probably more closely
>> resemble the Java 8 streams, so I wouldn't overcomplicated this specific
>> API.
>>
>>>>>>> 3. Where this method should live.  Right now this method is on the
>>>>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>>>>> efficient as possible we really need a Streaming API so that we don't
>>>>>>> have multiple copies of data in memory at multiple places at the same
>>>>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>>>>> have a different API for remote vs embedded.
>>>>>>>
>>>>> IMO, we should have different API for embedded and remote.
>>> +1 That is what I was thinking as well that we make the embedded API
>>> nicer to use.  We can try to figure out the specifics of the remote
>>> API later then.
>>>
>>>>> BasicCache does not seems a good place to put it. It can be placed in
>>>>> AdvancedCache.
>>> +1 I was thinking this may be the safest for now and we can always
>>> move it to a higher class later if needed, however we can't move it
>>> down easily.
>> +1 for AdvancedCache for now, for Infinispan 8 Galder and I will discuss
>> interfaces next week.
>>
>> Tristan
>>
>> --
>> Tristan Tarrant
>> Infinispan Lead
>> JBoss, a division of Red Hat
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From galder at redhat.com  Thu Mar 26 04:14:55 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Thu, 26 Mar 2015 09:14:55 +0100
Subject: [infinispan-dev] Random SocketTimeoutExceptions demonstrate
	we're not using Netty the right way
In-Reply-To: <CA+nfvwRVY9wnca=Z4MuDEiRzAOCSr=3i5gu70ZA-URs_H72jiQ@mail.gmail.com>
References: <FECA2706-C717-412A-BD62-F4F7C10557E2@redhat.com>
	<CA+nfvwRVY9wnca=Z4MuDEiRzAOCSr=3i5gu70ZA-URs_H72jiQ@mail.gmail.com>
Message-ID: <F82CD3FE-0E7A-4C1D-904D-44B7710B142C@redhat.com>


> On 20 Mar 2015, at 18:32, Dan Berindei <dan.berindei at gmail.com> wrote:
> 
> Hi Galder
> 
> Using cache.putAsync on the server will use a thread from the "async"
> transport executor. While the default number of threads in the
> transport executor is higher (25), it's still going to be exhausted at
> some point as you're generating more and more duplicate operations
> from the client.

^ Actually, I discovered that I had 2 as worker threads because the testsuite was overriding the default and setting it to 2.

> Using async operations will be a better solution once the blocked
> operations don't need to keep a thread busy in core. But ATM I believe
> a better solution would be to increase the socket timeout on the
> client so that we get a TimeoutException from the server before we get
> the SocketTimeoutException on the client.

Indeed, currently socket timeout needs to be aligned with the rest of configuration details, and we do often recommend increasing it in systems under load, but the point is that in the current set up, server worker threads numbers also need to be aligned correctly.

Cheers,

> 
> Cheers
> Dan
> 
> 
> On Fri, Mar 20, 2015 at 5:22 PM, Galder Zamarre?o <galder at redhat.com> wrote:
>> Hi all,
>> 
>> Summary: I've been debugging [1] and found that an unexpected operation throwing a SocketTE was being thrown due to the worker thread pool being too small, but the real problem is the fact that we can block within Netty's worker thread, something we're hoping to fix with Gustavo's work to implement [2].
>> 
>> The test in [1] was failing randomly as a result of this sequence of events:
>> 
>> 1. Server configured with worker thread pool being 2 (in code, this is 2 * Runtime.getRuntime().availableProcessors())
>> 2. We emulate a server side operation hanging with a sleep injected by an interceptor.
>> 3. Client gets SocketTimeoutException and retries the operation.
>> 4. The retried operation hangs because it cannot acquire the lock.
>> 5. Client gets SocketTimeoutException again, runs out of retries.
>> 6. Test now executes an operation that should not throw a SocketTE but it gets it because the 2 worker threads are in use. The first is in the sleep, and the retried one waiting for the lock, so it cannot make any progress and hence ends up getting a SocketTE.
>> 
>> You might think this can be solved by increasing the default worker thread pool size but you're just moving the problem around. You could have N operations that hang at the same time, e.g. if there's a long GC pause.
>> 
>> So really, we need to stop blocking within the worker thread and implement [2].
>> 
>> I think we might have seen this issue already back in Infinispan 5, because we were already getting SocketTEs back then in another SocketTE test [3]. I wasn't able to replicate it locally at the time and solved it by ignoring it :|
>> 
>> As far as ISPN-5314 is concerned, it can easily be solved by increasing the worker thread pool for that particular test, but we should revert that back when [2] is implemented.
>> 
>> Cheers,
>> 
>> [1] https://issues.jboss.org/browse/ISPN-5314
>> [2] https://issues.jboss.org/browse/ISPN-5083
>> [3] https://issues.jboss.org/browse/ISPN-2110
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From galder at redhat.com  Thu Mar 26 05:14:57 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Thu, 26 Mar 2015 10:14:57 +0100
Subject: [infinispan-dev] Status update weekly meeting
Message-ID: <A0807E2D-C8CB-430C-BAB3-A98CE940EC2D@redhat.com>

Hi,

Forgot to send this week's meeting log, here it is:
http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2015/infinispan.2015-03-23-15.05.html

Cheers,
--
Galder Zamarre?o
galder at redhat.com






From galder at redhat.com  Thu Mar 26 05:28:49 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Thu, 26 Mar 2015 10:28:49 +0100
Subject: [infinispan-dev] Multi get API
In-Reply-To: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
Message-ID: <9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>

Comments inline:

> On 24 Mar 2015, at 14:22, William Burns <mudokonman at gmail.com> wrote:
> 
> I am nearing completion of the new multi get command [1], allowing for
> more efficient retrieval of multiple keys at the same time.  Thanks
> again to Radim for a lot of the initial work.
> 
> In doing so though I want to make sure I get feedback on how we want
> the API to actually look, since this is much more difficult to change
> down the line.
> 
> There are a few things in particular I wanted to discuss.
> 
> 1. The actual name of the method.  The main suggestions I have seen
> are getAll or getMany.  I do like the naming of the former, however it
> seems a bit misleading (remember API is getAll(Set) since we are
> really getting a subset.  So I am thinking the possibilities at this
> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
> the last one (getMany).  I am open to any suggestions though.

JSR-107 has getAll(Set), which I think it's a bit confusing too naming wise.

getMany or getAllOf sound better to me.

> 2. What should the return value be for this method.  Currently this
> returns a Map<K, V> which makes sense when we retain these values in
> the local transactional context and is pretty nice and clean for end
> users.
> 
> The other idea is to use a streaming API possibly returning an
> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
> as intuitive to me, but opens us up for more efficient usage
> especially when there may be a lot of keys passed in (also results can
> be processed concurrently while retrieval is still occurring).
> 
> I would lean towards returning the Map<K, V> value, however the next
> point could change that.
> 
> 3. Where this method should live.  Right now this method is on the
> BasicCache interface which is a parent of both Cache (embedded) and
> RemoteCache (remote).  Unfortunately for remote clients to be as
> efficient as possible we really need a Streaming API so that we don't
> have multiple copies of data in memory at multiple places at the same
> time.  For that reason I suggest we use the streaminig API for both or
> have a different API for remote vs embedded.

Hmmm, I've been thinking about the same thing for Java 8 API (see the proposal wiki), and I think returning a Map limits things. On Java 8, I'm considering switching `evalAll` from returning a Map of CompletableFutures to a Stream... More to come next week.

Now, for the current version, I think Map is limiting for both remote and embedded, because even for embedded, retrieving an entry might be a remote operation, so that forces you to wait for all of the entries to be available before you can return the map. Same thing for Remote.

What about returning an Iterator? That could be lazy and more suited for the use case? If you need to return a Map, e.g. to implement JCache.getAll(), you can wait for all and construct a map.

Now, whether to have a separate API for this, since we have Java 8 API coming up, I'd not worry too much at this point.

Cheers,

> 
> Let me know what you guys think.
> 
> Cheers,
> 
> - Will
> 
> [1] https://issues.jboss.org/browse/ISPN-2183
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From rvansa at redhat.com  Thu Mar 26 06:04:00 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 26 Mar 2015 11:04:00 +0100
Subject: [infinispan-dev] Multi get API
In-Reply-To: <9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>
Message-ID: <5513D990.7000600@redhat.com>

On 03/26/2015 10:28 AM, Galder Zamarre?o wrote:
> Comments inline:
>
>> On 24 Mar 2015, at 14:22, William Burns <mudokonman at gmail.com> wrote:
>>
>> I am nearing completion of the new multi get command [1], allowing for
>> more efficient retrieval of multiple keys at the same time.  Thanks
>> again to Radim for a lot of the initial work.
>>
>> In doing so though I want to make sure I get feedback on how we want
>> the API to actually look, since this is much more difficult to change
>> down the line.
>>
>> There are a few things in particular I wanted to discuss.
>>
>> 1. The actual name of the method.  The main suggestions I have seen
>> are getAll or getMany.  I do like the naming of the former, however it
>> seems a bit misleading (remember API is getAll(Set) since we are
>> really getting a subset.  So I am thinking the possibilities at this
>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>> the last one (getMany).  I am open to any suggestions though.
> JSR-107 has getAll(Set), which I think it's a bit confusing too naming wise.
>
> getMany or getAllOf sound better to me.
>
>> 2. What should the return value be for this method.  Currently this
>> returns a Map<K, V> which makes sense when we retain these values in
>> the local transactional context and is pretty nice and clean for end
>> users.
>>
>> The other idea is to use a streaming API possibly returning an
>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>> as intuitive to me, but opens us up for more efficient usage
>> especially when there may be a lot of keys passed in (also results can
>> be processed concurrently while retrieval is still occurring).
>>
>> I would lean towards returning the Map<K, V> value, however the next
>> point could change that.
>>
>> 3. Where this method should live.  Right now this method is on the
>> BasicCache interface which is a parent of both Cache (embedded) and
>> RemoteCache (remote).  Unfortunately for remote clients to be as
>> efficient as possible we really need a Streaming API so that we don't
>> have multiple copies of data in memory at multiple places at the same
>> time.  For that reason I suggest we use the streaminig API for both or
>> have a different API for remote vs embedded.
> Hmmm, I've been thinking about the same thing for Java 8 API (see the proposal wiki), and I think returning a Map limits things. On Java 8, I'm considering switching `evalAll` from returning a Map of CompletableFutures to a Stream... More to come next week.
>
> Now, for the current version, I think Map is limiting for both remote and embedded, because even for embedded, retrieving an entry might be a remote operation, so that forces you to wait for all of the entries to be available before you can return the map. Same thing for Remote.
>
> What about returning an Iterator? That could be lazy and more suited for the use case? If you need to return a Map, e.g. to implement JCache.getAll(), you can wait for all and construct a map.

What's the expected usecase? I would expect that users want to just 
retrieve few keys at once, without any fancy lazy 
wait-for-completion-and-do-after stuff. Just imagine that you want to 
pass the two values into another method, Iterator/Stream API wouldn't 
help you with that much.

It seems to me that it is very useful to have multi get with Map<K, V> 
as return type in the API - we don't want to force constructs like:

Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
for (; it.hasNext();) {
     Map.Entry<K, V> entry = it.next();
     if (entry.getKey().equals(key1)) value1 = entry.getValue();
     else if (entry.getKey().equals(key2)) value2 = entry.getValue();
     else if (entry.getKey().equals(key3)) value3 = entry.getValue();
}

or let them flip it into the map manually (HashMap does not have 
constructor accepting iterator)

Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
Map<K, V> map = new HashMap<>();
for (; it.hasNext();) {
     Map.Entry<K, V> entry = it.next();
     map.put(entry.getKey(), entry.getValue());
}


>
> Now, whether to have a separate API for this, since we have Java 8 API coming up, I'd not worry too much at this point.

It seems to me that with 8, we're going to have complicated lazy 
omnipotent API, and several facades simplifying that into Map, JCache or 
whatever. So, let's provide the simplistic API with Map now, and the 
lazy versions can get into the lazy API as soon as we'll see how the 
other methods will look like.

Radim

>
> Cheers,
>
>> Let me know what you guys think.
>>
>> Cheers,
>>
>> - Will
>>
>> [1] https://issues.jboss.org/browse/ISPN-2183
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Galder Zamarre?o
> galder at redhat.com
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From sanne at infinispan.org  Thu Mar 26 08:50:02 2015
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 26 Mar 2015 12:50:02 +0000
Subject: [infinispan-dev] Multi get API
In-Reply-To: <5513D990.7000600@redhat.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>
	<5513D990.7000600@redhat.com>
Message-ID: <CAFm4XO1=MNLpf+u6OMZX0Y5_1T6q=j1_3fFehOSazxNHAfnXMg@mail.gmail.com>

In terms of naming, I'd prefer _getMulti_.
My reasoning is it's nice if has a common prefix with _get_ as that
helps someone exploring the API from an IDE, giving better assistance
to people learning.

getAll is misleading, as is getMany or get getAllOf as they imply I'm
searching for something based on a pattern but you're expecting the
keys and returning tuples key/value.

I like the idea of returning a Map, as you can actually make the map
behave like a Future: the Map instance could be returned right away,
but retrieving of some entries would be blocking until they are
retrieved. You wouldn't need to block on having all entries, just the
ones being actively asked for. In other words, it's better than
returning a Future<Map>.
This should invalidate some of the previous concerns about using Map ?

@William I didn't understand your concern on upper and lower bounds.
The returned types should be coupled to the Cache types; being able to
pass Object might be nice but let's make sure no casting is required
on the client? Could you make some examples of your alternative
proposal?

Cheers,
Sanne



On 26 March 2015 at 10:04, Radim Vansa <rvansa at redhat.com> wrote:
> On 03/26/2015 10:28 AM, Galder Zamarre?o wrote:
>> Comments inline:
>>
>>> On 24 Mar 2015, at 14:22, William Burns <mudokonman at gmail.com> wrote:
>>>
>>> I am nearing completion of the new multi get command [1], allowing for
>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>> again to Radim for a lot of the initial work.
>>>
>>> In doing so though I want to make sure I get feedback on how we want
>>> the API to actually look, since this is much more difficult to change
>>> down the line.
>>>
>>> There are a few things in particular I wanted to discuss.
>>>
>>> 1. The actual name of the method.  The main suggestions I have seen
>>> are getAll or getMany.  I do like the naming of the former, however it
>>> seems a bit misleading (remember API is getAll(Set) since we are
>>> really getting a subset.  So I am thinking the possibilities at this
>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>> the last one (getMany).  I am open to any suggestions though.
>> JSR-107 has getAll(Set), which I think it's a bit confusing too naming wise.
>>
>> getMany or getAllOf sound better to me.
>>
>>> 2. What should the return value be for this method.  Currently this
>>> returns a Map<K, V> which makes sense when we retain these values in
>>> the local transactional context and is pretty nice and clean for end
>>> users.
>>>
>>> The other idea is to use a streaming API possibly returning an
>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>> as intuitive to me, but opens us up for more efficient usage
>>> especially when there may be a lot of keys passed in (also results can
>>> be processed concurrently while retrieval is still occurring).
>>>
>>> I would lean towards returning the Map<K, V> value, however the next
>>> point could change that.
>>>
>>> 3. Where this method should live.  Right now this method is on the
>>> BasicCache interface which is a parent of both Cache (embedded) and
>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>> efficient as possible we really need a Streaming API so that we don't
>>> have multiple copies of data in memory at multiple places at the same
>>> time.  For that reason I suggest we use the streaminig API for both or
>>> have a different API for remote vs embedded.
>> Hmmm, I've been thinking about the same thing for Java 8 API (see the proposal wiki), and I think returning a Map limits things. On Java 8, I'm considering switching `evalAll` from returning a Map of CompletableFutures to a Stream... More to come next week.
>>
>> Now, for the current version, I think Map is limiting for both remote and embedded, because even for embedded, retrieving an entry might be a remote operation, so that forces you to wait for all of the entries to be available before you can return the map. Same thing for Remote.
>>
>> What about returning an Iterator? That could be lazy and more suited for the use case? If you need to return a Map, e.g. to implement JCache.getAll(), you can wait for all and construct a map.
>
> What's the expected usecase? I would expect that users want to just
> retrieve few keys at once, without any fancy lazy
> wait-for-completion-and-do-after stuff. Just imagine that you want to
> pass the two values into another method, Iterator/Stream API wouldn't
> help you with that much.
>
> It seems to me that it is very useful to have multi get with Map<K, V>
> as return type in the API - we don't want to force constructs like:
>
> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
> for (; it.hasNext();) {
>      Map.Entry<K, V> entry = it.next();
>      if (entry.getKey().equals(key1)) value1 = entry.getValue();
>      else if (entry.getKey().equals(key2)) value2 = entry.getValue();
>      else if (entry.getKey().equals(key3)) value3 = entry.getValue();
> }
>
> or let them flip it into the map manually (HashMap does not have
> constructor accepting iterator)
>
> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
> Map<K, V> map = new HashMap<>();
> for (; it.hasNext();) {
>      Map.Entry<K, V> entry = it.next();
>      map.put(entry.getKey(), entry.getValue());
> }
>
>
>>
>> Now, whether to have a separate API for this, since we have Java 8 API coming up, I'd not worry too much at this point.
>
> It seems to me that with 8, we're going to have complicated lazy
> omnipotent API, and several facades simplifying that into Map, JCache or
> whatever. So, let's provide the simplistic API with Map now, and the
> lazy versions can get into the lazy API as soon as we'll see how the
> other methods will look like.
>
> Radim
>
>>
>> Cheers,
>>
>>> Let me know what you guys think.
>>>
>>> Cheers,
>>>
>>> - Will
>>>
>>> [1] https://issues.jboss.org/browse/ISPN-2183
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Galder Zamarre?o
>> galder at redhat.com
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From mudokonman at gmail.com  Thu Mar 26 09:43:17 2015
From: mudokonman at gmail.com (William Burns)
Date: Thu, 26 Mar 2015 09:43:17 -0400
Subject: [infinispan-dev] Multi get API
In-Reply-To: <CAFm4XO1=MNLpf+u6OMZX0Y5_1T6q=j1_3fFehOSazxNHAfnXMg@mail.gmail.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>
	<5513D990.7000600@redhat.com>
	<CAFm4XO1=MNLpf+u6OMZX0Y5_1T6q=j1_3fFehOSazxNHAfnXMg@mail.gmail.com>
Message-ID: <CA+YCuUMmL=jrk-SD5JFhdChrJ=0oA64QA8ftbaqb5K0BfMdB1A@mail.gmail.com>

On Thu, Mar 26, 2015 at 8:50 AM, Sanne Grinovero <sanne at infinispan.org> wrote:
> In terms of naming, I'd prefer _getMulti_.
> My reasoning is it's nice if has a common prefix with _get_ as that
> helps someone exploring the API from an IDE, giving better assistance
> to people learning.
>
> getAll is misleading, as is getMany or get getAllOf as they imply I'm
> searching for something based on a pattern but you're expecting the
> keys and returning tuples key/value.

I can understand your point, however since JCache already defined this
as getAll I can't see but using that name.

>
> I like the idea of returning a Map, as you can actually make the map
> behave like a Future: the Map instance could be returned right away,
> but retrieving of some entries would be blocking until they are
> retrieved. You wouldn't need to block on having all entries, just the
> ones being actively asked for. In other words, it's better than
> returning a Future<Map>.
> This should invalidate some of the previous concerns about using Map ?

I had thought about this briefly, but didn't give it much though since
it makes things quite complicated (interceptor stack and tx context I
am looking at you).  I would say to do this later if we wanted, since
the API doesn't have to be different (other than the fact that people
should start possibly expecting CacheExceptions from calling Map.get
and other methods).

>
> @William I didn't understand your concern on upper and lower bounds.
> The returned types should be coupled to the Cache types; being able to

The return type is coupled to the Cache types, this was about the
input type defined on the Set.

> pass Object might be nice but let's make sure no casting is required
> on the client? Could you make some examples of your alternative
> proposal?

With Set<?> the client would never have to do any casting.

>
> Cheers,
> Sanne
>
>
>
> On 26 March 2015 at 10:04, Radim Vansa <rvansa at redhat.com> wrote:
>> On 03/26/2015 10:28 AM, Galder Zamarre?o wrote:
>>> Comments inline:
>>>
>>>> On 24 Mar 2015, at 14:22, William Burns <mudokonman at gmail.com> wrote:
>>>>
>>>> I am nearing completion of the new multi get command [1], allowing for
>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>> again to Radim for a lot of the initial work.
>>>>
>>>> In doing so though I want to make sure I get feedback on how we want
>>>> the API to actually look, since this is much more difficult to change
>>>> down the line.
>>>>
>>>> There are a few things in particular I wanted to discuss.
>>>>
>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>> really getting a subset.  So I am thinking the possibilities at this
>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>> the last one (getMany).  I am open to any suggestions though.
>>> JSR-107 has getAll(Set), which I think it's a bit confusing too naming wise.
>>>
>>> getMany or getAllOf sound better to me.
>>>
>>>> 2. What should the return value be for this method.  Currently this
>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>> the local transactional context and is pretty nice and clean for end
>>>> users.
>>>>
>>>> The other idea is to use a streaming API possibly returning an
>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>> as intuitive to me, but opens us up for more efficient usage
>>>> especially when there may be a lot of keys passed in (also results can
>>>> be processed concurrently while retrieval is still occurring).
>>>>
>>>> I would lean towards returning the Map<K, V> value, however the next
>>>> point could change that.
>>>>
>>>> 3. Where this method should live.  Right now this method is on the
>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>> efficient as possible we really need a Streaming API so that we don't
>>>> have multiple copies of data in memory at multiple places at the same
>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>> have a different API for remote vs embedded.
>>> Hmmm, I've been thinking about the same thing for Java 8 API (see the proposal wiki), and I think returning a Map limits things. On Java 8, I'm considering switching `evalAll` from returning a Map of CompletableFutures to a Stream... More to come next week.
>>>
>>> Now, for the current version, I think Map is limiting for both remote and embedded, because even for embedded, retrieving an entry might be a remote operation, so that forces you to wait for all of the entries to be available before you can return the map. Same thing for Remote.
>>>
>>> What about returning an Iterator? That could be lazy and more suited for the use case? If you need to return a Map, e.g. to implement JCache.getAll(), you can wait for all and construct a map.
>>
>> What's the expected usecase? I would expect that users want to just
>> retrieve few keys at once, without any fancy lazy
>> wait-for-completion-and-do-after stuff. Just imagine that you want to
>> pass the two values into another method, Iterator/Stream API wouldn't
>> help you with that much.
>>
>> It seems to me that it is very useful to have multi get with Map<K, V>
>> as return type in the API - we don't want to force constructs like:
>>
>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>> for (; it.hasNext();) {
>>      Map.Entry<K, V> entry = it.next();
>>      if (entry.getKey().equals(key1)) value1 = entry.getValue();
>>      else if (entry.getKey().equals(key2)) value2 = entry.getValue();
>>      else if (entry.getKey().equals(key3)) value3 = entry.getValue();
>> }
>>
>> or let them flip it into the map manually (HashMap does not have
>> constructor accepting iterator)
>>
>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>> Map<K, V> map = new HashMap<>();
>> for (; it.hasNext();) {
>>      Map.Entry<K, V> entry = it.next();
>>      map.put(entry.getKey(), entry.getValue());
>> }
>>
>>
>>>
>>> Now, whether to have a separate API for this, since we have Java 8 API coming up, I'd not worry too much at this point.
>>
>> It seems to me that with 8, we're going to have complicated lazy
>> omnipotent API, and several facades simplifying that into Map, JCache or
>> whatever. So, let's provide the simplistic API with Map now, and the
>> lazy versions can get into the lazy API as soon as we'll see how the
>> other methods will look like.
>>
>> Radim
>>
>>>
>>> Cheers,
>>>
>>>> Let me know what you guys think.
>>>>
>>>> Cheers,
>>>>
>>>> - Will
>>>>
>>>> [1] https://issues.jboss.org/browse/ISPN-2183
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Galder Zamarre?o
>>> galder at redhat.com
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From sanne at infinispan.org  Thu Mar 26 10:00:27 2015
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 26 Mar 2015 14:00:27 +0000
Subject: [infinispan-dev] Multi get API
In-Reply-To: <CA+YCuUMmL=jrk-SD5JFhdChrJ=0oA64QA8ftbaqb5K0BfMdB1A@mail.gmail.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>
	<5513D990.7000600@redhat.com>
	<CAFm4XO1=MNLpf+u6OMZX0Y5_1T6q=j1_3fFehOSazxNHAfnXMg@mail.gmail.com>
	<CA+YCuUMmL=jrk-SD5JFhdChrJ=0oA64QA8ftbaqb5K0BfMdB1A@mail.gmail.com>
Message-ID: <CAFm4XO2zT2Q=qzxqhyvYsbmBYLFpGb5ePx7eyqHdSrL9+t1R0w@mail.gmail.com>

On 26 March 2015 at 13:43, William Burns <mudokonman at gmail.com> wrote:
> On Thu, Mar 26, 2015 at 8:50 AM, Sanne Grinovero <sanne at infinispan.org> wrote:
>> In terms of naming, I'd prefer _getMulti_.
>> My reasoning is it's nice if has a common prefix with _get_ as that
>> helps someone exploring the API from an IDE, giving better assistance
>> to people learning.
>>
>> getAll is misleading, as is getMany or get getAllOf as they imply I'm
>> searching for something based on a pattern but you're expecting the
>> keys and returning tuples key/value.
>
> I can understand your point, however since JCache already defined this
> as getAll I can't see but using that name.

Ok that makes it win a bit of points, and at least it starts with _get_.


>> I like the idea of returning a Map, as you can actually make the map
>> behave like a Future: the Map instance could be returned right away,
>> but retrieving of some entries would be blocking until they are
>> retrieved. You wouldn't need to block on having all entries, just the
>> ones being actively asked for. In other words, it's better than
>> returning a Future<Map>.
>> This should invalidate some of the previous concerns about using Map ?
>
> I had thought about this briefly, but didn't give it much though since
> it makes things quite complicated (interceptor stack and tx context I
> am looking at you).  I would say to do this later if we wanted, since
> the API doesn't have to be different (other than the fact that people
> should start possibly expecting CacheExceptions from calling Map.get
> and other methods).
>
>>
>> @William I didn't understand your concern on upper and lower bounds.
>> The returned types should be coupled to the Cache types; being able to
>
> The return type is coupled to the Cache types, this was about the
> input type defined on the Set.
>
>> pass Object might be nice but let's make sure no casting is required
>> on the client? Could you make some examples of your alternative
>> proposal?
>
> With Set<?> the client would never have to do any casting.

I'm confused. Not sure which Set you're talking about. And remember
that if you're being very liberal with the promise of returned types,
how is the implementation going to handle the guarantee?

For example, what's going to happen if the requested objects don't
match the user expected type? The API shouldn't make promises it can't
maintain.

Thanks,
Sanne

>> On 26 March 2015 at 10:04, Radim Vansa <rvansa at redhat.com> wrote:
>>> On 03/26/2015 10:28 AM, Galder Zamarre?o wrote:
>>>> Comments inline:
>>>>
>>>>> On 24 Mar 2015, at 14:22, William Burns <mudokonman at gmail.com> wrote:
>>>>>
>>>>> I am nearing completion of the new multi get command [1], allowing for
>>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>>> again to Radim for a lot of the initial work.
>>>>>
>>>>> In doing so though I want to make sure I get feedback on how we want
>>>>> the API to actually look, since this is much more difficult to change
>>>>> down the line.
>>>>>
>>>>> There are a few things in particular I wanted to discuss.
>>>>>
>>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>>> really getting a subset.  So I am thinking the possibilities at this
>>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>>> the last one (getMany).  I am open to any suggestions though.
>>>> JSR-107 has getAll(Set), which I think it's a bit confusing too naming wise.
>>>>
>>>> getMany or getAllOf sound better to me.
>>>>
>>>>> 2. What should the return value be for this method.  Currently this
>>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>>> the local transactional context and is pretty nice and clean for end
>>>>> users.
>>>>>
>>>>> The other idea is to use a streaming API possibly returning an
>>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>>> as intuitive to me, but opens us up for more efficient usage
>>>>> especially when there may be a lot of keys passed in (also results can
>>>>> be processed concurrently while retrieval is still occurring).
>>>>>
>>>>> I would lean towards returning the Map<K, V> value, however the next
>>>>> point could change that.
>>>>>
>>>>> 3. Where this method should live.  Right now this method is on the
>>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>>> efficient as possible we really need a Streaming API so that we don't
>>>>> have multiple copies of data in memory at multiple places at the same
>>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>>> have a different API for remote vs embedded.
>>>> Hmmm, I've been thinking about the same thing for Java 8 API (see the proposal wiki), and I think returning a Map limits things. On Java 8, I'm considering switching `evalAll` from returning a Map of CompletableFutures to a Stream... More to come next week.
>>>>
>>>> Now, for the current version, I think Map is limiting for both remote and embedded, because even for embedded, retrieving an entry might be a remote operation, so that forces you to wait for all of the entries to be available before you can return the map. Same thing for Remote.
>>>>
>>>> What about returning an Iterator? That could be lazy and more suited for the use case? If you need to return a Map, e.g. to implement JCache.getAll(), you can wait for all and construct a map.
>>>
>>> What's the expected usecase? I would expect that users want to just
>>> retrieve few keys at once, without any fancy lazy
>>> wait-for-completion-and-do-after stuff. Just imagine that you want to
>>> pass the two values into another method, Iterator/Stream API wouldn't
>>> help you with that much.
>>>
>>> It seems to me that it is very useful to have multi get with Map<K, V>
>>> as return type in the API - we don't want to force constructs like:
>>>
>>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>>> for (; it.hasNext();) {
>>>      Map.Entry<K, V> entry = it.next();
>>>      if (entry.getKey().equals(key1)) value1 = entry.getValue();
>>>      else if (entry.getKey().equals(key2)) value2 = entry.getValue();
>>>      else if (entry.getKey().equals(key3)) value3 = entry.getValue();
>>> }
>>>
>>> or let them flip it into the map manually (HashMap does not have
>>> constructor accepting iterator)
>>>
>>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>>> Map<K, V> map = new HashMap<>();
>>> for (; it.hasNext();) {
>>>      Map.Entry<K, V> entry = it.next();
>>>      map.put(entry.getKey(), entry.getValue());
>>> }
>>>
>>>
>>>>
>>>> Now, whether to have a separate API for this, since we have Java 8 API coming up, I'd not worry too much at this point.
>>>
>>> It seems to me that with 8, we're going to have complicated lazy
>>> omnipotent API, and several facades simplifying that into Map, JCache or
>>> whatever. So, let's provide the simplistic API with Map now, and the
>>> lazy versions can get into the lazy API as soon as we'll see how the
>>> other methods will look like.
>>>
>>> Radim
>>>
>>>>
>>>> Cheers,
>>>>
>>>>> Let me know what you guys think.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> - Will
>>>>>
>>>>> [1] https://issues.jboss.org/browse/ISPN-2183
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>> Galder Zamarre?o
>>>> galder at redhat.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>> --
>>> Radim Vansa <rvansa at redhat.com>
>>> JBoss Performance Team
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From rvansa at redhat.com  Thu Mar 26 10:46:24 2015
From: rvansa at redhat.com (Radim Vansa)
Date: Thu, 26 Mar 2015 15:46:24 +0100
Subject: [infinispan-dev] Multi get API
In-Reply-To: <CAFm4XO2zT2Q=qzxqhyvYsbmBYLFpGb5ePx7eyqHdSrL9+t1R0w@mail.gmail.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>	<9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>	<5513D990.7000600@redhat.com>	<CAFm4XO1=MNLpf+u6OMZX0Y5_1T6q=j1_3fFehOSazxNHAfnXMg@mail.gmail.com>	<CA+YCuUMmL=jrk-SD5JFhdChrJ=0oA64QA8ftbaqb5K0BfMdB1A@mail.gmail.com>
	<CAFm4XO2zT2Q=qzxqhyvYsbmBYLFpGb5ePx7eyqHdSrL9+t1R0w@mail.gmail.com>
Message-ID: <55141BC0.1050303@redhat.com>

On 03/26/2015 03:00 PM, Sanne Grinovero wrote:
> On 26 March 2015 at 13:43, William Burns <mudokonman at gmail.com> wrote:
>> On Thu, Mar 26, 2015 at 8:50 AM, Sanne Grinovero <sanne at infinispan.org> wrote:
>>> In terms of naming, I'd prefer _getMulti_.
>>> My reasoning is it's nice if has a common prefix with _get_ as that
>>> helps someone exploring the API from an IDE, giving better assistance
>>> to people learning.
>>>
>>> getAll is misleading, as is getMany or get getAllOf as they imply I'm
>>> searching for something based on a pattern but you're expecting the
>>> keys and returning tuples key/value.
>> I can understand your point, however since JCache already defined this
>> as getAll I can't see but using that name.
> Ok that makes it win a bit of points, and at least it starts with _get_.

Was there any discussion regarding this during JSR-107 development? I 
wonder why have they chosen such misleading name. Galder?

>
>
>>> I like the idea of returning a Map, as you can actually make the map
>>> behave like a Future: the Map instance could be returned right away,
>>> but retrieving of some entries would be blocking until they are
>>> retrieved. You wouldn't need to block on having all entries, just the
>>> ones being actively asked for. In other words, it's better than
>>> returning a Future<Map>.
>>> This should invalidate some of the previous concerns about using Map ?
>> I had thought about this briefly, but didn't give it much though since
>> it makes things quite complicated (interceptor stack and tx context I
>> am looking at you).  I would say to do this later if we wanted, since
>> the API doesn't have to be different (other than the fact that people
>> should start possibly expecting CacheExceptions from calling Map.get
>> and other methods).
>>
>>> @William I didn't understand your concern on upper and lower bounds.
>>> The returned types should be coupled to the Cache types; being able to
>> The return type is coupled to the Cache types, this was about the
>> input type defined on the Set.
>>
>>> pass Object might be nice but let's make sure no casting is required
>>> on the client? Could you make some examples of your alternative
>>> proposal?
>> With Set<?> the client would never have to do any casting.
> I'm confused. Not sure which Set you're talking about. And remember
> that if you're being very liberal with the promise of returned types,
> how is the implementation going to handle the guarantee?
>
> For example, what's going to happen if the requested objects don't
> match the user expected type? The API shouldn't make promises it can't
> maintain.

The cache.get(k1) should return object stored under key k2 such that

k1.hashCode() == k2.hashCode() && k1.equals(k2)

Nothing is said about k1's type (though, Object.equals() states that 
equals should always form symmetric relation).
The contract was once set for all java collections, let's stick to that.

Radim

>
> Thanks,
> Sanne
>
>>> On 26 March 2015 at 10:04, Radim Vansa <rvansa at redhat.com> wrote:
>>>> On 03/26/2015 10:28 AM, Galder Zamarre?o wrote:
>>>>> Comments inline:
>>>>>
>>>>>> On 24 Mar 2015, at 14:22, William Burns <mudokonman at gmail.com> wrote:
>>>>>>
>>>>>> I am nearing completion of the new multi get command [1], allowing for
>>>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>>>> again to Radim for a lot of the initial work.
>>>>>>
>>>>>> In doing so though I want to make sure I get feedback on how we want
>>>>>> the API to actually look, since this is much more difficult to change
>>>>>> down the line.
>>>>>>
>>>>>> There are a few things in particular I wanted to discuss.
>>>>>>
>>>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>>>> really getting a subset.  So I am thinking the possibilities at this
>>>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>>>> the last one (getMany).  I am open to any suggestions though.
>>>>> JSR-107 has getAll(Set), which I think it's a bit confusing too naming wise.
>>>>>
>>>>> getMany or getAllOf sound better to me.
>>>>>
>>>>>> 2. What should the return value be for this method.  Currently this
>>>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>>>> the local transactional context and is pretty nice and clean for end
>>>>>> users.
>>>>>>
>>>>>> The other idea is to use a streaming API possibly returning an
>>>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>>>> as intuitive to me, but opens us up for more efficient usage
>>>>>> especially when there may be a lot of keys passed in (also results can
>>>>>> be processed concurrently while retrieval is still occurring).
>>>>>>
>>>>>> I would lean towards returning the Map<K, V> value, however the next
>>>>>> point could change that.
>>>>>>
>>>>>> 3. Where this method should live.  Right now this method is on the
>>>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>>>> efficient as possible we really need a Streaming API so that we don't
>>>>>> have multiple copies of data in memory at multiple places at the same
>>>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>>>> have a different API for remote vs embedded.
>>>>> Hmmm, I've been thinking about the same thing for Java 8 API (see the proposal wiki), and I think returning a Map limits things. On Java 8, I'm considering switching `evalAll` from returning a Map of CompletableFutures to a Stream... More to come next week.
>>>>>
>>>>> Now, for the current version, I think Map is limiting for both remote and embedded, because even for embedded, retrieving an entry might be a remote operation, so that forces you to wait for all of the entries to be available before you can return the map. Same thing for Remote.
>>>>>
>>>>> What about returning an Iterator? That could be lazy and more suited for the use case? If you need to return a Map, e.g. to implement JCache.getAll(), you can wait for all and construct a map.
>>>> What's the expected usecase? I would expect that users want to just
>>>> retrieve few keys at once, without any fancy lazy
>>>> wait-for-completion-and-do-after stuff. Just imagine that you want to
>>>> pass the two values into another method, Iterator/Stream API wouldn't
>>>> help you with that much.
>>>>
>>>> It seems to me that it is very useful to have multi get with Map<K, V>
>>>> as return type in the API - we don't want to force constructs like:
>>>>
>>>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>>>> for (; it.hasNext();) {
>>>>       Map.Entry<K, V> entry = it.next();
>>>>       if (entry.getKey().equals(key1)) value1 = entry.getValue();
>>>>       else if (entry.getKey().equals(key2)) value2 = entry.getValue();
>>>>       else if (entry.getKey().equals(key3)) value3 = entry.getValue();
>>>> }
>>>>
>>>> or let them flip it into the map manually (HashMap does not have
>>>> constructor accepting iterator)
>>>>
>>>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>>>> Map<K, V> map = new HashMap<>();
>>>> for (; it.hasNext();) {
>>>>       Map.Entry<K, V> entry = it.next();
>>>>       map.put(entry.getKey(), entry.getValue());
>>>> }
>>>>
>>>>
>>>>> Now, whether to have a separate API for this, since we have Java 8 API coming up, I'd not worry too much at this point.
>>>> It seems to me that with 8, we're going to have complicated lazy
>>>> omnipotent API, and several facades simplifying that into Map, JCache or
>>>> whatever. So, let's provide the simplistic API with Map now, and the
>>>> lazy versions can get into the lazy API as soon as we'll see how the
>>>> other methods will look like.
>>>>
>>>> Radim
>>>>
>>>>> Cheers,
>>>>>
>>>>>> Let me know what you guys think.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> - Will
>>>>>>
>>>>>> [1] https://issues.jboss.org/browse/ISPN-2183
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> --
>>>>> Galder Zamarre?o
>>>>> galder at redhat.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>> Radim Vansa <rvansa at redhat.com>
>>>> JBoss Performance Team
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From sanne at infinispan.org  Thu Mar 26 11:27:03 2015
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 26 Mar 2015 15:27:03 +0000
Subject: [infinispan-dev] Infinispan Directory Provider moving from
 Hibernate Search, suggestions on names
In-Reply-To: <550FE91E.5050903@redhat.com>
References: <CAH8Ud1Eyi==TLYLQLgF1gURH6nrJcCF51EFxv4S2sef8_a43jA@mail.gmail.com>
	<550C2A16.9040808@redhat.com>
	<CAH8Ud1Hndefw0MtqH1PCj8ESJ0rw790+nfMZdMM4fs3K3-xg+g@mail.gmail.com>
	<550FE91E.5050903@redhat.com>
Message-ID: <CAFm4XO1OpVo_iLJBG2RezEUGxvKCkjZw4xh5Hh9YDyZaSkNO8A@mail.gmail.com>

On 23 March 2015 at 10:21, Tristan Tarrant <ttarrant at redhat.com> wrote:
> On 23/03/2015 10:21, Gustavo Fernandes wrote:
>> On Fri, Mar 20, 2015 at 2:09 PM, Tristan Tarrant <ttarrant at redhat.com
>> <mailto:ttarrant at redhat.com>> wrote:
>>
>>     Hi Gustavo,
>>
>>     org.infinispan.directory-provider doesn't actually tell me much, it
>>     should be org.infinispan.lucene-directory-provider.
>>
>>
>> org.infinispan.lucene-directory-provider gives the impression it is
>> about the Lucene Directory packaging, without the hibernate parts.
>>
>> What about org.infinispan.hibernate-search.directory-provider?
> +1

I like that, but:
 - you don't have tons of Hibernate Search extensions
 - we're moving away from exposing "DirectoryProvider" as a user
facing concept (trying to simplify)

what about

 org.infinispan.hibernate-search-extensions

Sanne

From rdouglas at umd.edu  Thu Mar 26 12:51:42 2015
From: rdouglas at umd.edu (Raymond Rennell Douglass III)
Date: Thu, 26 Mar 2015 16:51:42 +0000
Subject: [infinispan-dev] Grouping lucene query on Infinispan
Message-ID: <E31DA3C5470DBE47AF109B7CAA450AD40B0E5FA6@OITMX1006.AD.UMD.EDU>

Hello all,

This is my first time mailing this list, so please let me know if this is not the right place for this.

I use a replicated and/or distributed cache to store some Objects and it is indexed using Infinispan's indexing via Hibernate Search. I'm using Infinispan 7.1.x.

I wanted to use Lucene's GroupingSearch functionality, which I could not find anywhere in the Infinispan or Hibernate Search docs, so I used this: https://docs.jboss.org/infinispan/7.1/apidocs/org/infinispan/query/SearchManager.html#getSearchFactory%28%29
I wrote code like so:
SearchIntegrator searchIntegrator = Search.getSearchManager(cache).getSearchFactory();
IndexReader reader = searchIntegrator.getIndexReaderAccessor().open(ENTITY_CLASSES);
IndexSearcher searcher = new IndexSearcher(reader);
String groupedBy = ...;
GroupingSearch groupingSearch = new GroupingSearch(groupedBy);
TopGroups<BytesRef> result = groupingSearch.search(searcher, query, offset, limit);

I noticed in the javadocs for the getSearchFactory method that it's deprecated (maybe just in Infinispan 7.0?). So I wanted to ask:
1. Is there are better way to perform a grouping search?
2. Will this API be available in future Infinispan versions?

I personally think access to an IndexReader is very helpful for those strange edge cases to run custom Lucene queries.

Thanks,

--
Ray Douglass

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20150326/c98538ba/attachment-0001.html 

From dereed at redhat.com  Thu Mar 26 13:09:48 2015
From: dereed at redhat.com (Dennis Reed)
Date: Thu, 26 Mar 2015 12:09:48 -0500
Subject: [infinispan-dev] Multi get API
In-Reply-To: <55141BC0.1050303@redhat.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>	<9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>	<5513D990.7000600@redhat.com>	<CAFm4XO1=MNLpf+u6OMZX0Y5_1T6q=j1_3fFehOSazxNHAfnXMg@mail.gmail.com>	<CA+YCuUMmL=jrk-SD5JFhdChrJ=0oA64QA8ftbaqb5K0BfMdB1A@mail.gmail.com>	<CAFm4XO2zT2Q=qzxqhyvYsbmBYLFpGb5ePx7eyqHdSrL9+t1R0w@mail.gmail.com>
	<55141BC0.1050303@redhat.com>
Message-ID: <55143D5C.9000606@redhat.com>

On 03/26/2015 09:46 AM, Radim Vansa wrote:
> On 03/26/2015 03:00 PM, Sanne Grinovero wrote:
>> On 26 March 2015 at 13:43, William Burns <mudokonman at gmail.com> wrote:
>>> On Thu, Mar 26, 2015 at 8:50 AM, Sanne Grinovero <sanne at infinispan.org> wrote:
>>>> In terms of naming, I'd prefer _getMulti_.
>>>> My reasoning is it's nice if has a common prefix with _get_ as that
>>>> helps someone exploring the API from an IDE, giving better assistance
>>>> to people learning.
>>>>
>>>> getAll is misleading, as is getMany or get getAllOf as they imply I'm
>>>> searching for something based on a pattern but you're expecting the
>>>> keys and returning tuples key/value.
>>> I can understand your point, however since JCache already defined this
>>> as getAll I can't see but using that name.
>> Ok that makes it win a bit of points, and at least it starts with _get_.
>
> Was there any discussion regarding this during JSR-107 development? I
> wonder why have they chosen such misleading name. Galder?

I don't understand what's misleading about the name getAll.

getAll(keys) -- gets all the entries referenced by those keys.

This is a commonly used naming pattern (i.e. Map.putAll), so -1000 to 
making up our own different name.

-Dennis

From sanne at infinispan.org  Thu Mar 26 15:49:58 2015
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 26 Mar 2015 19:49:58 +0000
Subject: [infinispan-dev] Grouping lucene query on Infinispan
In-Reply-To: <E31DA3C5470DBE47AF109B7CAA450AD40B0E5FA6@OITMX1006.AD.UMD.EDU>
References: <E31DA3C5470DBE47AF109B7CAA450AD40B0E5FA6@OITMX1006.AD.UMD.EDU>
Message-ID: <CAFm4XO3y3t_aPc+EUWjKa-V7Jd_F3=OEd5yGODo0jvtbbeboRQ@mail.gmail.com>

Hi Ray,
yes it's the right place for such questions.

This is a great example of why we expose the IndexReader so far: so
that power users can do more than what we expose over simplified APIs,
be it because the feature really need low level access, or simply
because - like in this case - we didn't add a nicer/simpler API for
the feature yet.

The method getSearchFactory() was deprecated as we're trying to
simplify the API, but I'm realizing now that this would also prevent
accessing the IndexReader(s), and there definitely will always be good
reasons to need that. Another reason for deprecating accessing the
underlying search engine, is that we have plans for aggressive
automatic index sharding, and it gets complex for users if we don't
provide a good abstraction.

But accessing an IndexReader for Grouping is a very strong case, I
agree we should not take that away and created:
 - https://issues.jboss.org/browse/ISPN-5332

Also - while there will always be good reasons to access the
IndexReader - some experiments have started in Hibernate Search to
provide an higher level API to the Grouping API
 - https://github.com/hibernate/hibernate-search/pull/791

The API will have to change (for one because the method name is not
matching the returned type anymore), and we might not want to expose
the SearchIntegrator but we will expose IndexReader instances in some
way.

Thanks for your feedback! Glad it came before we actually removed it.
Sanne

On 26 March 2015 at 16:51, Raymond Rennell Douglass III
<rdouglas at umd.edu> wrote:
> Hello all,
>
>
>
> This is my first time mailing this list, so please let me know if this is
> not the right place for this.
>
>
>
> I use a replicated and/or distributed cache to store some Objects and it is
> indexed using Infinispan?s indexing via Hibernate Search. I?m using
> Infinispan 7.1.x.
>
>
>
> I wanted to use Lucene?s GroupingSearch functionality, which I could not
> find anywhere in the Infinispan or Hibernate Search docs, so I used this:
> https://docs.jboss.org/infinispan/7.1/apidocs/org/infinispan/query/SearchManager.html#getSearchFactory%28%29
>
> I wrote code like so:
>
> SearchIntegrator searchIntegrator =
> Search.getSearchManager(cache).getSearchFactory();
>
> IndexReader reader =
> searchIntegrator.getIndexReaderAccessor().open(ENTITY_CLASSES);
>
> IndexSearcher searcher = new IndexSearcher(reader);
>
> String groupedBy = ...;
>
> GroupingSearch groupingSearch = new GroupingSearch(groupedBy);
>
> TopGroups<BytesRef> result = groupingSearch.search(searcher, query, offset,
> limit);
>
>
>
> I noticed in the javadocs for the getSearchFactory method that it?s
> deprecated (maybe just in Infinispan 7.0?). So I wanted to ask:
>
> 1. Is there are better way to perform a grouping search?
>
> 2. Will this API be available in future Infinispan versions?
>
>
>
> I personally think access to an IndexReader is very helpful for those
> strange edge cases to run custom Lucene queries.
>
>
>
> Thanks,
>
>
>
> --
>
> Ray Douglass
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From rory.odonnell at oracle.com  Tue Mar 31 04:21:12 2015
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Tue, 31 Mar 2015 09:21:12 +0100
Subject: [infinispan-dev] Early Access builds for JDK 9 b55 and JDK 8u60 b08
 are available on java.net
Message-ID: <551A58F8.6030608@oracle.com>


Hi Galder,

Early Access build for JDK 9 b55 <https://jdk9.java.net/download/> 
available on java.net, summary of  changes are listed here 
<http://www.java.net/download/jdk9/changes/jdk9-b55.html>

Early Access build for JDK 8u60 b08 <http://jdk8.java.net/download.html> 
is available on java.net, summary of changes are listed here. 
<http://www.java.net/download/jdk8u60/changes/jdk8u60-b08.html>

Rgds,Rory

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20150331/5bbeb457/attachment.html 

From galder at redhat.com  Tue Mar 31 11:25:41 2015
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Tue, 31 Mar 2015 17:25:41 +0200
Subject: [infinispan-dev] Multi get API
In-Reply-To: <55141BC0.1050303@redhat.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>
	<5513D990.7000600@redhat.com>
	<CAFm4XO1=MNLpf+u6OMZX0Y5_1T6q=j1_3fFehOSazxNHAfnXMg@mail.gmail.com>
	<CA+YCuUMmL=jrk-SD5JFhdChrJ=0oA64QA8ftbaqb5K0BfMdB1A@mail.gmail.com>
	<CAFm4XO2zT2Q=qzxqhyvYsbmBYLFpGb5ePx7eyqHdSrL9+t1R0w@mail.gmail.com>
	<55141BC0.1050303@redhat.com>
Message-ID: <DBCC89B2-D5ED-4B92-B993-76D4D77697AA@redhat.com>


> On 26 Mar 2015, at 15:46, Radim Vansa <rvansa at redhat.com> wrote:
> 
> On 03/26/2015 03:00 PM, Sanne Grinovero wrote:
>> On 26 March 2015 at 13:43, William Burns <mudokonman at gmail.com> wrote:
>>> On Thu, Mar 26, 2015 at 8:50 AM, Sanne Grinovero <sanne at infinispan.org> wrote:
>>>> In terms of naming, I'd prefer _getMulti_.
>>>> My reasoning is it's nice if has a common prefix with _get_ as that
>>>> helps someone exploring the API from an IDE, giving better assistance
>>>> to people learning.
>>>> 
>>>> getAll is misleading, as is getMany or get getAllOf as they imply I'm
>>>> searching for something based on a pattern but you're expecting the
>>>> keys and returning tuples key/value.
>>> I can understand your point, however since JCache already defined this
>>> as getAll I can't see but using that name.
>> Ok that makes it win a bit of points, and at least it starts with _get_.
> 
> Was there any discussion regarding this during JSR-107 development? I 
> wonder why have they chosen such misleading name. Galder?

Dunno, never saw it discussed, but I supposed they chose that name cos of Map.putAll

I think it's the wrong name, getMany, getMulti are better IMO, regardless of what JCache/java.util.Map do.

Also, see what they did with CacheWriter and CacheLoader, they're no symmetrical... and we followed the same pattern in our persistence :|

> 
>> 
>> 
>>>> I like the idea of returning a Map, as you can actually make the map
>>>> behave like a Future: the Map instance could be returned right away,
>>>> but retrieving of some entries would be blocking until they are
>>>> retrieved. You wouldn't need to block on having all entries, just the
>>>> ones being actively asked for. In other words, it's better than
>>>> returning a Future<Map>.
>>>> This should invalidate some of the previous concerns about using Map ?
>>> I had thought about this briefly, but didn't give it much though since
>>> it makes things quite complicated (interceptor stack and tx context I
>>> am looking at you).  I would say to do this later if we wanted, since
>>> the API doesn't have to be different (other than the fact that people
>>> should start possibly expecting CacheExceptions from calling Map.get
>>> and other methods).
>>> 
>>>> @William I didn't understand your concern on upper and lower bounds.
>>>> The returned types should be coupled to the Cache types; being able to
>>> The return type is coupled to the Cache types, this was about the
>>> input type defined on the Set.
>>> 
>>>> pass Object might be nice but let's make sure no casting is required
>>>> on the client? Could you make some examples of your alternative
>>>> proposal?
>>> With Set<?> the client would never have to do any casting.
>> I'm confused. Not sure which Set you're talking about. And remember
>> that if you're being very liberal with the promise of returned types,
>> how is the implementation going to handle the guarantee?
>> 
>> For example, what's going to happen if the requested objects don't
>> match the user expected type? The API shouldn't make promises it can't
>> maintain.
> 
> The cache.get(k1) should return object stored under key k2 such that
> 
> k1.hashCode() == k2.hashCode() && k1.equals(k2)
> 
> Nothing is said about k1's type (though, Object.equals() states that 
> equals should always form symmetric relation).
> The contract was once set for all java collections, let's stick to that.
> 
> Radim
> 
>> 
>> Thanks,
>> Sanne
>> 
>>>> On 26 March 2015 at 10:04, Radim Vansa <rvansa at redhat.com> wrote:
>>>>> On 03/26/2015 10:28 AM, Galder Zamarre?o wrote:
>>>>>> Comments inline:
>>>>>> 
>>>>>>> On 24 Mar 2015, at 14:22, William Burns <mudokonman at gmail.com> wrote:
>>>>>>> 
>>>>>>> I am nearing completion of the new multi get command [1], allowing for
>>>>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>>>>> again to Radim for a lot of the initial work.
>>>>>>> 
>>>>>>> In doing so though I want to make sure I get feedback on how we want
>>>>>>> the API to actually look, since this is much more difficult to change
>>>>>>> down the line.
>>>>>>> 
>>>>>>> There are a few things in particular I wanted to discuss.
>>>>>>> 
>>>>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>>>>> really getting a subset.  So I am thinking the possibilities at this
>>>>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>>>>> the last one (getMany).  I am open to any suggestions though.
>>>>>> JSR-107 has getAll(Set), which I think it's a bit confusing too naming wise.
>>>>>> 
>>>>>> getMany or getAllOf sound better to me.
>>>>>> 
>>>>>>> 2. What should the return value be for this method.  Currently this
>>>>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>>>>> the local transactional context and is pretty nice and clean for end
>>>>>>> users.
>>>>>>> 
>>>>>>> The other idea is to use a streaming API possibly returning an
>>>>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>>>>> as intuitive to me, but opens us up for more efficient usage
>>>>>>> especially when there may be a lot of keys passed in (also results can
>>>>>>> be processed concurrently while retrieval is still occurring).
>>>>>>> 
>>>>>>> I would lean towards returning the Map<K, V> value, however the next
>>>>>>> point could change that.
>>>>>>> 
>>>>>>> 3. Where this method should live.  Right now this method is on the
>>>>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>>>>> efficient as possible we really need a Streaming API so that we don't
>>>>>>> have multiple copies of data in memory at multiple places at the same
>>>>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>>>>> have a different API for remote vs embedded.
>>>>>> Hmmm, I've been thinking about the same thing for Java 8 API (see the proposal wiki), and I think returning a Map limits things. On Java 8, I'm considering switching `evalAll` from returning a Map of CompletableFutures to a Stream... More to come next week.
>>>>>> 
>>>>>> Now, for the current version, I think Map is limiting for both remote and embedded, because even for embedded, retrieving an entry might be a remote operation, so that forces you to wait for all of the entries to be available before you can return the map. Same thing for Remote.
>>>>>> 
>>>>>> What about returning an Iterator? That could be lazy and more suited for the use case? If you need to return a Map, e.g. to implement JCache.getAll(), you can wait for all and construct a map.
>>>>> What's the expected usecase? I would expect that users want to just
>>>>> retrieve few keys at once, without any fancy lazy
>>>>> wait-for-completion-and-do-after stuff. Just imagine that you want to
>>>>> pass the two values into another method, Iterator/Stream API wouldn't
>>>>> help you with that much.
>>>>> 
>>>>> It seems to me that it is very useful to have multi get with Map<K, V>
>>>>> as return type in the API - we don't want to force constructs like:
>>>>> 
>>>>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>>>>> for (; it.hasNext();) {
>>>>>      Map.Entry<K, V> entry = it.next();
>>>>>      if (entry.getKey().equals(key1)) value1 = entry.getValue();
>>>>>      else if (entry.getKey().equals(key2)) value2 = entry.getValue();
>>>>>      else if (entry.getKey().equals(key3)) value3 = entry.getValue();
>>>>> }
>>>>> 
>>>>> or let them flip it into the map manually (HashMap does not have
>>>>> constructor accepting iterator)
>>>>> 
>>>>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>>>>> Map<K, V> map = new HashMap<>();
>>>>> for (; it.hasNext();) {
>>>>>      Map.Entry<K, V> entry = it.next();
>>>>>      map.put(entry.getKey(), entry.getValue());
>>>>> }
>>>>> 
>>>>> 
>>>>>> Now, whether to have a separate API for this, since we have Java 8 API coming up, I'd not worry too much at this point.
>>>>> It seems to me that with 8, we're going to have complicated lazy
>>>>> omnipotent API, and several facades simplifying that into Map, JCache or
>>>>> whatever. So, let's provide the simplistic API with Map now, and the
>>>>> lazy versions can get into the lazy API as soon as we'll see how the
>>>>> other methods will look like.
>>>>> 
>>>>> Radim
>>>>> 
>>>>>> Cheers,
>>>>>> 
>>>>>>> Let me know what you guys think.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> 
>>>>>>> - Will
>>>>>>> 
>>>>>>> [1] https://issues.jboss.org/browse/ISPN-2183
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> --
>>>>>> Galder Zamarre?o
>>>>>> galder at redhat.com
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> 
>>>>> --
>>>>> Radim Vansa <rvansa at redhat.com>
>>>>> JBoss Performance Team
>>>>> 
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> -- 
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarre?o
galder at redhat.com






From sanne at infinispan.org  Tue Mar 31 14:41:40 2015
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 31 Mar 2015 19:41:40 +0100
Subject: [infinispan-dev] Multi get API
In-Reply-To: <DBCC89B2-D5ED-4B92-B993-76D4D77697AA@redhat.com>
References: <CA+YCuUNfr8Lgh4yb+7BQhOVSGHNeSvkWBg5Xyp80f-TdUgaqqA@mail.gmail.com>
	<9E0BA454-9A6F-4E51-BDCD-AA8392ABF987@redhat.com>
	<5513D990.7000600@redhat.com>
	<CAFm4XO1=MNLpf+u6OMZX0Y5_1T6q=j1_3fFehOSazxNHAfnXMg@mail.gmail.com>
	<CA+YCuUMmL=jrk-SD5JFhdChrJ=0oA64QA8ftbaqb5K0BfMdB1A@mail.gmail.com>
	<CAFm4XO2zT2Q=qzxqhyvYsbmBYLFpGb5ePx7eyqHdSrL9+t1R0w@mail.gmail.com>
	<55141BC0.1050303@redhat.com>
	<DBCC89B2-D5ED-4B92-B993-76D4D77697AA@redhat.com>
Message-ID: <CAFm4XO3m44fp_81352QzdZnYukBj_iCQ_2pi-aoH4Sjjsr8RWg@mail.gmail.com>

On 31 March 2015 at 16:25, Galder Zamarre?o <galder at redhat.com> wrote:
>
>> On 26 Mar 2015, at 15:46, Radim Vansa <rvansa at redhat.com> wrote:
>>
>> On 03/26/2015 03:00 PM, Sanne Grinovero wrote:
>>> On 26 March 2015 at 13:43, William Burns <mudokonman at gmail.com> wrote:
>>>> On Thu, Mar 26, 2015 at 8:50 AM, Sanne Grinovero <sanne at infinispan.org> wrote:
>>>>> In terms of naming, I'd prefer _getMulti_.
>>>>> My reasoning is it's nice if has a common prefix with _get_ as that
>>>>> helps someone exploring the API from an IDE, giving better assistance
>>>>> to people learning.
>>>>>
>>>>> getAll is misleading, as is getMany or get getAllOf as they imply I'm
>>>>> searching for something based on a pattern but you're expecting the
>>>>> keys and returning tuples key/value.
>>>> I can understand your point, however since JCache already defined this
>>>> as getAll I can't see but using that name.
>>> Ok that makes it win a bit of points, and at least it starts with _get_.
>>
>> Was there any discussion regarding this during JSR-107 development? I
>> wonder why have they chosen such misleading name. Galder?
>
> Dunno, never saw it discussed, but I supposed they chose that name cos of Map.putAll
>
> I think it's the wrong name, getMany, getMulti are better IMO, regardless of what JCache/java.util.Map do.
>
> Also, see what they did with CacheWriter and CacheLoader, they're no symmetrical... and we followed the same pattern in our persistence :|

My initial assumption was that we'd use a varargs parameter to list
the keys to be fetched; in that case getAll() could have been invoked
with zero parameters, and it would look like as it should return all
entries of the grid.

I guess having now a Set for the keys makes the getAll() not too bad.

@William +1 to not implement the future-like behaviour (lazy loaded
HashMap) in the first step. I wasn't asking for that, just thinking
ahead about the merits of returning such a type.

Sanne

>
>>
>>>
>>>
>>>>> I like the idea of returning a Map, as you can actually make the map
>>>>> behave like a Future: the Map instance could be returned right away,
>>>>> but retrieving of some entries would be blocking until they are
>>>>> retrieved. You wouldn't need to block on having all entries, just the
>>>>> ones being actively asked for. In other words, it's better than
>>>>> returning a Future<Map>.
>>>>> This should invalidate some of the previous concerns about using Map ?
>>>> I had thought about this briefly, but didn't give it much though since
>>>> it makes things quite complicated (interceptor stack and tx context I
>>>> am looking at you).  I would say to do this later if we wanted, since
>>>> the API doesn't have to be different (other than the fact that people
>>>> should start possibly expecting CacheExceptions from calling Map.get
>>>> and other methods).
>>>>
>>>>> @William I didn't understand your concern on upper and lower bounds.
>>>>> The returned types should be coupled to the Cache types; being able to
>>>> The return type is coupled to the Cache types, this was about the
>>>> input type defined on the Set.
>>>>
>>>>> pass Object might be nice but let's make sure no casting is required
>>>>> on the client? Could you make some examples of your alternative
>>>>> proposal?
>>>> With Set<?> the client would never have to do any casting.
>>> I'm confused. Not sure which Set you're talking about. And remember
>>> that if you're being very liberal with the promise of returned types,
>>> how is the implementation going to handle the guarantee?
>>>
>>> For example, what's going to happen if the requested objects don't
>>> match the user expected type? The API shouldn't make promises it can't
>>> maintain.
>>
>> The cache.get(k1) should return object stored under key k2 such that
>>
>> k1.hashCode() == k2.hashCode() && k1.equals(k2)
>>
>> Nothing is said about k1's type (though, Object.equals() states that
>> equals should always form symmetric relation).
>> The contract was once set for all java collections, let's stick to that.
>>
>> Radim
>>
>>>
>>> Thanks,
>>> Sanne
>>>
>>>>> On 26 March 2015 at 10:04, Radim Vansa <rvansa at redhat.com> wrote:
>>>>>> On 03/26/2015 10:28 AM, Galder Zamarre?o wrote:
>>>>>>> Comments inline:
>>>>>>>
>>>>>>>> On 24 Mar 2015, at 14:22, William Burns <mudokonman at gmail.com> wrote:
>>>>>>>>
>>>>>>>> I am nearing completion of the new multi get command [1], allowing for
>>>>>>>> more efficient retrieval of multiple keys at the same time.  Thanks
>>>>>>>> again to Radim for a lot of the initial work.
>>>>>>>>
>>>>>>>> In doing so though I want to make sure I get feedback on how we want
>>>>>>>> the API to actually look, since this is much more difficult to change
>>>>>>>> down the line.
>>>>>>>>
>>>>>>>> There are a few things in particular I wanted to discuss.
>>>>>>>>
>>>>>>>> 1. The actual name of the method.  The main suggestions I have seen
>>>>>>>> are getAll or getMany.  I do like the naming of the former, however it
>>>>>>>> seems a bit misleading (remember API is getAll(Set) since we are
>>>>>>>> really getting a subset.  So I am thinking the possibilities at this
>>>>>>>> point are getAllOf, getAllFor or getMany.  I am leaning maybe towards
>>>>>>>> the last one (getMany).  I am open to any suggestions though.
>>>>>>> JSR-107 has getAll(Set), which I think it's a bit confusing too naming wise.
>>>>>>>
>>>>>>> getMany or getAllOf sound better to me.
>>>>>>>
>>>>>>>> 2. What should the return value be for this method.  Currently this
>>>>>>>> returns a Map<K, V> which makes sense when we retain these values in
>>>>>>>> the local transactional context and is pretty nice and clean for end
>>>>>>>> users.
>>>>>>>>
>>>>>>>> The other idea is to use a streaming API possibly returning an
>>>>>>>> Iterable<CacheEntry<K, V>>.  The streaming return value doesn't seem
>>>>>>>> as intuitive to me, but opens us up for more efficient usage
>>>>>>>> especially when there may be a lot of keys passed in (also results can
>>>>>>>> be processed concurrently while retrieval is still occurring).
>>>>>>>>
>>>>>>>> I would lean towards returning the Map<K, V> value, however the next
>>>>>>>> point could change that.
>>>>>>>>
>>>>>>>> 3. Where this method should live.  Right now this method is on the
>>>>>>>> BasicCache interface which is a parent of both Cache (embedded) and
>>>>>>>> RemoteCache (remote).  Unfortunately for remote clients to be as
>>>>>>>> efficient as possible we really need a Streaming API so that we don't
>>>>>>>> have multiple copies of data in memory at multiple places at the same
>>>>>>>> time.  For that reason I suggest we use the streaminig API for both or
>>>>>>>> have a different API for remote vs embedded.
>>>>>>> Hmmm, I've been thinking about the same thing for Java 8 API (see the proposal wiki), and I think returning a Map limits things. On Java 8, I'm considering switching `evalAll` from returning a Map of CompletableFutures to a Stream... More to come next week.
>>>>>>>
>>>>>>> Now, for the current version, I think Map is limiting for both remote and embedded, because even for embedded, retrieving an entry might be a remote operation, so that forces you to wait for all of the entries to be available before you can return the map. Same thing for Remote.
>>>>>>>
>>>>>>> What about returning an Iterator? That could be lazy and more suited for the use case? If you need to return a Map, e.g. to implement JCache.getAll(), you can wait for all and construct a map.
>>>>>> What's the expected usecase? I would expect that users want to just
>>>>>> retrieve few keys at once, without any fancy lazy
>>>>>> wait-for-completion-and-do-after stuff. Just imagine that you want to
>>>>>> pass the two values into another method, Iterator/Stream API wouldn't
>>>>>> help you with that much.
>>>>>>
>>>>>> It seems to me that it is very useful to have multi get with Map<K, V>
>>>>>> as return type in the API - we don't want to force constructs like:
>>>>>>
>>>>>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>>>>>> for (; it.hasNext();) {
>>>>>>      Map.Entry<K, V> entry = it.next();
>>>>>>      if (entry.getKey().equals(key1)) value1 = entry.getValue();
>>>>>>      else if (entry.getKey().equals(key2)) value2 = entry.getValue();
>>>>>>      else if (entry.getKey().equals(key3)) value3 = entry.getValue();
>>>>>> }
>>>>>>
>>>>>> or let them flip it into the map manually (HashMap does not have
>>>>>> constructor accepting iterator)
>>>>>>
>>>>>> Iterator<Map.Entry<K, V>> it = cache.multiGet(key1, key2, key3);
>>>>>> Map<K, V> map = new HashMap<>();
>>>>>> for (; it.hasNext();) {
>>>>>>      Map.Entry<K, V> entry = it.next();
>>>>>>      map.put(entry.getKey(), entry.getValue());
>>>>>> }
>>>>>>
>>>>>>
>>>>>>> Now, whether to have a separate API for this, since we have Java 8 API coming up, I'd not worry too much at this point.
>>>>>> It seems to me that with 8, we're going to have complicated lazy
>>>>>> omnipotent API, and several facades simplifying that into Map, JCache or
>>>>>> whatever. So, let's provide the simplistic API with Map now, and the
>>>>>> lazy versions can get into the lazy API as soon as we'll see how the
>>>>>> other methods will look like.
>>>>>>
>>>>>> Radim
>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>>> Let me know what you guys think.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> - Will
>>>>>>>>
>>>>>>>> [1] https://issues.jboss.org/browse/ISPN-2183
>>>>>>>> _______________________________________________
>>>>>>>> infinispan-dev mailing list
>>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>> --
>>>>>>> Galder Zamarre?o
>>>>>>> galder at redhat.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>> --
>>>>>> Radim Vansa <rvansa at redhat.com>
>>>>>> JBoss Performance Team
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> --
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> --
> Galder Zamarre?o
> galder at redhat.com
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev