From rvansa at redhat.com  Tue Aug  1 03:09:40 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Tue, 1 Aug 2017 09:09:40 +0200
Subject: [infinispan-dev] not sure if this is the place to post a
 question on infinispan
In-Reply-To: <51C4569C9C1C58419D764E855FB0E3090DA80CCD@exchange1-tor.impactmobile.local>
References: <51C4569C9C1C58419D764E855FB0E3090DA80CCD@exchange1-tor.impactmobile.local>
Message-ID: <23fb5d1f-f05f-5693-23f0-e1728a3b8af7@redhat.com>

Hi Wayne,

the proper place to ask for help are the forums [1], this mailing list 
is rather for discussing of development (as the -dev suggests).

I am not sure if I understand your question, though; what do you mean by 
re-building the cache? In invalidation mode, one node inserts the data 
into cache and others wipe the (out-of-date) entry out. The nodes don't 
try to fetch the data from anywhere, this should be done by application 
which does not find the data in cache and therefore loads it from 
external source and then caches it using 
cache.withFlags(Flag.PUT_FROM_EXTERNAL_READ).put(...). This flag causes 
the node to not invalidate remote nodes (as the regular put which 
inserts "updated" data).

Since you reply on the Conflict-Manager thread, is your question anyhow 
related to partition handling? (afaik this is not supported in 
invalidation caches).

Radim

[1] 
https://developer.jboss.org/en/infinispan/content?filterID=contentstatus%5Bpublished%5D~objecttype~objecttype%5Bthread%5D

On 07/31/2017 10:31 PM, Wayne Wang wrote:
> Hi All,
>
> I am not sure if this is the place I could post a question on infinispan.
>
> Basically, I am testing a scenario of invalidation cache in a cluster environment. It looks like the instance that actually modified the object has not problem of re-building the cache after data is updated, but the instances (in the same cluster) receiving the signal to invalidate the cache will indeed invalidate the cache, but can not re-build the cache until cache expiration.
>
> Is this intended design, or there is some wrong in the configuration?
>
> Thanks,
>
> Wayne
>
> -----Original Message-----
> From: infinispan-dev-bounces at lists.jboss.org [mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Sanne Grinovero
> Sent: Monday, July 31, 2017 12:04 PM
> To: infinispan -Dev List
> Subject: Re: [infinispan-dev] Conflict Manager and Partition Handling Blog
>
> Great job! I love to see this improved and being described in detail.
>
> +1 to add some practical examples, as I'm afraid we only notice
> limitations in features like this when thinking about specific use-cases.
>
> The option `REMOVE_ALL` seems sensible for the disposable Cache use case. One question though: if one partition has a defined value for a key, while the other partition has no value (null) for this same key, is it considered a conflict?
> I think you need to clarify if a "null" in a subset of partitions causes the conflict merge to be triggered or not. I think it should:
> for example having the cache use case in mind, an explicit invalidation needs to be propagated safely.
>
> Thanks,
> Sanne
>
>
> On 26 July 2017 at 13:41, Ryan Emerson <remerson at redhat.com> wrote:
>> Hi Galder,
>>
>> Thanks for the feedback.
>>
>> Conflicts are detected by applying a predicate to the returned
>> Map<Address, CacheEntry> for each cache entry. Currently this
>> predicate utilises Stream::distinct (so .equals), to check for
>> multiple versions of an entry. So implementing pluggable strategies
>> for defining a conflict should be trivial :)
>>
>> Good idea about a simple tutorial/demo, I'll look into it when I get a chance.
>>
>> Cheers
>> Ryan
>>
>> ----- Original Message -----
>>> Oh, if we can't find a simple tutorial for it, there's always
>>> https://github.com/infinispan-demos :)
>>>
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>>> On 25 Jul 2017, at 17:11, Galder Zamarre?o <galder at redhat.com> wrote:
>>>>
>>>> One more thing: have you thought how we could have a simple
>>>> tutorial on this feature?
>>>>
>>>> It'd be great to find a simple, reduced, example to show it off :)
>>>>
>>>> Cheers,
>>>> --
>>>> Galder Zamarre?o
>>>> Infinispan, Red Hat
>>>>
>>>>> On 25 Jul 2017, at 16:54, Galder Zamarre?o <galder at redhat.com> wrote:
>>>>>
>>>>> Hey Ryan,
>>>>>
>>>>> Very detailed blog post! Great work on both the post and the
>>>>> feature! :D
>>>>>
>>>>> While reading, the following question came to my mind: how does
>>>>> Infinispan determine there's a conflict? Does it rely on .equals() based equality?
>>>>>
>>>>> A follow up would be: whether in the future this could be pluggable, e.g.
>>>>> when comparing a version field is enough to realise there's a conflict.
>>>>> As opposed of relying in .equals(), if that's what's being used
>>>>> inside :)
>>>>>
>>>>> Cheers,
>>>>> --
>>>>> Galder Zamarre?o
>>>>> Infinispan, Red Hat
>>>>>
>>>>>> On 17 Jul 2017, at 14:16, Ryan Emerson <remerson at redhat.com> wrote:
>>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> Here's a blog post on the introduction of ConflictManager and the
>>>>>> recent changes to partition handling.
>>>>>>
>>>>>> http://blog.infinispan.org/2017/07/conflict-management-and-partit
>>>>>> ion.html
>>>>>>
>>>>>> Cheers
>>>>>> Ryan
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From remerson at redhat.com  Tue Aug  1 05:19:55 2017
From: remerson at redhat.com (Ryan Emerson)
Date: Tue, 1 Aug 2017 05:19:55 -0400 (EDT)
Subject: [infinispan-dev] Conflict Manager and Partition Handling Blog
In-Reply-To: <CAFm4XO24B8wX1uJWmcHQnB+nhoMw2tq9sCKtJQGXKR7poKG+jg@mail.gmail.com>
References: <658199000.28091136.1500293787173.JavaMail.zimbra@redhat.com>
	<1A277579-1486-4125-A879-33EECB81FC29@redhat.com>
	<99A51374-C1E9-41EA-A26E-9E2EE4A7CB2A@redhat.com>
	<F4623ADB-E3F4-426A-86D6-8A9557424DED@redhat.com>
	<73413902.30250862.1501072916488.JavaMail.zimbra@redhat.com>
	<CAFm4XO24B8wX1uJWmcHQnB+nhoMw2tq9sCKtJQGXKR7poKG+jg@mail.gmail.com>
Message-ID: <696710556.31663883.1501579195537.JavaMail.zimbra@redhat.com>

> The option `REMOVE_ALL` seems sensible for the disposable Cache use
> case. One question though: if one partition has a defined value for a
> key, while the other partition has no value (null) for this same key,
> is it considered a conflict?
> I think you need to clarify if a "null" in a subset of partitions
> causes the conflict merge to be triggered or not. I think it should:
> for example having the cache use case in mind, an explicit
> invalidation needs to be propagated safely.

Yes a combination of null/non-null entries is detected as a conflict. So in the use-case you describe, utilising the REMOVE_ALL strategy would result in the cache entry being removed from the cache on merge.

Cheers
Ryan

From sanne at infinispan.org  Tue Aug  1 08:33:23 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Tue, 1 Aug 2017 15:33:23 +0300
Subject: [infinispan-dev] Conflict Manager and Partition Handling Blog
In-Reply-To: <696710556.31663883.1501579195537.JavaMail.zimbra@redhat.com>
References: <658199000.28091136.1500293787173.JavaMail.zimbra@redhat.com>
	<1A277579-1486-4125-A879-33EECB81FC29@redhat.com>
	<99A51374-C1E9-41EA-A26E-9E2EE4A7CB2A@redhat.com>
	<F4623ADB-E3F4-426A-86D6-8A9557424DED@redhat.com>
	<73413902.30250862.1501072916488.JavaMail.zimbra@redhat.com>
	<CAFm4XO24B8wX1uJWmcHQnB+nhoMw2tq9sCKtJQGXKR7poKG+jg@mail.gmail.com>
	<696710556.31663883.1501579195537.JavaMail.zimbra@redhat.com>
Message-ID: <CAFm4XO3jJV_e=F6p0aKyn4MO9y+5ud3w2evV1Y3XZokyPqRWLQ@mail.gmail.com>

On 1 August 2017 at 12:19, Ryan Emerson <remerson at redhat.com> wrote:
>> The option `REMOVE_ALL` seems sensible for the disposable Cache use
>> case. One question though: if one partition has a defined value for a
>> key, while the other partition has no value (null) for this same key,
>> is it considered a conflict?
>> I think you need to clarify if a "null" in a subset of partitions
>> causes the conflict merge to be triggered or not. I think it should:
>> for example having the cache use case in mind, an explicit
>> invalidation needs to be propagated safely.
>
> Yes a combination of null/non-null entries is detected as a conflict. So in the use-case you describe, utilising the REMOVE_ALL strategy would result in the cache entry being removed from the cache on merge.

Thanks, looks great. Would you mind clarifying the docs about this?

Sanne

>
> Cheers
> Ryan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

From Wayne.Wang at impactmobile.com  Tue Aug  1 08:54:24 2017
From: Wayne.Wang at impactmobile.com (Wayne Wang)
Date: Tue, 1 Aug 2017 12:54:24 +0000
Subject: [infinispan-dev] Conflict Manager and Partition Handling Blog
In-Reply-To: <696710556.31663883.1501579195537.JavaMail.zimbra@redhat.com>
References: <658199000.28091136.1500293787173.JavaMail.zimbra@redhat.com>
	<1A277579-1486-4125-A879-33EECB81FC29@redhat.com>
	<99A51374-C1E9-41EA-A26E-9E2EE4A7CB2A@redhat.com>
	<F4623ADB-E3F4-426A-86D6-8A9557424DED@redhat.com>
	<73413902.30250862.1501072916488.JavaMail.zimbra@redhat.com>
	<CAFm4XO24B8wX1uJWmcHQnB+nhoMw2tq9sCKtJQGXKR7poKG+jg@mail.gmail.com>
	<696710556.31663883.1501579195537.JavaMail.zimbra@redhat.com>
Message-ID: <51C4569C9C1C58419D764E855FB0E3090DA80F07@exchange1-tor.impactmobile.local>

Hi Ryan,

I understand that infinispan can be used by many platforms/applications. In addition, I was not able to find more information as to how infinispan cache manager actually work.

What I described is the infinispan embedded in wildfly 10.1.10.Final. This observation happened in a cluster of two wildfly 10.1.0.Final instances

Basically, the console output of the wildfly instance that actually made the modification of an object printed out update statement (expected). Later on, if a user revisit the same object (hotel), the console will not print out any query statement. This is expected since the latest data was cached.

For the other wildfly instance(cluster member), if a user visit the same object (hotel) first time, a query to select for a specific object(hotel) was printed out. This is expected since the cache should be invalidated and query needs to be executed to retrieve latest data.  However, when the user repeatedly revisit the same object (hotel), the query got printed out again. This will happen until a period of time (may be expiration setting).

This observation indicated that the wildfly instance which actually made the modification was able retrieve the latest data and cache it again. But the wildfly instance that receives the signal (to invalidate cache) indeed invalidated the cache, however, it did just that without cache the latest data.

Is this behavior the intended design or some other configuration I need to make to ensure the other cluster members can also cache the latest data?

Thanks,

Wayne


-----Original Message-----
From: infinispan-dev-bounces at lists.jboss.org [mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Ryan Emerson
Sent: Tuesday, August 1, 2017 5:20 AM
To: infinispan -Dev List
Subject: Re: [infinispan-dev] Conflict Manager and Partition Handling Blog

> The option `REMOVE_ALL` seems sensible for the disposable Cache use 
> case. One question though: if one partition has a defined value for a 
> key, while the other partition has no value (null) for this same key, 
> is it considered a conflict?
> I think you need to clarify if a "null" in a subset of partitions 
> causes the conflict merge to be triggered or not. I think it should:
> for example having the cache use case in mind, an explicit 
> invalidation needs to be propagated safely.

Yes a combination of null/non-null entries is detected as a conflict. So in the use-case you describe, utilising the REMOVE_ALL strategy would result in the cache entry being removed from the cache on merge.

Cheers
Ryan
_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev


From remerson at redhat.com  Tue Aug  1 11:10:47 2017
From: remerson at redhat.com (Ryan Emerson)
Date: Tue, 1 Aug 2017 11:10:47 -0400 (EDT)
Subject: [infinispan-dev] Conflict Manager and Partition Handling Blog
In-Reply-To: <CAFm4XO3jJV_e=F6p0aKyn4MO9y+5ud3w2evV1Y3XZokyPqRWLQ@mail.gmail.com>
References: <658199000.28091136.1500293787173.JavaMail.zimbra@redhat.com>
	<1A277579-1486-4125-A879-33EECB81FC29@redhat.com>
	<99A51374-C1E9-41EA-A26E-9E2EE4A7CB2A@redhat.com>
	<F4623ADB-E3F4-426A-86D6-8A9557424DED@redhat.com>
	<73413902.30250862.1501072916488.JavaMail.zimbra@redhat.com>
	<CAFm4XO24B8wX1uJWmcHQnB+nhoMw2tq9sCKtJQGXKR7poKG+jg@mail.gmail.com>
	<696710556.31663883.1501579195537.JavaMail.zimbra@redhat.com>
	<CAFm4XO3jJV_e=F6p0aKyn4MO9y+5ud3w2evV1Y3XZokyPqRWLQ@mail.gmail.com>
Message-ID: <323813020.31743858.1501600247405.JavaMail.zimbra@redhat.com>

No problem, my intention is to issue a PR later this week.

----- Original Message -----
> On 1 August 2017 at 12:19, Ryan Emerson <remerson at redhat.com> wrote:
> >> The option `REMOVE_ALL` seems sensible for the disposable Cache use
> >> case. One question though: if one partition has a defined value for a
> >> key, while the other partition has no value (null) for this same key,
> >> is it considered a conflict?
> >> I think you need to clarify if a "null" in a subset of partitions
> >> causes the conflict merge to be triggered or not. I think it should:
> >> for example having the cache use case in mind, an explicit
> >> invalidation needs to be propagated safely.
> >
> > Yes a combination of null/non-null entries is detected as a conflict. So in
> > the use-case you describe, utilising the REMOVE_ALL strategy would result
> > in the cache entry being removed from the cache on merge.
> 
> Thanks, looks great. Would you mind clarifying the docs about this?
> 
> Sanne
> 
> >
> > Cheers
> > Ryan
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From remerson at redhat.com  Tue Aug  1 11:21:50 2017
From: remerson at redhat.com (Ryan Emerson)
Date: Tue, 1 Aug 2017 11:21:50 -0400 (EDT)
Subject: [infinispan-dev] Conflict Manager and Partition Handling Blog
In-Reply-To: <51C4569C9C1C58419D764E855FB0E3090DA80F07@exchange1-tor.impactmobile.local>
References: <658199000.28091136.1500293787173.JavaMail.zimbra@redhat.com>
	<1A277579-1486-4125-A879-33EECB81FC29@redhat.com>
	<99A51374-C1E9-41EA-A26E-9E2EE4A7CB2A@redhat.com>
	<F4623ADB-E3F4-426A-86D6-8A9557424DED@redhat.com>
	<73413902.30250862.1501072916488.JavaMail.zimbra@redhat.com>
	<CAFm4XO24B8wX1uJWmcHQnB+nhoMw2tq9sCKtJQGXKR7poKG+jg@mail.gmail.com>
	<696710556.31663883.1501579195537.JavaMail.zimbra@redhat.com>
	<51C4569C9C1C58419D764E855FB0E3090DA80F07@exchange1-tor.impactmobile.local>
Message-ID: <831023895.31748158.1501600910790.JavaMail.zimbra@redhat.com>

Hi Wayne,

I see that you have raised your query in the Infinispan forum [1], which is the correct place for these kinds of queries, not this mailing list. This mailing list is for technical discussions regarding the development of Infinispan. Furthermore, in the future please try to avoid hijacking existing threads on mailing lists if they are not directly related to your issue/query. 

[1] https://developer.jboss.org/thread/275678

Thanks
Ryan

----- Original Message -----
> Hi Ryan,
> 
> I understand that infinispan can be used by many platforms/applications. In
> addition, I was not able to find more information as to how infinispan cache
> manager actually work.
> 
> What I described is the infinispan embedded in wildfly 10.1.10.Final. This
> observation happened in a cluster of two wildfly 10.1.0.Final instances
> 
> Basically, the console output of the wildfly instance that actually made the
> modification of an object printed out update statement (expected). Later on,
> if a user revisit the same object (hotel), the console will not print out
> any query statement. This is expected since the latest data was cached.
> 
> For the other wildfly instance(cluster member), if a user visit the same
> object (hotel) first time, a query to select for a specific object(hotel)
> was printed out. This is expected since the cache should be invalidated and
> query needs to be executed to retrieve latest data.  However, when the user
> repeatedly revisit the same object (hotel), the query got printed out again.
> This will happen until a period of time (may be expiration setting).
> 
> This observation indicated that the wildfly instance which actually made the
> modification was able retrieve the latest data and cache it again. But the
> wildfly instance that receives the signal (to invalidate cache) indeed
> invalidated the cache, however, it did just that without cache the latest
> data.
> 
> Is this behavior the intended design or some other configuration I need to
> make to ensure the other cluster members can also cache the latest data?
> 
> Thanks,
> 
> Wayne
> 
> 
> -----Original Message-----
> From: infinispan-dev-bounces at lists.jboss.org
> [mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Ryan Emerson
> Sent: Tuesday, August 1, 2017 5:20 AM
> To: infinispan -Dev List
> Subject: Re: [infinispan-dev] Conflict Manager and Partition Handling Blog
> 
> > The option `REMOVE_ALL` seems sensible for the disposable Cache use
> > case. One question though: if one partition has a defined value for a
> > key, while the other partition has no value (null) for this same key,
> > is it considered a conflict?
> > I think you need to clarify if a "null" in a subset of partitions
> > causes the conflict merge to be triggered or not. I think it should:
> > for example having the cache use case in mind, an explicit
> > invalidation needs to be propagated safely.
> 
> Yes a combination of null/non-null entries is detected as a conflict. So in
> the use-case you describe, utilising the REMOVE_ALL strategy would result in
> the cache entry being removed from the cache on merge.
> 
> Cheers
> Ryan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

From galder at redhat.com  Wed Aug  2 07:02:26 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Wed, 2 Aug 2017 13:02:26 +0200
Subject: [infinispan-dev] Late end invalidation delivery stops put from load
	- ISPN-8114
Message-ID: <CFD9A636-C32F-42A9-B97E-36B6B97EFA8D@redhat.com>

Hey Radim,

Re: https://issues.jboss.org/browse/ISPN-8114

I've been looking at the trace logs of this failure. I've extracted the most interesting parts of this failure into [1].

What happens is that after loading the entries into the cache, the end invalidation message to allow put from loads to succeed does not get executed in time before the put from load is attempted. As a result of this, the put from load does not happen and hence the entry is not loaded into the cache.

The end invalidation message eventually gets through. There's a gap between receiving the JGroups message and the actual execution, but that's due to the delivery mode of the message.

I'm not sure how we should fix this. Options:

1) A thread sleep before loading entries "might work" but for a CI test this could always backfire with the right timing sets.

2) Find a way to hook into the PFLValidator class and only load after we know end invalidation has been received by all nodes.

3) Make end invalidation message sync? This would expensive. Even with async, changing delivery mode might have worked here... but under the right circumstances you could still get the same issue with async.

I'm keen on trying to find a potential solution using 2), but wondered if you have other ideas.

Cheers,

[1] https://gist.github.com/galderz/0bce6dce16de018375e43e25c0cf3913
--
Galder Zamarre?o
Infinispan, Red Hat



From ttarrant at redhat.com  Wed Aug  2 09:45:33 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Wed, 2 Aug 2017 15:45:33 +0200
Subject: [infinispan-dev] URGENT: Master test failures
In-Reply-To: <efd9a783-2c71-be0a-5773-dff799616a70@redhat.com>
References: <efd9a783-2c71-be0a-5773-dff799616a70@redhat.com>
Message-ID: <e9afa27b-289c-211b-2022-661cf6add3b2@redhat.com>

Since we have many open PRs and CI is rebuilding each one on every 
master commit, I'm going to close all feature PRs temporarily so that we 
can move faster.

Tristan


-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From rvansa at redhat.com  Wed Aug  2 12:41:55 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Wed, 2 Aug 2017 18:41:55 +0200
Subject: [infinispan-dev] Late end invalidation delivery stops put from
	load - ISPN-8114
In-Reply-To: <CFD9A636-C32F-42A9-B97E-36B6B97EFA8D@redhat.com>
References: <CFD9A636-C32F-42A9-B97E-36B6B97EFA8D@redhat.com>
Message-ID: <507e3f97-fb75-e03f-d00b-34ec713fc7a0@redhat.com>

On 08/02/2017 01:02 PM, Galder Zamarre?o wrote:
> Hey Radim,
>
> Re: https://issues.jboss.org/browse/ISPN-8114
>
> I've been looking at the trace logs of this failure. I've extracted the most interesting parts of this failure into [1].
>
> What happens is that after loading the entries into the cache, the end invalidation message to allow put from loads to succeed does not get executed in time before the put from load is attempted. As a result of this, the put from load does not happen and hence the entry is not loaded into the cache.
>
> The end invalidation message eventually gets through. There's a gap between receiving the JGroups message and the actual execution, but that's due to the delivery mode of the message.
>
> I'm not sure how we should fix this. Options:
>
> 1) A thread sleep before loading entries "might work" but for a CI test this could always backfire with the right timing sets.
>
> 2) Find a way to hook into the PFLValidator class and only load after we know end invalidation has been received by all nodes.
>
> 3) Make end invalidation message sync? This would expensive. Even with async, changing delivery mode might have worked here... but under the right circumstances you could still get the same issue with async.

1) is unreliable and wrong from a testsuite perspective, and 3) is 
completely wrong (making sth sync just because it's easier to test it 
that way).

Spying on PFVL is an option, but I would rather wrap 
PerCacheInboundInvocationHandler (I hope that's the correct way). Note 
that there's even a TestingUtil.wrapInboundInvocationHandler() helper 
method.

If I am missing the complexity, please elaborate.

Radim

> I'm keen on trying to find a potential solution using 2), but wondered if you have other ideas.
>
> Cheers,
>
> [1] https://gist.github.com/galderz/0bce6dce16de018375e43e25c0cf3913
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From galder at redhat.com  Thu Aug  3 05:34:02 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Thu, 3 Aug 2017 11:34:02 +0200
Subject: [infinispan-dev] Late end invalidation delivery stops put from
	load - ISPN-8114
In-Reply-To: <507e3f97-fb75-e03f-d00b-34ec713fc7a0@redhat.com>
References: <CFD9A636-C32F-42A9-B97E-36B6B97EFA8D@redhat.com>
	<507e3f97-fb75-e03f-d00b-34ec713fc7a0@redhat.com>
Message-ID: <326DC9C0-6952-4463-A0C5-FB30CC4A26D0@redhat.com>

Thx Radim, I'll look into wrapping PerCacheInboundInvocationHandler.

Cheers,
--
Galder Zamarre?o
Infinispan, Red Hat

> On 2 Aug 2017, at 18:41, Radim Vansa <rvansa at redhat.com> wrote:
> 
> On 08/02/2017 01:02 PM, Galder Zamarre?o wrote:
>> Hey Radim,
>> 
>> Re: https://issues.jboss.org/browse/ISPN-8114
>> 
>> I've been looking at the trace logs of this failure. I've extracted the most interesting parts of this failure into [1].
>> 
>> What happens is that after loading the entries into the cache, the end invalidation message to allow put from loads to succeed does not get executed in time before the put from load is attempted. As a result of this, the put from load does not happen and hence the entry is not loaded into the cache.
>> 
>> The end invalidation message eventually gets through. There's a gap between receiving the JGroups message and the actual execution, but that's due to the delivery mode of the message.
>> 
>> I'm not sure how we should fix this. Options:
>> 
>> 1) A thread sleep before loading entries "might work" but for a CI test this could always backfire with the right timing sets.
>> 
>> 2) Find a way to hook into the PFLValidator class and only load after we know end invalidation has been received by all nodes.
>> 
>> 3) Make end invalidation message sync? This would expensive. Even with async, changing delivery mode might have worked here... but under the right circumstances you could still get the same issue with async.
> 
> 1) is unreliable and wrong from a testsuite perspective, and 3) is completely wrong (making sth sync just because it's easier to test it that way).
> 
> Spying on PFVL is an option, but I would rather wrap PerCacheInboundInvocationHandler (I hope that's the correct way). Note that there's even a TestingUtil.wrapInboundInvocationHandler() helper method.
> 
> If I am missing the complexity, please elaborate.
> 
> Radim
> 
>> I'm keen on trying to find a potential solution using 2), but wondered if you have other ideas.
>> 
>> Cheers,
>> 
>> [1] https://gist.github.com/galderz/0bce6dce16de018375e43e25c0cf3913
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>> 
> 
> 
> -- 
> Radim Vansa <rvansa at redhat.com>
> JBoss Performance Team



From galder at redhat.com  Thu Aug  3 06:57:47 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Thu, 3 Aug 2017 12:57:47 +0200
Subject: [infinispan-dev] Late end invalidation delivery stops put from
	load - ISPN-8114
In-Reply-To: <326DC9C0-6952-4463-A0C5-FB30CC4A26D0@redhat.com>
References: <CFD9A636-C32F-42A9-B97E-36B6B97EFA8D@redhat.com>
	<507e3f97-fb75-e03f-d00b-34ec713fc7a0@redhat.com>
	<326DC9C0-6952-4463-A0C5-FB30CC4A26D0@redhat.com>
Message-ID: <E6BBC5FB-D477-41F8-AD9B-5D1E6D8917B3@redhat.com>

FYI: https://github.com/infinispan/infinispan/pull/5350

--
Galder Zamarre?o
Infinispan, Red Hat

> On 3 Aug 2017, at 11:34, Galder Zamarre?o <galder at redhat.com> wrote:
> 
> Thx Radim, I'll look into wrapping PerCacheInboundInvocationHandler.
> 
> Cheers,
> --
> Galder Zamarre?o
> Infinispan, Red Hat
> 
>> On 2 Aug 2017, at 18:41, Radim Vansa <rvansa at redhat.com> wrote:
>> 
>> On 08/02/2017 01:02 PM, Galder Zamarre?o wrote:
>>> Hey Radim,
>>> 
>>> Re: https://issues.jboss.org/browse/ISPN-8114
>>> 
>>> I've been looking at the trace logs of this failure. I've extracted the most interesting parts of this failure into [1].
>>> 
>>> What happens is that after loading the entries into the cache, the end invalidation message to allow put from loads to succeed does not get executed in time before the put from load is attempted. As a result of this, the put from load does not happen and hence the entry is not loaded into the cache.
>>> 
>>> The end invalidation message eventually gets through. There's a gap between receiving the JGroups message and the actual execution, but that's due to the delivery mode of the message.
>>> 
>>> I'm not sure how we should fix this. Options:
>>> 
>>> 1) A thread sleep before loading entries "might work" but for a CI test this could always backfire with the right timing sets.
>>> 
>>> 2) Find a way to hook into the PFLValidator class and only load after we know end invalidation has been received by all nodes.
>>> 
>>> 3) Make end invalidation message sync? This would expensive. Even with async, changing delivery mode might have worked here... but under the right circumstances you could still get the same issue with async.
>> 
>> 1) is unreliable and wrong from a testsuite perspective, and 3) is completely wrong (making sth sync just because it's easier to test it that way).
>> 
>> Spying on PFVL is an option, but I would rather wrap PerCacheInboundInvocationHandler (I hope that's the correct way). Note that there's even a TestingUtil.wrapInboundInvocationHandler() helper method.
>> 
>> If I am missing the complexity, please elaborate.
>> 
>> Radim
>> 
>>> I'm keen on trying to find a potential solution using 2), but wondered if you have other ideas.
>>> 
>>> Cheers,
>>> 
>>> [1] https://gist.github.com/galderz/0bce6dce16de018375e43e25c0cf3913
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>> 
>> 
>> 
>> -- 
>> Radim Vansa <rvansa at redhat.com>
>> JBoss Performance Team
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



From sanne at infinispan.org  Thu Aug  3 07:49:40 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 3 Aug 2017 12:49:40 +0100
Subject: [infinispan-dev] NOVA: A Log-structured File system for Hybrid
 Volatile/Non-volatile Main Memories
Message-ID: <CAFm4XO24-0zasNWVkjR25x=_5JPXDS4UDNxERaJkqhQ7yz9EwA@mail.gmail.com>

This looks quite interesting!
 - http://lkml.iu.edu/hypermail/linux/kernel/1708.0/02224.html

Not being just a traditional file system, I wonder if we could
leverage some of its specific features like COW and the log structure
to avoid such redundant operations from our side.

For example Lucene's indexes work hard to re-implement a similar
schema on top of a general-purpose filesystem. I'd assume that the
filesystem has some oppoprtunities to do it "better".

Might be of interest for Our off-heap and CacheStores as well.

Thanks,
Sanne

From sanne at infinispan.org  Thu Aug  3 09:14:40 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 3 Aug 2017 14:14:40 +0100
Subject: [infinispan-dev] tuned profiles for Infinispan ?
In-Reply-To: <CAFm4XO1+mx4EAkifh301qRhzFFqD=vbED6Zm8bXZ1S83U=JCFg@mail.gmail.com>
References: <CAFm4XO0dr1YA+0hDbLgoBNpCY9n=gTz9LrquCN4j_mo9RoYN2A@mail.gmail.com>
	<CAA9R9O7QH+hgA1ykHahOUYyLxTbpFwy9S4+N7VBwhChtV1p2tw@mail.gmail.com>
	<B4736164-E0E4-4CE3-B524-9B182237B05E@hibernate.org>
	<CA+nfvwS8Ch0N2pxMg4JQ2XAKsM01_eryTAqW=Mv39HYasJRNMA@mail.gmail.com>
	<CAFm4XO1+mx4EAkifh301qRhzFFqD=vbED6Zm8bXZ1S83U=JCFg@mail.gmail.com>
Message-ID: <CAFm4XO2gD2W3Wgn=HEtL55_EWo6PDe7rSLZR+4g1CLpqh5j5MQ@mail.gmail.com>

On 19 Jul 2017 14:52, "Dan Berindei" <dan.berindei at gmail.com> wrote:

Can't we just copy a profile from Hibernate or WildFly?


I'm not aware of such profiles. Remember these are system wide settings,
I'm not sure how useful it is for libraries like Hibernate.

It seems more common for databases, I guess because they are more likely to
have a whole machine dedicated to their specialized use case.

WildFly is an interesting idea but I suspect they'd rather inherit from the
Infinispan tuning recommendations.



Dan

On Wed, Jul 19, 2017 at 2:40 PM, Emmanuel Bernard <emmanuel at hibernate.org>
wrote:

> I don?t think it discourages, the people you pention would simply use the
> ?default? profile. At least with a list of profiles, the idea of tuning
> pops into your mind and you can go further.
>
> On 18 Jul 2017, at 15:05, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
>
> I have mixed feelings about this to be honest. On one hand this gives a
> really good experience for new users (just pick a profile you want to use)
> but on the other hand tools like this discourage users for doing proper
> tuning work (why should I read any documentation and do anything if
> everything has already been provided by Infinispan authors).
>
> Nevertheless I think it might be worth to do a POC and host profiles in a
> separate repository (to avoid user confusion).
>
> On Tue, Jul 11, 2017 at 6:49 PM Sanne Grinovero <sanne at infinispan.org>
> wrote:
>
>> Hi all,
>>
>> tuned is a very nice utility to apply all kind of tuning options to a
>> machine focusing on performance options.
>>
>> Of course it doesn't replace the tuning that an expert could provide
>> for a specific system, but it gives people a quick an easy way to get
>> to a reasonable starting point, which is much better than the generic
>> out of the box of a Linux distribution.
>>
>> In many distributions it runs at boostrap transparently, for example
>> it will automatically apply a "laptop" profile if it's able to detect
>> running on a laptop, and might be the little tool which switches your
>> settings to an higher performance profile when you plug in the laptop.
>>
>> There's some good reference here:
>>  - https://access.redhat.com/documentation/en-US/Red_Hat_Enterp
>> rise_Linux/7/html/Performance_Tuning_Guide/sect-Red_Hat_
>> Enterprise_Linux-Performance_Tuning_Guide-Performance_
>> Monitoring_Tools-tuned_and_tuned_adm.html
>>
>> It's also easy to find it integrated with other tools, e.g. you can
>> use Ansible to set a profile.
>>
>> Distributions like Fedora have out of the box profiles included which
>> are good tuning base settings to run e.g. an Oracle RDBMS, an HANA
>> database, or just tune for latency rather than throughput.
>> Communities like Hadoop also provide suggested tuned settings.
>>
>> It would be great to distribute an Infinispan optimised profile? We
>> could ask the Fedora team to include it, I feel it's important to have
>> a profile there, or at least have one provided by any Infinispan RPMs.
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> --
> SEBASTIAN ?ASKAWIEC
>
> INFINISPAN DEVELOPER
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>


_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170803/cf1a8ae8/attachment.html 

From dan.berindei at gmail.com  Thu Aug  3 13:11:06 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Thu, 3 Aug 2017 20:11:06 +0300
Subject: [infinispan-dev] tuned profiles for Infinispan ?
In-Reply-To: <CAFm4XO2gD2W3Wgn=HEtL55_EWo6PDe7rSLZR+4g1CLpqh5j5MQ@mail.gmail.com>
References: <CAFm4XO0dr1YA+0hDbLgoBNpCY9n=gTz9LrquCN4j_mo9RoYN2A@mail.gmail.com>
	<CAA9R9O7QH+hgA1ykHahOUYyLxTbpFwy9S4+N7VBwhChtV1p2tw@mail.gmail.com>
	<B4736164-E0E4-4CE3-B524-9B182237B05E@hibernate.org>
	<CA+nfvwS8Ch0N2pxMg4JQ2XAKsM01_eryTAqW=Mv39HYasJRNMA@mail.gmail.com>
	<CAFm4XO1+mx4EAkifh301qRhzFFqD=vbED6Zm8bXZ1S83U=JCFg@mail.gmail.com>
	<CAFm4XO2gD2W3Wgn=HEtL55_EWo6PDe7rSLZR+4g1CLpqh5j5MQ@mail.gmail.com>
Message-ID: <CA+nfvwQth_Fh_BNTirfL6PODRhfGh6L4A4JroQ8+aiwdU2cdNg@mail.gmail.com>

On Thu, Aug 3, 2017 at 4:14 PM, Sanne Grinovero <sanne at infinispan.org>
wrote:

>
>
> On 19 Jul 2017 14:52, "Dan Berindei" <dan.berindei at gmail.com> wrote:
>
> Can't we just copy a profile from Hibernate or WildFly?
>
>
> I'm not aware of such profiles. Remember these are system wide settings,
> I'm not sure how useful it is for libraries like Hibernate.
>
> It seems more common for databases, I guess because they are more likely
> to have a whole machine dedicated to their specialized use case.
>
> WildFly is an interesting idea but I suspect they'd rather inherit from
> the Infinispan tuning recommendations.
>


That was exactly my point: let someone else do the work and copy it for
Infinispan :)

Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170803/fef01e0f/attachment.html 

From sanne at infinispan.org  Thu Aug  3 13:59:59 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Thu, 3 Aug 2017 18:59:59 +0100
Subject: [infinispan-dev] tuned profiles for Infinispan ?
In-Reply-To: <CA+nfvwQth_Fh_BNTirfL6PODRhfGh6L4A4JroQ8+aiwdU2cdNg@mail.gmail.com>
References: <CAFm4XO0dr1YA+0hDbLgoBNpCY9n=gTz9LrquCN4j_mo9RoYN2A@mail.gmail.com>
	<CAA9R9O7QH+hgA1ykHahOUYyLxTbpFwy9S4+N7VBwhChtV1p2tw@mail.gmail.com>
	<B4736164-E0E4-4CE3-B524-9B182237B05E@hibernate.org>
	<CA+nfvwS8Ch0N2pxMg4JQ2XAKsM01_eryTAqW=Mv39HYasJRNMA@mail.gmail.com>
	<CAFm4XO1+mx4EAkifh301qRhzFFqD=vbED6Zm8bXZ1S83U=JCFg@mail.gmail.com>
	<CAFm4XO2gD2W3Wgn=HEtL55_EWo6PDe7rSLZR+4g1CLpqh5j5MQ@mail.gmail.com>
	<CA+nfvwQth_Fh_BNTirfL6PODRhfGh6L4A4JroQ8+aiwdU2cdNg@mail.gmail.com>
Message-ID: <CAFm4XO2UnnqySp+vtwfoTA=XRPbCYE0HqV7Egtx1ZoH+X246+g@mail.gmail.com>

On 3 August 2017 at 18:11, Dan Berindei <dan.berindei at gmail.com> wrote:
>
>
> On Thu, Aug 3, 2017 at 4:14 PM, Sanne Grinovero <sanne at infinispan.org>
> wrote:
>>
>>
>>
>> On 19 Jul 2017 14:52, "Dan Berindei" <dan.berindei at gmail.com> wrote:
>>
>> Can't we just copy a profile from Hibernate or WildFly?
>>
>>
>> I'm not aware of such profiles. Remember these are system wide settings,
>> I'm not sure how useful it is for libraries like Hibernate.
>>
>> It seems more common for databases, I guess because they are more likely
>> to have a whole machine dedicated to their specialized use case.
>>
>> WildFly is an interesting idea but I suspect they'd rather inherit from
>> the Infinispan tuning recommendations.
>
>
>
> That was exactly my point: let someone else do the work and copy it for
> Infinispan :)

If all systems were equal the basic kernel defaults would be fine already no?

The only reason the Wildfly team might be interested into eventually
inheriting from the Infinispan tuned profile is that they heavily rely
on Infinispan. The opposite is not the same, as WildFly (and
Hibernate) don't benefit us much from system level tuning.

If you want to copy I already mentioned there's a profile for HANA -
that should be a reasonable starting point for an IMDG - and if you
read the pointers I gave you'd see it's a very readable file format so
it might take you just minutes to make some reasonable changes based
on your knowledge of Infinispan.

Thanks,
Sanne

From rory.odonnell at oracle.com  Tue Aug  8 06:00:02 2017
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Tue, 8 Aug 2017 11:00:02 +0100
Subject: [infinispan-dev] Ready for JDK 9 ?
Message-ID: <973bfcb1-d7c1-d479-4995-b0bfb3605c04@oracle.com>


Hi Galder,

Thank you very much for all your testing of JDK 9 during its 
development! Such contributions have significantly helped shape and 
improve JDK 9.

Now that we have reached the JDK 9 Final Release Candidate phase [1] , I 
would like to ask if your project can be considered to be 'ready for JDK 
9', or if there are any remaining show stopper issues which you've 
encountered when testing with the JDK 9 release candidate.

JDK 9  b181 is available at http://jdk.java.net/9/

If you have a public web page, mailing list post, or even a tweet 
announcing you project's readiness for JDK 9, I'd love to add the URL to 
the upcoming JDK 9 readiness page on the Quality Outreach wiki.


Looking forward to hearing from you,
Rory

[1] http://openjdk.java.net/projects/jdk9/

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland


From belaban at mailbox.org  Thu Aug 10 07:48:35 2017
From: belaban at mailbox.org (Bela Ban)
Date: Thu, 10 Aug 2017 13:48:35 +0200
Subject: [infinispan-dev] Fwd: [jgroups-users] Revamping the JGroups workshop
In-Reply-To: <mailman.40201.1502365151.9887.javagroups-users@lists.sourceforge.net>
References: <mailman.40201.1502365151.9887.javagroups-users@lists.sourceforge.net>
Message-ID: <385ad38f-12d9-d575-10de-f97a33d1fe13@mailbox.org>


FYI

-------- Forwarded Message --------
Subject: [jgroups-users] Revamping the JGroups workshop
Date: Thu, 10 Aug 2017 13:38:59 +0200
From: Questions/problems related to using JGroups 
<javagroups-users at lists.sourceforge.net>
Reply-To: javagroups-users at lists.sourceforge.net
To: jg-users <javagroups-users at lists.sourceforge.net>

I'm thinking about holding the JGroups workshop [1] in Europe in the 
fall and in the US early next year.

I'd have to upgrade the labs and slides to 4.0.x, and am thinking of 
revamping it as follows:

- Remove sections on distributed caching; this is done by Infinispan / 
JDG already

- Remove section on cross-datacenter replication (RELAY2)

- Expand sections on split brain issues, primary partition approach, 
eventual consistency,
CAP, issues with split brain and distributed locks/counters etc

- Add a section on jgroups-raft?

- Expand:
    - Common problems, their diagnosis and fixes
       - E.g. members don't find each other, frequent member exclusions, 
incorrect thread pool sizing etc
    - Monitoring with probe
    - JGroups and docker


Feedback is appreciated!

Any favorite locations? I'm thinking Munich or Berlin for Europe and 
Boston for the US...

Cheers,

[1] http://www.jgroups.org/workshops.html

-- 
Bela Ban | http://www.jgroups.org


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
javagroups-users mailing list
javagroups-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/javagroups-users

From gustavo at infinispan.org  Thu Aug 10 11:43:13 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Thu, 10 Aug 2017 16:43:13 +0100
Subject: [infinispan-dev] Rest server storage nuances
Message-ID: <CAH8Ud1G6Hr6VfOK9NhgWpRRP7Ut8yqZmBHLNXdCai57noCSW0w@mail.gmail.com>

Hi all,

With the ongoing encoding revamp work on Infinispan, time to decide how to
handle the rest server.

The rest server currently stores along with each entry, a string
representing the MimeType of that entry, which allows the user to POST/PUT
each entry with its own format.

At request time, using the accept header, the user can request the entry in
a particular format, and the rest server internally extracts the mime type
from the entry and converts it accordingly.

The issue with this approach is, apart from the extra space required, it
makes it challenging to expose via rest anything less trivial than put/get.
Think for example querying, consistent hash calculations, stream
operations: all those features will have a hard time dealing with a cache
where each entry has a different format.

Proposal:

Remove this behavior completely. For a certain cache, all entries will be
homogeneous, just like
Hot Rod, Memcached and embedded. The user can optionally configure the
MimeType at cache level.

Impacts:

- It may be required for users to add the media type configuration to the
cache, if supporting
multiple formats is required;
- Migrating from 9.1 to 9.2 may require re-populating the cache if multiple
formats are being used;
- From the API perspective, I don't expect any change: POST/PUT with
multiple formats should be still supported, as the internal transcoding
should convert them on the fly to the unified configured format.


Thoughts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170810/d315371b/attachment.html 

From ttarrant at redhat.com  Fri Aug 11 03:15:18 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Fri, 11 Aug 2017 09:15:18 +0200
Subject: [infinispan-dev] Rest server storage nuances
In-Reply-To: <CAH8Ud1G6Hr6VfOK9NhgWpRRP7Ut8yqZmBHLNXdCai57noCSW0w@mail.gmail.com>
References: <CAH8Ud1G6Hr6VfOK9NhgWpRRP7Ut8yqZmBHLNXdCai57noCSW0w@mail.gmail.com>
Message-ID: <c5b67c2f-dd4f-1ab1-8b11-51e596fd46ae@redhat.com>

On 10/08/17 17:43, Gustavo Fernandes wrote:

> Proposal:
> 
> Remove this behavior completely. For a certain cache, all entries will 
> be homogeneous, just like
> Hot Rod, Memcached and embedded. The user can optionally configure the 
> MimeType at cache level.

Yes, this is my preference and I believe that was the consensus last 
time we talked about this.


Tristan
-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From gustavo at infinispan.org  Fri Aug 11 03:59:46 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Fri, 11 Aug 2017 08:59:46 +0100
Subject: [infinispan-dev] Rest server storage nuances
In-Reply-To: <c5b67c2f-dd4f-1ab1-8b11-51e596fd46ae@redhat.com>
References: <CAH8Ud1G6Hr6VfOK9NhgWpRRP7Ut8yqZmBHLNXdCai57noCSW0w@mail.gmail.com>
	<c5b67c2f-dd4f-1ab1-8b11-51e596fd46ae@redhat.com>
Message-ID: <CAH8Ud1FwzHJb9mmTaxOdr+OX6K=PL4ND6MFwgmRxvHOFHbJGbQ@mail.gmail.com>

On Fri, Aug 11, 2017 at 8:15 AM, Tristan Tarrant <ttarrant at redhat.com>
wrote:

> On 10/08/17 17:43, Gustavo Fernandes wrote:
>
> Proposal:
>>
>> Remove this behavior completely. For a certain cache, all entries will be
>> homogeneous, just like
>> Hot Rod, Memcached and embedded. The user can optionally configure the
>> MimeType at cache level.
>>
>
> Yes, this is my preference and I believe that was the consensus last time
> we talked about this.
>


Yes, but we did not discuss which level of backwards compatibility to keep,
since we are doing this in a minor version.
Btw, removing the per-entry mimeMetadata is my preference as well. If
anyone has any concerns, it's the right time to speak up!


Cheers,
Gustavo


>
>
> Tristan
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170811/e86a86b0/attachment.html 

From galder at redhat.com  Fri Aug 11 09:14:31 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 11 Aug 2017 15:14:31 +0200
Subject: [infinispan-dev] Hot Rod client sending data to itself - ISPN-8186
Message-ID: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>

Hi,

Re: https://issues.jboss.org/browse/ISPN-8186

I've been looking at TRACE logs and what seems to happen is that is that sometimes, when the client needs to create a new Socket, it sends using the same localport as the Hot Rod server port. As a result, when the client sends something to the server, it also receives it, hence it ends finding a request instead of a response. Analysis of the logs linked in the JIRA can be found in [1].

What I'm not sure about is how to fix this... There are ways to potentially pass a specific localport to a Socket [2] but this could be a bit messy: It'd require us to generate a random local port and see if that works, making sure that's not the server port...

However, I think the real problem we're having here is the fact that both the server and client are bound to same IP address, 127.0.0.1. A simpler solution could be a way to get the server to be in a different IP address to the client, but what would that be that IP address and how to make sure it always works? Bind the server to eth0? 

Any other ideas?

Cheers,

[1] https://gist.github.com/galderz/b8549259ff65cb74505c9268eeec96a7
[2] http://docs.oracle.com/javase/6/docs/api/java/net/Socket.html#Socket(java.net.InetAddress,%20int,%20java.net.InetAddress,%20int)
--
Galder Zamarre?o
Infinispan, Red Hat



From sanne at infinispan.org  Fri Aug 11 10:30:46 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 11 Aug 2017 15:30:46 +0100
Subject: [infinispan-dev] Hot Rod client sending data to itself -
	ISPN-8186
In-Reply-To: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
Message-ID: <CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>

On 11 August 2017 at 14:14, Galder Zamarre?o <galder at redhat.com> wrote:
> Hi,
>
> Re: https://issues.jboss.org/browse/ISPN-8186
>
> I've been looking at TRACE logs and what seems to happen is that is that sometimes, when the client needs to create a new Socket, it sends using the same localport as the Hot Rod server port. As a result, when the client sends something to the server, it also receives it, hence it ends finding a request instead of a response. Analysis of the logs linked in the JIRA can be found in [1].
>
> What I'm not sure about is how to fix this... There are ways to potentially pass a specific localport to a Socket [2] but this could be a bit messy: It'd require us to generate a random local port and see if that works, making sure that's not the server port...
>
> However, I think the real problem we're having here is the fact that both the server and client are bound to same IP address, 127.0.0.1. A simpler solution could be a way to get the server to be in a different IP address to the client, but what would that be that IP address and how to make sure it always works? Bind the server to eth0?
>
> Any other ideas?

You could create multiple aliases for the same loopback device, and
assign a different IP address to each of them.

But I fail to understand why you don't have specific ports for each
purpose? That's the point for using ports in the first place, no?

Thanks,
Sanne


>
> Cheers,
>
> [1] https://gist.github.com/galderz/b8549259ff65cb74505c9268eeec96a7
> [2] http://docs.oracle.com/javase/6/docs/api/java/net/Socket.html#Socket(java.net.InetAddress,%20int,%20java.net.InetAddress,%20int)
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From dereed at redhat.com  Fri Aug 11 11:23:21 2017
From: dereed at redhat.com (Dennis Reed)
Date: Fri, 11 Aug 2017 10:23:21 -0500
Subject: [infinispan-dev] Hot Rod client sending data to itself -
 ISPN-8186
In-Reply-To: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
Message-ID: <0dcc1af5-8df4-58b9-044e-5893b002d4c4@redhat.com>

On 08/11/2017 08:14 AM, Galder Zamarre?o wrote:
> I've been looking at TRACE logs and what seems to happen is that is that sometimes, when the client needs to create a new Socket, it sends using the same localport as the Hot Rod server port. As a result, when the client sends something to the server, it also receives it, hence it ends finding a request instead of a response.

> What I'm not sure about is how to fix this... There are ways to potentially pass a specific localport to a Socket [2] but this could be a bit messy: It'd require us to generate a random local port and see if that works, making sure that's not the server port...

To fix that don't use an ephemeral port for the server.

> However, I think the real problem we're having here is the fact that both the server and client are bound to same IP address, 127.0.0.1. A simpler solution could be a way to get the server to be in a different IP address to the client, but what would that be that IP address and how to make sure it always works? Bind the server to eth0? 

You can't use a different IP for the client and server unless you have
two public IPs that can communicate with each other (not very common),
and explicitly set the client's IP.  That would be overkill just to work
around this particular issue.

Also note that it can only happen if the client is connecting to an
unused IP:port.  If the server is currently running on the port, it can't.

-Dennis

From gustavo at infinispan.org  Fri Aug 11 11:26:14 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Fri, 11 Aug 2017 16:26:14 +0100
Subject: [infinispan-dev] Hot Rod client sending data to itself -
	ISPN-8186
In-Reply-To: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
Message-ID: <CAH8Ud1FFX5h9X5niyzDEa3iymoaV0OPxeUk4Ez0AqzCZmcteuw@mail.gmail.com>

You could try binding to 127.0.0.2?


On Fri, Aug 11, 2017 at 2:14 PM, Galder Zamarre?o <galder at redhat.com> wrote:

> Hi,
>
> Re: https://issues.jboss.org/browse/ISPN-8186
>
> I've been looking at TRACE logs and what seems to happen is that is that
> sometimes, when the client needs to create a new Socket, it sends using the
> same localport as the Hot Rod server port. As a result, when the client
> sends something to the server, it also receives it, hence it ends finding a
> request instead of a response. Analysis of the logs linked in the JIRA can
> be found in [1].
>
> What I'm not sure about is how to fix this... There are ways to
> potentially pass a specific localport to a Socket [2] but this could be a
> bit messy: It'd require us to generate a random local port and see if that
> works, making sure that's not the server port...
>
> However, I think the real problem we're having here is the fact that both
> the server and client are bound to same IP address, 127.0.0.1. A simpler
> solution could be a way to get the server to be in a different IP address
> to the client, but what would that be that IP address and how to make sure
> it always works? Bind the server to eth0?
>
> Any other ideas?
>
> Cheers,
>
> [1] https://gist.github.com/galderz/b8549259ff65cb74505c9268eeec96a7
> [2] http://docs.oracle.com/javase/6/docs/api/java/net/Socket.
> html#Socket(java.net.InetAddress,%20int,%20java.net.InetAddress,%20int)
> <http://docs.oracle.com/javase/6/docs/api/java/net/Socket.html#Socket%28java.net.InetAddress,%20int,%20java.net.InetAddress,%20int%29>
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170811/cf879645/attachment.html 

From rvansa at redhat.com  Fri Aug 11 12:17:12 2017
From: rvansa at redhat.com (Radim Vansa)
Date: Fri, 11 Aug 2017 18:17:12 +0200
Subject: [infinispan-dev] Hot Rod client sending data to itself -
 ISPN-8186
In-Reply-To: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
Message-ID: <c424b389-ac19-331d-615b-5b014a09d0d7@redhat.com>

On 08/11/2017 03:14 PM, Galder Zamarre?o wrote:
> Hi,
>
> Re: https://issues.jboss.org/browse/ISPN-8186
>
> I've been looking at TRACE logs and what seems to happen is that is that sometimes, when the client needs to create a new Socket, it sends using the same localport as the Hot Rod server port. As a result, when the client sends something to the server, it also receives it, hence it ends finding a request instead of a response. Analysis of the logs linked in the JIRA can be found in [1].

I've always assumed that a TCP connection is a two-end pipe, both being 
successfully connected to the remote side and looping back messages to 
the client sounds strange... have you googled if this is a bug or 
feature? Or is the client talking to itself only despite the server 
being successfully bound to listen on the port?
>
> What I'm not sure about is how to fix this... There are ways to potentially pass a specific localport to a Socket [2] but this could be a bit messy: It'd require us to generate a random local port and see if that works, making sure that's not the server port...

Clients use ephemeral ports that have limited range (32k+ on Linux?), 
what about making sure that the server port is out of this range?
> However, I think the real problem we're having here is the fact that both the server and client are bound to same IP address, 127.0.0.1. A simpler solution could be a way to get the server to be in a different IP address to the client, but what would that be that IP address and how to make sure it always works? Bind the server to eth0?

I don't think that having both on the same IP is a real problem. 
Testsuite should not listen on externally accessible ports, besides the 
fact that the computer may not have any ports (ok, that's rather 
theoretical nowadays) there's a security risk involved. And if the 
machine has firewall set to block connection to externally accessible 
ports (even from local addresses), the testsuite would fail.

Radim

>
> Any other ideas?
>
> Cheers,
>
> [1] https://gist.github.com/galderz/b8549259ff65cb74505c9268eeec96a7
> [2] http://docs.oracle.com/javase/6/docs/api/java/net/Socket.html#Socket(java.net.InetAddress,%20int,%20java.net.InetAddress,%20int)
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team


From gustavo at infinispan.org  Fri Aug 11 12:48:01 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Fri, 11 Aug 2017 17:48:01 +0100
Subject: [infinispan-dev] Hot Rod client sending data to itself -
	ISPN-8186
In-Reply-To: <CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
Message-ID: <CAH8Ud1E8FBV5Aq=i9=LAasmXSpQ3LoUcdCfsQMDieqH85BbKaw@mail.gmail.com>

On Fri, Aug 11, 2017 at 3:30 PM, Sanne Grinovero <sanne at infinispan.org>
wrote:

> On 11 August 2017 at 14:14, Galder Zamarre?o <galder at redhat.com> wrote:
> > Hi,
> >
> > Re: https://issues.jboss.org/browse/ISPN-8186
> >
> > I've been looking at TRACE logs and what seems to happen is that is that
> sometimes, when the client needs to create a new Socket, it sends using the
> same localport as the Hot Rod server port. As a result, when the client
> sends something to the server, it also receives it, hence it ends finding a
> request instead of a response. Analysis of the logs linked in the JIRA can
> be found in [1].
> >
> > What I'm not sure about is how to fix this... There are ways to
> potentially pass a specific localport to a Socket [2] but this could be a
> bit messy: It'd require us to generate a random local port and see if that
> works, making sure that's not the server port...
> >
> > However, I think the real problem we're having here is the fact that
> both the server and client are bound to same IP address, 127.0.0.1. A
> simpler solution could be a way to get the server to be in a different IP
> address to the client, but what would that be that IP address and how to
> make sure it always works? Bind the server to eth0?
> >
> > Any other ideas?
>
> You could create multiple aliases for the same loopback device, and
> assign a different IP address to each of them.
>

No need for aliases, we could use directly use any IP from the range
127.0.0.0/8
Just a reminder, the testsuite should be able to run on Linux/Win/MacOS so
I'd stay
away for OS specific extra steps


>
> But I fail to understand why you don't have specific ports for each
> purpose? That's the point for using ports in the first place, no?
>
>

The issue is the suite runs 15 tests in parallel, with each test starting
potentially more than 1 server;
the client(s) obtains the server port(s) dynamically and start a
connection, but with bad luck the client's
localport happens to be the same as the server random port.



> Thanks,
> Sanne
>
>
> >
> > Cheers,
> >
> > [1] https://gist.github.com/galderz/b8549259ff65cb74505c9268eeec96a7
> > [2] http://docs.oracle.com/javase/6/docs/api/java/net/Socket.
> html#Socket(java.net.InetAddress,%20int,%20java.net.InetAddress,%20int)
> > --
> > Galder Zamarre?o
> > Infinispan, Red Hat
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170811/1ac9273c/attachment-0001.html 

From galder at redhat.com  Fri Aug 11 12:50:17 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Fri, 11 Aug 2017 18:50:17 +0200
Subject: [infinispan-dev] Hot Rod client sending data to itself -
 ISPN-8186
In-Reply-To: <CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
Message-ID: <DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>



> On 11 Aug 2017, at 16:30, Sanne Grinovero <sanne at infinispan.org> wrote:
> 
> On 11 August 2017 at 14:14, Galder Zamarre?o <galder at redhat.com> wrote:
>> Hi,
>> 
>> Re: https://issues.jboss.org/browse/ISPN-8186
>> 
>> I've been looking at TRACE logs and what seems to happen is that is that sometimes, when the client needs to create a new Socket, it sends using the same localport as the Hot Rod server port. As a result, when the client sends something to the server, it also receives it, hence it ends finding a request instead of a response. Analysis of the logs linked in the JIRA can be found in [1].
>> 
>> What I'm not sure about is how to fix this... There are ways to potentially pass a specific localport to a Socket [2] but this could be a bit messy: It'd require us to generate a random local port and see if that works, making sure that's not the server port...
>> 
>> However, I think the real problem we're having here is the fact that both the server and client are bound to same IP address, 127.0.0.1. A simpler solution could be a way to get the server to be in a different IP address to the client, but what would that be that IP address and how to make sure it always works? Bind the server to eth0?
>> 
>> Any other ideas?
> 
> You could create multiple aliases for the same loopback device, and
> assign a different IP address to each of them.
> 
> But I fail to understand why you don't have specific ports for each
> purpose? That's the point for using ports in the first place, no?

^ Hmmm? 

The servers in the test use a random port that's available. The clients connect to these ports. The local ports used by the clients are random. You need to use APIs such as [2] to fix them.

So, what exactly are you talking about? Are you saying we should fix the local client ports? As I said in the first post, we could try to find a random port that's not the server one...

I must admit this scenario sounds very weird... how does Java allow you for a local port to be bound to a port that's already in use by the server? It doesn't make sense. I'll be trying to replicate this in a small unit test next few days...

Cheers,

> 
> Thanks,
> Sanne
> 
> 
>> 
>> Cheers,
>> 
>> [1] https://gist.github.com/galderz/b8549259ff65cb74505c9268eeec96a7
>> [2] http://docs.oracle.com/javase/6/docs/api/java/net/Socket.html#Socket(java.net.InetAddress,%20int,%20java.net.InetAddress,%20int)
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarre?o
Infinispan, Red Hat



From gustavo at infinispan.org  Fri Aug 11 13:07:12 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Fri, 11 Aug 2017 18:07:12 +0100
Subject: [infinispan-dev] Hot Rod client sending data to itself -
	ISPN-8186
In-Reply-To: <CAH8Ud1E8FBV5Aq=i9=LAasmXSpQ3LoUcdCfsQMDieqH85BbKaw@mail.gmail.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
	<CAH8Ud1E8FBV5Aq=i9=LAasmXSpQ3LoUcdCfsQMDieqH85BbKaw@mail.gmail.com>
Message-ID: <CAH8Ud1H3ydh1n0gv=t=ciRK2qnTe2FpgELwfsFh=HoTXvZekpQ@mail.gmail.com>

On Fri, Aug 11, 2017 at 5:48 PM, Gustavo Fernandes <gustavo at infinispan.org>
wrote:

> On Fri, Aug 11, 2017 at 3:30 PM, Sanne Grinovero <sanne at infinispan.org>
> wrote:
>
>> On 11 August 2017 at 14:14, Galder Zamarre?o <galder at redhat.com> wrote:
>> > Hi,
>> >
>> > Re: https://issues.jboss.org/browse/ISPN-8186
>> >
>> > I've been looking at TRACE logs and what seems to happen is that is
>> that sometimes, when the client needs to create a new Socket, it sends
>> using the same localport as the Hot Rod server port. As a result, when the
>> client sends something to the server, it also receives it, hence it ends
>> finding a request instead of a response. Analysis of the logs linked in the
>> JIRA can be found in [1].
>> >
>> > What I'm not sure about is how to fix this... There are ways to
>> potentially pass a specific localport to a Socket [2] but this could be a
>> bit messy: It'd require us to generate a random local port and see if that
>> works, making sure that's not the server port...
>> >
>> > However, I think the real problem we're having here is the fact that
>> both the server and client are bound to same IP address, 127.0.0.1. A
>> simpler solution could be a way to get the server to be in a different IP
>> address to the client, but what would that be that IP address and how to
>> make sure it always works? Bind the server to eth0?
>> >
>> > Any other ideas?
>>
>> You could create multiple aliases for the same loopback device, and
>> assign a different IP address to each of them.
>>
>
> No need for aliases, we could use directly use any IP from the range
> 127.0.0.0/8
> Just a reminder, the testsuite should be able to run on Linux/Win/MacOS so
> I'd stay
> away for OS specific extra steps
>


I tested on a recent Linux and Win, it works, but MacOS requires an alias
to be created...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170811/5a18491b/attachment.html 

From sanne at infinispan.org  Fri Aug 11 14:49:42 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Fri, 11 Aug 2017 19:49:42 +0100
Subject: [infinispan-dev] Hot Rod client sending data to itself -
	ISPN-8186
In-Reply-To: <DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
	<DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>
Message-ID: <CAFm4XO1J96Xn_PSunAY9SLRAM91b8NorUDvPFeSL2qqjpStcDg@mail.gmail.com>

On 11 August 2017 at 17:50, Galder Zamarre?o <galder at redhat.com> wrote:
>
>
>> On 11 Aug 2017, at 16:30, Sanne Grinovero <sanne at infinispan.org> wrote:
>>
>> On 11 August 2017 at 14:14, Galder Zamarre?o <galder at redhat.com> wrote:
>>> Hi,
>>>
>>> Re: https://issues.jboss.org/browse/ISPN-8186
>>>
>>> I've been looking at TRACE logs and what seems to happen is that is that sometimes, when the client needs to create a new Socket, it sends using the same localport as the Hot Rod server port. As a result, when the client sends something to the server, it also receives it, hence it ends finding a request instead of a response. Analysis of the logs linked in the JIRA can be found in [1].
>>>
>>> What I'm not sure about is how to fix this... There are ways to potentially pass a specific localport to a Socket [2] but this could be a bit messy: It'd require us to generate a random local port and see if that works, making sure that's not the server port...
>>>
>>> However, I think the real problem we're having here is the fact that both the server and client are bound to same IP address, 127.0.0.1. A simpler solution could be a way to get the server to be in a different IP address to the client, but what would that be that IP address and how to make sure it always works? Bind the server to eth0?
>>>
>>> Any other ideas?
>>
>> You could create multiple aliases for the same loopback device, and
>> assign a different IP address to each of them.
>>
>> But I fail to understand why you don't have specific ports for each
>> purpose? That's the point for using ports in the first place, no?
>
> ^ Hmmm?
>
> The servers in the test use a random port that's available. The clients connect to these ports. The local ports used by the clients are random. You need to use APIs such as [2] to fix them.
>
> So, what exactly are you talking about? Are you saying we should fix the local client ports? As I said in the first post, we could try to find a random port that's not the server one...

It should be clear that all sources of randomness are your enemy?

For obvious reasons we have to endure with reproducibility complexity
caused by using threads, timing, network packets interleaving... you
can't do much about these but you could at least try to introduce more
of it.

I would just hard code all ports in the testsuite, and test for their
availability during testsuite initialization. This would have some
other nice consequences, like improving the likelyhood to detect
leaked sockets.

You might remember that the main reason I proposed a "timer service"
years ago was to be able to mock time passage and make that all
reproducible even in the presence of GC and slow machines; eventually
you'd have been able to remove all reasons to actually run tests in
parallel as all needs for idle waiting would be controlled, so further
reducing variability.

My whole point has always been that you need to reduce randomness to
improve the testsuite reliability.

Thanks,
Sanne


>
> I must admit this scenario sounds very weird... how does Java allow you for a local port to be bound to a port that's already in use by the server? It doesn't make sense. I'll be trying to replicate this in a small unit test next few days...
>
> Cheers,
>
>>
>> Thanks,
>> Sanne
>>
>>
>>>
>>> Cheers,
>>>
>>> [1] https://gist.github.com/galderz/b8549259ff65cb74505c9268eeec96a7
>>> [2] http://docs.oracle.com/javase/6/docs/api/java/net/Socket.html#Socket(java.net.InetAddress,%20int,%20java.net.InetAddress,%20int)
>>> --
>>> Galder Zamarre?o
>>> Infinispan, Red Hat
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Galder Zamarre?o
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From dereed at redhat.com  Fri Aug 11 14:56:51 2017
From: dereed at redhat.com (Dennis Reed)
Date: Fri, 11 Aug 2017 13:56:51 -0500
Subject: [infinispan-dev] Hot Rod client sending data to itself -
 ISPN-8186
In-Reply-To: <DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
	<DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>
Message-ID: <c6c636d1-2189-5995-1e89-6c58cc677402@redhat.com>

On 08/11/2017 11:50 AM, Galder Zamarre?o wrote:
> I must admit this scenario sounds very weird... how does Java allow you for a local port to be bound to a port that's already in use by the server? It doesn't make sense.

You cannot bind to a port that's already in use.

But if you're trying to connect to a port in the ephemeral range that's
not in use, and the OS happens to assign that same IP:port to the local
socket, it can connect to itself.

(We've run into this in JGroups before, and it was a pain to track down
what was going on).

-Dennis

From belaban at mailbox.org  Mon Aug 14 02:48:52 2017
From: belaban at mailbox.org (Bela Ban)
Date: Mon, 14 Aug 2017 08:48:52 +0200
Subject: [infinispan-dev] Hot Rod client sending data to itself -
 ISPN-8186
In-Reply-To: <c6c636d1-2189-5995-1e89-6c58cc677402@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
	<DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>
	<c6c636d1-2189-5995-1e89-6c58cc677402@redhat.com>
Message-ID: <03b90a50-6834-c671-2b7b-8b37fc05fa02@mailbox.org>

Right: the localHost:localPort combo of the client socket cannot be the 
same as that of the remoteHost:remotePort.

Do you happen to have the link to the JGroups issue? I also remember 
this, but googling I failed to find it. Perhaps we can use the same 
solution here that we used for the JGRP issue.

I vaguely recall we checked the client's local address:port against some 
server address:port and closed/re-created it is the same.

On 11/08/17 20:56, Dennis Reed wrote:
> On 08/11/2017 11:50 AM, Galder Zamarre?o wrote:
>> I must admit this scenario sounds very weird... how does Java allow you for a local port to be bound to a port that's already in use by the server? It doesn't make sense.
> 
> You cannot bind to a port that's already in use.
> 
> But if you're trying to connect to a port in the ephemeral range that's
> not in use, and the OS happens to assign that same IP:port to the local
> socket, it can connect to itself.
> 
> (We've run into this in JGroups before, and it was a pain to track down
> what was going on).
> 
> -Dennis
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Bela Ban | http://www.jgroups.org


From vjuranek at redhat.com  Mon Aug 14 04:07:14 2017
From: vjuranek at redhat.com (Vojtech Juranek)
Date: Mon, 14 Aug 2017 10:07:14 +0200
Subject: [infinispan-dev] Hot Rod client sending data to itself -
	ISPN-8186
In-Reply-To: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
Message-ID: <2158521.pbHcckdm2G@dhcp-10-40-4-103.brq.redhat.com>

> Any other ideas?

I'd put some limits on random server port. If server port is under 32k, it 
should work on most of platforms I guess. But I'd agree with Sanne here and 
reconsider, if we should use random ports at all.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part.
Url : http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170814/7d19d4ad/attachment.bin 

From galder at redhat.com  Mon Aug 14 09:04:50 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Mon, 14 Aug 2017 15:04:50 +0200
Subject: [infinispan-dev] Hot Rod client sending data to itself -
 ISPN-8186
In-Reply-To: <03b90a50-6834-c671-2b7b-8b37fc05fa02@mailbox.org>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
	<DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>
	<c6c636d1-2189-5995-1e89-6c58cc677402@redhat.com>
	<03b90a50-6834-c671-2b7b-8b37fc05fa02@mailbox.org>
Message-ID: <D01DA4AE-FA1F-4B11-B483-A113FD3BA568@redhat.com>

I did run a local test and indeed you get a bind exception if trying to bind a local port that's in use as server port:

https://github.com/galderz/java-sandbox/blob/master/src/main/java/j/net/LocalPortClash.java

I'll check JGRP source and JIRA and try to dig this further.

Cheers,

> On 14 Aug 2017, at 08:48, Bela Ban <belaban at mailbox.org> wrote:
> 
> Right: the localHost:localPort combo of the client socket cannot be the 
> same as that of the remoteHost:remotePort.
> 
> Do you happen to have the link to the JGroups issue? I also remember 
> this, but googling I failed to find it. Perhaps we can use the same 
> solution here that we used for the JGRP issue.
> 
> I vaguely recall we checked the client's local address:port against some 
> server address:port and closed/re-created it is the same.
> 
> On 11/08/17 20:56, Dennis Reed wrote:
>> On 08/11/2017 11:50 AM, Galder Zamarre?o wrote:
>>> I must admit this scenario sounds very weird... how does Java allow you for a local port to be bound to a port that's already in use by the server? It doesn't make sense.
>> 
>> You cannot bind to a port that's already in use.
>> 
>> But if you're trying to connect to a port in the ephemeral range that's
>> not in use, and the OS happens to assign that same IP:port to the local
>> socket, it can connect to itself.
>> 
>> (We've run into this in JGroups before, and it was a pain to track down
>> what was going on).
>> 
>> -Dennis
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
> 
> -- 
> Bela Ban | http://www.jgroups.org
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarre?o
Infinispan, Red Hat



From afield at redhat.com  Mon Aug 14 09:39:33 2017
From: afield at redhat.com (Alan Field)
Date: Mon, 14 Aug 2017 09:39:33 -0400 (EDT)
Subject: [infinispan-dev] Hot Rod client sending data to itself -
 ISPN-8186
In-Reply-To: <D01DA4AE-FA1F-4B11-B483-A113FD3BA568@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
	<DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>
	<c6c636d1-2189-5995-1e89-6c58cc677402@redhat.com>
	<03b90a50-6834-c671-2b7b-8b37fc05fa02@mailbox.org>
	<D01DA4AE-FA1F-4B11-B483-A113FD3BA568@redhat.com>
Message-ID: <938163991.369247.1502717972999.JavaMail.zimbra@redhat.com>

Is this the issue, Bela?

https://issues.jboss.org/browse/JGRP-1116

Thanks,
Alan

----- Original Message -----
> From: "Galder Zamarre?o" <galder at redhat.com>
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Monday, August 14, 2017 9:04:50 AM
> Subject: Re: [infinispan-dev] Hot Rod client sending data to itself - ISPN-8186
> 
> I did run a local test and indeed you get a bind exception if trying to bind
> a local port that's in use as server port:
> 
> https://github.com/galderz/java-sandbox/blob/master/src/main/java/j/net/LocalPortClash.java
> 
> I'll check JGRP source and JIRA and try to dig this further.
> 
> Cheers,
> 
> > On 14 Aug 2017, at 08:48, Bela Ban <belaban at mailbox.org> wrote:
> > 
> > Right: the localHost:localPort combo of the client socket cannot be the
> > same as that of the remoteHost:remotePort.
> > 
> > Do you happen to have the link to the JGroups issue? I also remember
> > this, but googling I failed to find it. Perhaps we can use the same
> > solution here that we used for the JGRP issue.
> > 
> > I vaguely recall we checked the client's local address:port against some
> > server address:port and closed/re-created it is the same.
> > 
> > On 11/08/17 20:56, Dennis Reed wrote:
> >> On 08/11/2017 11:50 AM, Galder Zamarre?o wrote:
> >>> I must admit this scenario sounds very weird... how does Java allow you
> >>> for a local port to be bound to a port that's already in use by the
> >>> server? It doesn't make sense.
> >> 
> >> You cannot bind to a port that's already in use.
> >> 
> >> But if you're trying to connect to a port in the ephemeral range that's
> >> not in use, and the OS happens to assign that same IP:port to the local
> >> socket, it can connect to itself.
> >> 
> >> (We've run into this in JGroups before, and it was a pain to track down
> >> what was going on).
> >> 
> >> -Dennis
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >> 
> > 
> > --
> > Bela Ban | http://www.jgroups.org
> > 
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> --
> Galder Zamarre?o
> Infinispan, Red Hat
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


From belaban at mailbox.org  Mon Aug 14 11:18:05 2017
From: belaban at mailbox.org (Bela Ban)
Date: Mon, 14 Aug 2017 17:18:05 +0200
Subject: [infinispan-dev] Hot Rod client sending data to itself -
 ISPN-8186
In-Reply-To: <938163991.369247.1502717972999.JavaMail.zimbra@redhat.com>
References: <97755BF4-A1F5-40B4-9547-8DDEB262CEBA@redhat.com>
	<CAFm4XO27vLA7u2fhq7deESHgKi_pvCzpYcakR=rPG8c=Ak-45g@mail.gmail.com>
	<DE1BF179-486F-4941-8543-42AA31BC7664@redhat.com>
	<c6c636d1-2189-5995-1e89-6c58cc677402@redhat.com>
	<03b90a50-6834-c671-2b7b-8b37fc05fa02@mailbox.org>
	<D01DA4AE-FA1F-4B11-B483-A113FD3BA568@redhat.com>
	<938163991.369247.1502717972999.JavaMail.zimbra@redhat.com>
Message-ID: <67732ba9-e288-f7fe-cdb0-7109dfb96e0d@mailbox.org>

Yes, thanks Alan,

so what we did was check the local (ephemeral) port after the bind, and 
close/re-open/re-bind if that local port was the same as the destination 
port (the server's port).

This should work as fix for this specific issue here, too, unless I'm 
mistaken.

On 14/08/17 15:39, Alan Field wrote:
> Is this the issue, Bela?
> 
> https://issues.jboss.org/browse/JGRP-1116
> 
> Thanks,
> Alan
> 
> ----- Original Message -----
>> From: "Galder Zamarre?o" <galder at redhat.com>
>> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>> Sent: Monday, August 14, 2017 9:04:50 AM
>> Subject: Re: [infinispan-dev] Hot Rod client sending data to itself - ISPN-8186
>>
>> I did run a local test and indeed you get a bind exception if trying to bind
>> a local port that's in use as server port:
>>
>> https://github.com/galderz/java-sandbox/blob/master/src/main/java/j/net/LocalPortClash.java
>>
>> I'll check JGRP source and JIRA and try to dig this further.
>>
>> Cheers,
>>
>>> On 14 Aug 2017, at 08:48, Bela Ban <belaban at mailbox.org> wrote:
>>>
>>> Right: the localHost:localPort combo of the client socket cannot be the
>>> same as that of the remoteHost:remotePort.
>>>
>>> Do you happen to have the link to the JGroups issue? I also remember
>>> this, but googling I failed to find it. Perhaps we can use the same
>>> solution here that we used for the JGRP issue.
>>>
>>> I vaguely recall we checked the client's local address:port against some
>>> server address:port and closed/re-created it is the same.
>>>
>>> On 11/08/17 20:56, Dennis Reed wrote:
>>>> On 08/11/2017 11:50 AM, Galder Zamarre?o wrote:
>>>>> I must admit this scenario sounds very weird... how does Java allow you
>>>>> for a local port to be bound to a port that's already in use by the
>>>>> server? It doesn't make sense.
>>>>
>>>> You cannot bind to a port that's already in use.
>>>>
>>>> But if you're trying to connect to a port in the ephemeral range that's
>>>> not in use, and the OS happens to assign that same IP:port to the local
>>>> socket, it can connect to itself.
>>>>
>>>> (We've run into this in JGroups before, and it was a pain to track down
>>>> what was going on).
>>>>
>>>> -Dennis
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>> --
>>> Bela Ban | http://www.jgroups.org
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Galder Zamarre?o
>> Infinispan, Red Hat
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Bela Ban | http://www.jgroups.org


From slaskawi at redhat.com  Thu Aug 17 08:43:43 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 17 Aug 2017 14:43:43 +0200
Subject: [infinispan-dev] Fwd: [jgroups-users] Revamping the JGroups
	workshop
In-Reply-To: <385ad38f-12d9-d575-10de-f97a33d1fe13@mailbox.org>
References: <mailman.40201.1502365151.9887.javagroups-users@lists.sourceforge.net>
	<385ad38f-12d9-d575-10de-f97a33d1fe13@mailbox.org>
Message-ID: <CAA9R9O4O9gtgvmcySaGRK4Csfa8p7udNxkHiPNGdsM_ZsrQe8g@mail.gmail.com>

Hey Bela,

I think RELAY2 section has a lot of value. Many people are scratching their
heads how to do cross DC replication.

Thanks,
Sebastian

On Thu, Aug 10, 2017 at 1:48 PM, Bela Ban <belaban at mailbox.org> wrote:

>
> FYI
>
> -------- Forwarded Message --------
> Subject: [jgroups-users] Revamping the JGroups workshop
> Date: Thu, 10 Aug 2017 13:38:59 +0200
> From: Questions/problems related to using JGroups
> <javagroups-users at lists.sourceforge.net>
> Reply-To: javagroups-users at lists.sourceforge.net
> To: jg-users <javagroups-users at lists.sourceforge.net>
>
> I'm thinking about holding the JGroups workshop [1] in Europe in the
> fall and in the US early next year.
>
> I'd have to upgrade the labs and slides to 4.0.x, and am thinking of
> revamping it as follows:
>
> - Remove sections on distributed caching; this is done by Infinispan /
> JDG already
>
> - Remove section on cross-datacenter replication (RELAY2)
>
> - Expand sections on split brain issues, primary partition approach,
> eventual consistency,
> CAP, issues with split brain and distributed locks/counters etc
>
> - Add a section on jgroups-raft?
>
> - Expand:
>     - Common problems, their diagnosis and fixes
>        - E.g. members don't find each other, frequent member exclusions,
> incorrect thread pool sizing etc
>     - Monitoring with probe
>     - JGroups and docker
>
>
> Feedback is appreciated!
>
> Any favorite locations? I'm thinking Munich or Berlin for Europe and
> Boston for the US...
>
> Cheers,
>
> [1] http://www.jgroups.org/workshops.html
>
> --
> Bela Ban | http://www.jgroups.org
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> javagroups-users mailing list
> javagroups-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170817/21291ba1/attachment.html 

From belaban at mailbox.org  Thu Aug 17 09:21:16 2017
From: belaban at mailbox.org (Bela Ban)
Date: Thu, 17 Aug 2017 15:21:16 +0200
Subject: [infinispan-dev] Fwd: [jgroups-users] Revamping the JGroups
 workshop
In-Reply-To: <CAA9R9O4O9gtgvmcySaGRK4Csfa8p7udNxkHiPNGdsM_ZsrQe8g@mail.gmail.com>
References: <mailman.40201.1502365151.9887.javagroups-users@lists.sourceforge.net>
	<385ad38f-12d9-d575-10de-f97a33d1fe13@mailbox.org>
	<CAA9R9O4O9gtgvmcySaGRK4Csfa8p7udNxkHiPNGdsM_ZsrQe8g@mail.gmail.com>
Message-ID: <a4b1fbb1-bbfb-5a32-7da8-d82a59be2606@mailbox.org>

I fall asleep when I teach this part... :-)

This should be part of the JDG workshop anyway!

On 17/08/17 14:43, Sebastian Laskawiec wrote:
> Hey Bela,
> 
> I think RELAY2 section has a lot of value. Many people are scratching 
> their heads how to do cross DC replication.
> 
> Thanks,
> Sebastian
> 
> On Thu, Aug 10, 2017 at 1:48 PM, Bela Ban <belaban at mailbox.org 
> <mailto:belaban at mailbox.org>> wrote:
> 
> 
>     FYI
> 
>     -------- Forwarded Message --------
>     Subject: [jgroups-users] Revamping the JGroups workshop
>     Date: Thu, 10 Aug 2017 13:38:59 +0200
>     From: Questions/problems related to using JGroups
>     <javagroups-users at lists.sourceforge.net
>     <mailto:javagroups-users at lists.sourceforge.net>>
>     Reply-To: javagroups-users at lists.sourceforge.net
>     <mailto:javagroups-users at lists.sourceforge.net>
>     To: jg-users <javagroups-users at lists.sourceforge.net
>     <mailto:javagroups-users at lists.sourceforge.net>>
> 
>     I'm thinking about holding the JGroups workshop [1] in Europe in the
>     fall and in the US early next year.
> 
>     I'd have to upgrade the labs and slides to 4.0.x, and am thinking of
>     revamping it as follows:
> 
>     - Remove sections on distributed caching; this is done by Infinispan /
>     JDG already
> 
>     - Remove section on cross-datacenter replication (RELAY2)
> 
>     - Expand sections on split brain issues, primary partition approach,
>     eventual consistency,
>     CAP, issues with split brain and distributed locks/counters etc
> 
>     - Add a section on jgroups-raft?
> 
>     - Expand:
>          - Common problems, their diagnosis and fixes
>             - E.g. members don't find each other, frequent member
>     exclusions,
>     incorrect thread pool sizing etc
>          - Monitoring with probe
>          - JGroups and docker
> 
> 
>     Feedback is appreciated!
> 
>     Any favorite locations? I'm thinking Munich or Berlin for Europe and
>     Boston for the US...
> 
>     Cheers,
> 
>     [1] http://www.jgroups.org/workshops.html
>     <http://www.jgroups.org/workshops.html>
> 
>     --
>     Bela Ban | http://www.jgroups.org
> 
> 
>     ------------------------------------------------------------------------------
>     Check out the vibrant tech community on one of the world's most
>     engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>     _______________________________________________
>     javagroups-users mailing list
>     javagroups-users at lists.sourceforge.net
>     <mailto:javagroups-users at lists.sourceforge.net>
>     https://lists.sourceforge.net/lists/listinfo/javagroups-users
>     <https://lists.sourceforge.net/lists/listinfo/javagroups-users>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Bela Ban | http://www.jgroups.org


From mgencur at redhat.com  Fri Aug 18 08:26:06 2017
From: mgencur at redhat.com (Martin Gencur)
Date: Fri, 18 Aug 2017 14:26:06 +0200
Subject: [infinispan-dev] Smoke test suite for hibernate-cache
Message-ID: <11383c31-d845-410f-7210-bc0e2ea0ae7a@redhat.com>

Hi all,
I'm currently in the process of refreshing the "smoke" test suite for 
Infinispan.
There's a relatively new module called hibernate-cache. Could someone 
suggest tests that should be part of the smoke test suite?
Ideally just a few tens of test cases (maybe a few hundreds at most but 
the test suite execution should finish in a few seconds).

A list of test classes as a reply to this email would be ideal:)

Thanks,
Martin





From gustavo at infinispan.org  Mon Aug 21 11:15:52 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Mon, 21 Aug 2017 16:15:52 +0100
Subject: [infinispan-dev] Weekly IRC Meeting logs 2017-08-21
Message-ID: <CAH8Ud1FH538M2qX2yRrVy2ES3ze99y_4jpQehpM3qdevskNfrg@mail.gmail.com>

Hi, here are the logs for our last weekly meeting:

http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2017/infinispan.2017-08-21-14.05.log.html

Cheers,
Gustavo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170821/ef6bf96e/attachment-0001.html 

From slaskawi at redhat.com  Tue Aug 22 03:14:31 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Tue, 22 Aug 2017 07:14:31 +0000
Subject: [infinispan-dev] Rest server storage nuances
In-Reply-To: <CAH8Ud1G6Hr6VfOK9NhgWpRRP7Ut8yqZmBHLNXdCai57noCSW0w@mail.gmail.com>
References: <CAH8Ud1G6Hr6VfOK9NhgWpRRP7Ut8yqZmBHLNXdCai57noCSW0w@mail.gmail.com>
Message-ID: <CAA9R9O5sSwVxiQKFghWigsmrXxY+6y=+Xe2uHbBNeffFsmOCXA@mail.gmail.com>

On Thu, Aug 10, 2017 at 6:24 PM Gustavo Fernandes <gustavo at infinispan.org>
wrote:

> Hi all,
>
> With the ongoing encoding revamp work on Infinispan, time to decide how to
> handle the rest server.
>
> The rest server currently stores along with each entry, a string
> representing the MimeType of that entry, which allows the user to POST/PUT
> each entry with its own format.
>
> At request time, using the accept header, the user can request the entry
> in a particular format, and the rest server internally extracts the mime
> type from the entry and converts it accordingly.
>
> The issue with this approach is, apart from the extra space required, it
> makes it challenging to expose via rest anything less trivial than put/get.
> Think for example querying, consistent hash calculations, stream
> operations: all those features will have a hard time dealing with a cache
> where each entry has a different format.
>

+1. When I was refactoring REST interface it seemed a little bit strange to
me that we perform this type of conversion. I'm all for removing it and
simplifying the code.


>
> Proposal:
>
> Remove this behavior completely. For a certain cache, all entries will be
> homogeneous, just like
> Hot Rod, Memcached and embedded. The user can optionally configure the
> MimeType at cache level.
>
> Impacts:
>
> - It may be required for users to add the media type configuration to the
> cache, if supporting
> multiple formats is required;
>

Sounds good to me. I'm just curious, what would happen if a cache is
configured to store Strings and I will POST binary content? Will I get HTTP
4XX?


> - Migrating from 9.1 to 9.2 may require re-populating the cache if
> multiple formats are being used;
>

I think this should be a part of a Remote Cache Store. This way we will
support it from Rolling Upgrade perspective.


> - From the API perspective, I don't expect any change: POST/PUT with
> multiple formats should be still supported, as the internal transcoding
> should convert them on the fly to the unified configured format.
>
>
> Thoughts?
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170822/2543f74f/attachment.html 

From gustavo at infinispan.org  Tue Aug 22 03:50:32 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Tue, 22 Aug 2017 08:50:32 +0100
Subject: [infinispan-dev] Rest server storage nuances
In-Reply-To: <CAA9R9O5sSwVxiQKFghWigsmrXxY+6y=+Xe2uHbBNeffFsmOCXA@mail.gmail.com>
References: <CAH8Ud1G6Hr6VfOK9NhgWpRRP7Ut8yqZmBHLNXdCai57noCSW0w@mail.gmail.com>
	<CAA9R9O5sSwVxiQKFghWigsmrXxY+6y=+Xe2uHbBNeffFsmOCXA@mail.gmail.com>
Message-ID: <CAH8Ud1HGwajXJaxJ4O96vW8wTdcBh8rnDuaP0Az_+pFcDyYEwg@mail.gmail.com>

> Sounds good to me. I'm just curious, what would happen if a cache is
> configured to store Strings and I will POST binary content? Will I get HTTP
> 4XX?
>


The post will succeed. But if the user asks for a different content type,
Infinispan will assume the content is text/plain and will use it to convert
to the required format.


>
>
>> - Migrating from 9.1 to 9.2 may require re-populating the cache if
>> multiple formats are being used;
>>
>
> I think this should be a part of a Remote Cache Store. This way we will
> support it from Rolling Upgrade perspective.
>

+1


Gustavo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170822/bfcc1d83/attachment.html 

From sanne at infinispan.org  Wed Aug 23 06:31:50 2017
From: sanne at infinispan.org (Sanne Grinovero)
Date: Wed, 23 Aug 2017 11:31:50 +0100
Subject: [infinispan-dev] Object instance reuse for reference data during
	Marshalling
Message-ID: <CAFm4XO0EZOE32X9HrWx6=1v+Ngoztpy5gPvD0a950=bYePCr2A@mail.gmail.com>

Hi all,

we discussed this need a while back [1] and there's great news: Galder
created a demo showing how this might work on github:
 - https://github.com/galderz/shared-object-spaces

In case you want to better understand what we're after I'd suggest to
have a look: the code is trivial to read and the example is spot-on.

This single test is the goal and the setup should be self-explanatory:
 - https://github.com/galderz/shared-object-spaces/blob/master/src/main/java/sos/SharedObjectSpaces.java#L38

The magic is of course in how the Externalizers are allowed to keep a
registry of instances, essentially to not be fully stateless anymore.

I will now explore building this capability in Hibernate OGM, as some
performance tests had shown that without this feature the amount of
data was "balooning" unexpectedly - especially when some object is
frequently read/written and there's no much variation. In short, we
need to reap the benefits of a properly normalized data structure.

Some questions for further evolution:

# Injection framework

I don't love this need to produce serialized metadata at build time
and have to include it in our jars. I have just been playing with the
demo to see if I can avoid the injection but I failed and can see that
it's necessary for now.

Essentially I'd prefer to have the CountryFactory *instance* to be
created and managed by OGM before bootstrapping Infinispan, and then
inject the *instance* into the componentregistry. I guess I'm just
asking if we'll eventually be able to inject actual instances in the
component registry rather than injecting either FQN or Class
instances?

Today I need to:

gcr.registerComponent(countryFactory, CountryFactory.class);

I'd love to do:

gcr.registerComponent(countryFactory, countryFactoryInstance);

This would allow me to look up the CountryFactory instance from the
componentRegistry in various other integration points, like during the
LifecycleCallbacks methods where I'd register the custom
externalizers.
[Incidentally I believe I asked the same capability for other
components, like the Transport to be injected by instance as in some
situations I'd want to configure JGroups programmatically].

# Cache scoping

These components are global now as the Externalizer registry is
global. I understand the need to have global externalizers for
Infinispan itself, but when it comes to user data encoding - which is
cache specific - could we separate this?
I'm guessing this would help with multi-tenancy designs and with
sharing of CacheManagers across different deployments (classloaders)
too.
Or - like we also asked some while back - to allow independent
CacheManagers to share some core components like the Transport. It's
another long time debate which I'd really hope to see improved.

# Size management in the registry implementations

This example code uses a simple hashmap. Clearly a real solution would
need a bounded container, possibly with some eviction or allow
explicit eviction control to the application.
I will need a way to either inject a local, bounded Cache instance
within this component, or possibly programmatically just create a
simple cache out of the scope of the typical CacheManager ?

I'm also wondering if this Cache should use weak references.

The primary use case in OGM is for trivially small and bounded data
(metadata identifiers) so this is not a problem for my specific use
case, I'm merely mentioning it as others might need this.

# Requirements?

OGM is still using Infinispan 8. Do you think I could try this out
already or should we finish the upgrade to Infinispan 9 first? The
demo seems to work with Infinispan 8.2 as well: just wondering if I'm
missing something.

Thanks!
Sanne

[1] - https://issues.jboss.org/browse/ISPN-2133 ,
http://lists.jboss.org/pipermail/infinispan-dev/2012-June/010925.html
, http://lists.jboss.org/pipermail/infinispan-dev/2016-April/016602.html

From slaskawi at redhat.com  Mon Aug 28 07:41:28 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Mon, 28 Aug 2017 11:41:28 +0000
Subject: [infinispan-dev] A tool for adjusting configuration
Message-ID: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>

Hey,

Our cloud integration bits require a tool for adjusting the configuration
for certain use cases. A common example would be - take this `cloud.xml`
file, remove all caches, add a new, replicated cache as default one.

The tool should take either configuration or a file name as input (e.g.
`config-tool --add-default-cache -f cloud.xml` or `cat cloud.xml |
config-tool --add-default-cache > cloud-new.xml`) and print out
configuration either to System Out or to a file.

Do you have any ideas what could I use to write such a tool? Those
technologies come into my mind:

   - Perl
   - Python
   - Java (probably with some XPath library)

Thoughts? Ideas? Recommendations?

Thanks,
Sebastian
-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170828/41d051cc/attachment.html 

From karesti at redhat.com  Mon Aug 28 10:05:32 2017
From: karesti at redhat.com (Katia Aresti)
Date: Mon, 28 Aug 2017 16:05:32 +0200
Subject: [infinispan-dev] A tool for adjusting configuration
In-Reply-To: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
References: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
Message-ID: <CAKypdvmvAX_bTKZtB4Gi2cZ8Bnrux=p9sSfR+y6eH6+K0bx99w@mail.gmail.com>

Hi !

What about Golang ?

Cheers,

Katia

On Mon, Aug 28, 2017 at 1:41 PM, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> Hey,
>
> Our cloud integration bits require a tool for adjusting the configuration
> for certain use cases. A common example would be - take this `cloud.xml`
> file, remove all caches, add a new, replicated cache as default one.
>
> The tool should take either configuration or a file name as input (e.g.
> `config-tool --add-default-cache -f cloud.xml` or `cat cloud.xml |
> config-tool --add-default-cache > cloud-new.xml`) and print out
> configuration either to System Out or to a file.
>
> Do you have any ideas what could I use to write such a tool? Those
> technologies come into my mind:
>
>    - Perl
>    - Python
>    - Java (probably with some XPath library)
>
> Thoughts? Ideas? Recommendations?
>
> Thanks,
> Sebastian
> --
>
> SEBASTIAN ?ASKAWIEC
>
> INFINISPAN DEVELOPER
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170828/b07efaf3/attachment-0001.html 

From galder at redhat.com  Mon Aug 28 11:00:18 2017
From: galder at redhat.com (=?utf-8?Q?Galder_Zamarre=C3=B1o?=)
Date: Mon, 28 Aug 2017 17:00:18 +0200
Subject: [infinispan-dev] A tool for adjusting configuration
In-Reply-To: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
References: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
Message-ID: <DAD9A525-2808-4278-BA39-CE62D811E05B@redhat.com>

Before you start messing with XML itself, you might want to see what we do in Hibernate 2L.

We first load the XML configuration [1], and using the ConfigurationBuilderHolder we can swap cache configurations, we can tweak them...etc, and eventually we create a cache manager out of that.

>From the tweaked configuration, you could potentiallt chuck out XML. That's a better approach IMO than using XPath or similar tech.

Cheers,

[1] https://github.com/infinispan/infinispan/blob/master/hibernate-cache/src/main/java/org/infinispan/hibernate/cache/InfinispanRegionFactory.java#L547

> On 28 Aug 2017, at 13:41, Sebastian Laskawiec <slaskawi at redhat.com> wrote:
> 
> Hey,
> 
> Our cloud integration bits require a tool for adjusting the configuration for certain use cases. A common example would be - take this `cloud.xml` file, remove all caches, add a new, replicated cache as default one.
> 
> The tool should take either configuration or a file name as input (e.g. `config-tool --add-default-cache -f cloud.xml` or `cat cloud.xml | config-tool --add-default-cache > cloud-new.xml`) and print out configuration either to System Out or to a file.
> 
> Do you have any ideas what could I use to write such a tool? Those technologies come into my mind:
> 	? Perl
> 	? Python
> 	? Java (probably with some XPath library)
> Thoughts? Ideas? Recommendations?
> 
> Thanks,
> Sebastian
> -- 
> SEBASTIAN ?ASKAWIEC
> INFINISPAN DEVELOPER
> Red Hat EMEA
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarre?o
Infinispan, Red Hat



From dan.berindei at gmail.com  Mon Aug 28 11:35:33 2017
From: dan.berindei at gmail.com (Dan Berindei)
Date: Mon, 28 Aug 2017 18:35:33 +0300
Subject: [infinispan-dev] A tool for adjusting configuration
In-Reply-To: <CAKypdvmvAX_bTKZtB4Gi2cZ8Bnrux=p9sSfR+y6eH6+K0bx99w@mail.gmail.com>
References: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
	<CAKypdvmvAX_bTKZtB4Gi2cZ8Bnrux=p9sSfR+y6eH6+K0bx99w@mail.gmail.com>
Message-ID: <CA+nfvwRK4kDvQm7m6PEGM2B12PLjvCJJr0=3vVSBLiOUYq4Lgg@mail.gmail.com>

I suggest Java, any other language will be used as an excuse to leave all
the maintenance work to you :)

Dan


On Mon, Aug 28, 2017 at 5:05 PM, Katia Aresti <karesti at redhat.com> wrote:

> Hi !
>
> What about Golang ?
>
> Cheers,
>
> Katia
>
> On Mon, Aug 28, 2017 at 1:41 PM, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
>
>> Hey,
>>
>> Our cloud integration bits require a tool for adjusting the configuration
>> for certain use cases. A common example would be - take this `cloud.xml`
>> file, remove all caches, add a new, replicated cache as default one.
>>
>> The tool should take either configuration or a file name as input (e.g.
>> `config-tool --add-default-cache -f cloud.xml` or `cat cloud.xml |
>> config-tool --add-default-cache > cloud-new.xml`) and print out
>> configuration either to System Out or to a file.
>>
>> Do you have any ideas what could I use to write such a tool? Those
>> technologies come into my mind:
>>
>>    - Perl
>>    - Python
>>    - Java (probably with some XPath library)
>>
>> Thoughts? Ideas? Recommendations?
>>
>> Thanks,
>> Sebastian
>> --
>>
>> SEBASTIAN ?ASKAWIEC
>>
>> INFINISPAN DEVELOPER
>>
>> Red Hat EMEA <https://www.redhat.com/>
>> <https://red.ht/sig>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170828/1db356da/attachment.html 

From ttarrant at redhat.com  Mon Aug 28 12:56:46 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Mon, 28 Aug 2017 18:56:46 +0200
Subject: [infinispan-dev] A tool for adjusting configuration
In-Reply-To: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
References: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
Message-ID: <ab91efc0-c0a7-1232-c230-951411e76ef8@redhat.com>

For server the tool already exists: the server CLI can work in offline 
mode and manipulate a configuration using DMR ops.

Tristan

On 8/28/17 1:41 PM, Sebastian Laskawiec wrote:
> Hey,
> 
> Our cloud integration bits require a tool for adjusting the 
> configuration for certain use cases. A common example would be - take 
> this `cloud.xml` file, remove all caches, add a new, replicated cache as 
> default one.
> 
> The tool should take either configuration or a file name as input (e.g. 
> `config-tool --add-default-cache -f cloud.xml` or `cat cloud.xml | 
> config-tool --add-default-cache > cloud-new.xml`) and print out 
> configuration either to System Out or to a file.
> 
> Do you have any ideas what could I use to write such a tool? Those 
> technologies come into my mind:
> 
>   * Perl
>   * Python
>   * Java (probably with some XPath library)
> 
> Thoughts? Ideas? Recommendations?
> 
> Thanks,
> Sebastian
> -- 
> 
> SEBASTIAN?ASKAWIEC
> 
> INFINISPAN DEVELOPER
> 
> Red HatEMEA <https://www.redhat.com/>
> 
> <https://red.ht/sig>
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From slaskawi at redhat.com  Tue Aug 29 02:28:22 2017
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Tue, 29 Aug 2017 06:28:22 +0000
Subject: [infinispan-dev] A tool for adjusting configuration
In-Reply-To: <ab91efc0-c0a7-1232-c230-951411e76ef8@redhat.com>
References: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
	<ab91efc0-c0a7-1232-c230-951411e76ef8@redhat.com>
Message-ID: <CAA9R9O45X_B3d5G7Hh-AWafHH9v-OQUh=ptdDy2Qesm1Mtk0eQ@mail.gmail.com>

Thanks a lot for all tips.

It seems Tristan's solution fits the best. I've just tried it out and it
seems to do the job:
> ./cli.sh
> embed-server --server-config=cloud.xml --std-out=echo
> /subsystem=datagrid-infinispan-endpoint/rest-connector=rest-connector:remove
> stop-embedded-server

I might wrap some of the commands with a simple bash script (e.g. when I
need to incorporate it into larger commands including grep, awk or sed) but
it's more or less what I needed.

Thanks,
Sebastian


On Mon, Aug 28, 2017 at 7:46 PM Tristan Tarrant <ttarrant at redhat.com> wrote:

> For server the tool already exists: the server CLI can work in offline
> mode and manipulate a configuration using DMR ops.
>
> Tristan
>
> On 8/28/17 1:41 PM, Sebastian Laskawiec wrote:
> > Hey,
> >
> > Our cloud integration bits require a tool for adjusting the
> > configuration for certain use cases. A common example would be - take
> > this `cloud.xml` file, remove all caches, add a new, replicated cache as
> > default one.
> >
> > The tool should take either configuration or a file name as input (e.g.
> > `config-tool --add-default-cache -f cloud.xml` or `cat cloud.xml |
> > config-tool --add-default-cache > cloud-new.xml`) and print out
> > configuration either to System Out or to a file.
> >
> > Do you have any ideas what could I use to write such a tool? Those
> > technologies come into my mind:
> >
> >   * Perl
> >   * Python
> >   * Java (probably with some XPath library)
> >
> > Thoughts? Ideas? Recommendations?
> >
> > Thanks,
> > Sebastian
> > --
> >
> > SEBASTIAN?ASKAWIEC
> >
> > INFINISPAN DEVELOPER
> >
> > Red HatEMEA <https://www.redhat.com/>
> >
> > <https://red.ht/sig>
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 

SEBASTIAN ?ASKAWIEC

INFINISPAN DEVELOPER

Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170829/22de6cdb/attachment.html 

From ttarrant at redhat.com  Tue Aug 29 08:00:58 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Tue, 29 Aug 2017 14:00:58 +0200
Subject: [infinispan-dev] Infinispan weekly IRC meeting log 2017-08-29
Message-ID: <cb3eb8d0-1106-663f-579e-c0e57d3b9bb3@redhat.com>

Hey all,

because the bot was down, I am pasting them as a Gist:

https://gist.github.com/tristantarrant/ee46a7c392001e53e086fec430c929e2

Tristan
-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From ttarrant at redhat.com  Tue Aug 29 08:01:07 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Tue, 29 Aug 2017 14:01:07 +0200
Subject: [infinispan-dev] Infinispan weekly IRC meeting log 2017-08-29
Message-ID: <8e35dc3d-f54e-4d82-132b-78d0a54b0598@redhat.com>

Hey all,

because the bot was down, I am pasting the logs as a Gist:

https://gist.github.com/tristantarrant/ee46a7c392001e53e086fec430c929e2

Tristan
-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat

From belaban at mailbox.org  Wed Aug 30 01:38:18 2017
From: belaban at mailbox.org (Bela Ban)
Date: Wed, 30 Aug 2017 07:38:18 +0200
Subject: [infinispan-dev] JGroups workshops in Berlin and Rome
Message-ID: <503bb32f-7afd-a19e-d5bf-c75b6ebd89b2@mailbox.org>

Hi all,

I've revamped the JGroups workshop and will be teaching it in Berlin and 
Rome in November.

JGroups [1] is a clustering library and is used by JBoss Data Grid (JDG) 
/ Infinispan, EAP/Wildfly clustering, Hibernate Search etc.

To sign up, use [2] and EARLYBIRD as promotion code to get the discount. 
If you're Red Hat employee, contact me offline for a bigger discount... :-)
Cheers,

[1] http://www.jgroups.org
[2] http://www.jgroups.org/workshops.html

-- 
Bela Ban | http://www.jgroups.org


From emmanuel at hibernate.org  Wed Aug 30 10:42:30 2017
From: emmanuel at hibernate.org (Emmanuel Bernard)
Date: Wed, 30 Aug 2017 16:42:30 +0200
Subject: [infinispan-dev] A tool for adjusting configuration
In-Reply-To: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
References: <CAA9R9O6JyHCvAHCLoUno5MFTqDiujTqG_-OujtXuN3fzRFBCLw@mail.gmail.com>
Message-ID: <20170830144230.GC29232@hibernate.org>

If built-in solutions like what Tristan proposed don't work. Copnsider
Windup, it must likely have ways to transform configurations.
Windup is a generic tool to migrate from some version of X to some
versions of Y. Worth talking to the team.

Emmanuel

On Mon 17-08-28 11:41, Sebastian Laskawiec wrote:
>Hey,
>
>Our cloud integration bits require a tool for adjusting the configuration
>for certain use cases. A common example would be - take this `cloud.xml`
>file, remove all caches, add a new, replicated cache as default one.
>
>The tool should take either configuration or a file name as input (e.g.
>`config-tool --add-default-cache -f cloud.xml` or `cat cloud.xml |
>config-tool --add-default-cache > cloud-new.xml`) and print out
>configuration either to System Out or to a file.
>
>Do you have any ideas what could I use to write such a tool? Those
>technologies come into my mind:
>
>   - Perl
>   - Python
>   - Java (probably with some XPath library)
>
>Thoughts? Ideas? Recommendations?
>
>Thanks,
>Sebastian
>-- 
>
>SEBASTIAN ?ASKAWIEC
>
>INFINISPAN DEVELOPER
>
>Red Hat EMEA <https://www.redhat.com/>
><https://red.ht/sig>

>_______________________________________________
>infinispan-dev mailing list
>infinispan-dev at lists.jboss.org
>https://lists.jboss.org/mailman/listinfo/infinispan-dev


From avaccarezza at c2b2.co.uk  Wed Aug 30 13:16:25 2017
From: avaccarezza at c2b2.co.uk (Alessandro Vaccarezza)
Date: Wed, 30 Aug 2017 17:16:25 +0000
Subject: [infinispan-dev] infinispan cache retrieval - byte[]
Message-ID: <HE1PR0301MB252211FFEC174F41185A792B889C0@HE1PR0301MB2522.eurprd03.prod.outlook.com>

Hi guys,

we ar facing an issue since quite a long time and can't come up with a solution.

Our simple application is deployed on a wildfly instance and make use of a local infinispan data store (infinispan-9.1.0.Final).

Wildfly resides on a CentOS 6.8 instance, Java version is 1.8.0_112.

We implemented a very basic servlet and registered as a listener. Within this servlet we use an org.infinispan.manager.EmbeddedCacheManager to retrieve an
existing cache and add values to it.

Compatibility mode is on.

The values added to the cache are instance of a simple Java POJO.

When we try to retrieve what we just added, within the same servlet, we get a byte array, no way to retrieve the original POJO.
When we force the cast we get the obvious:

java.lang.ClassCastException: [B cannot be cast to com.my.pojo.SamplePOJO

Servlet: https://gist.github.com/anonymous/e8752cc8457301c2c70b3255e5c3bbc5
POJO: https://gist.github.com/anonymous/f35900875500a4bd6b4bb3b0cf73a363
Infinispan config file: https://gist.github.com/anonymous/b8482595f6b86ecc5642cceb6a99ad5e

Retrieving the cache via Hot Rod client works fine; also, if compatibility mode is turned off everything works fine as well.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170830/0249d94d/attachment.html 

From gustavo at infinispan.org  Wed Aug 30 17:35:13 2017
From: gustavo at infinispan.org (Gustavo Fernandes)
Date: Wed, 30 Aug 2017 22:35:13 +0100
Subject: [infinispan-dev] infinispan cache retrieval - byte[]
In-Reply-To: <HE1PR0301MB252211FFEC174F41185A792B889C0@HE1PR0301MB2522.eurprd03.prod.outlook.com>
References: <HE1PR0301MB252211FFEC174F41185A792B889C0@HE1PR0301MB2522.eurprd03.prod.outlook.com>
Message-ID: <CAH8Ud1Fje_t-HmH0atQ41xOq1wkoGZjOynEyB9HsO-vMrQbsXA@mail.gmail.com>

Hi Alessandro, this kind of question is better suited for the forums [1],
since this list if for development related discussions.

But since we're here, regarding your question:

What's probably happening is that since you are creating a cache inside the
servlet from the cache manager that is also used by the Infinispan server,
this cache is internally being created with an encoding scheme suitable to
used for remote clients, meaning storage will be POJO but reads/writes will
be done using byte[].

In Infinispan 9.1.0 you can choose the encoding to read and write from a
particular cache: so in your case, from you servlet, you can specify that
you don't want
any encoding by wrapping your cache with:

Cache c = cache.getAdvancedCache().withEncoding(IdentityEncoder.class)

[1] http://infinispan.org/community/

On Wed, Aug 30, 2017 at 6:16 PM, Alessandro Vaccarezza <
avaccarezza at c2b2.co.uk> wrote:

> Hi guys,
>
>
>
> we ar facing an issue since quite a long time and can't come up with a
> solution.
>
>
>
> Our simple application is deployed on a wildfly instance and make use of a
> local infinispan data store (infinispan-9.1.0.Final).
>
>
>
> Wildfly resides on a CentOS 6.8 instance, Java version is 1.8.0_112.
>
>
>
> We implemented a very basic servlet and registered as a listener. Within
> this servlet we use an org.infinispan.manager.EmbeddedCacheManager to
> retrieve an
>
> existing cache and add values to it.
>
>
>
> Compatibility mode is on.
>
>
>
> The values added to the cache are instance of a simple Java POJO.
>
>
>
> When we try to retrieve what we just added, within the same servlet, we
> get a byte array, no way to retrieve the original POJO.
>
> When we force the cast we get the obvious:
>
>
>
> *java.lang.ClassCastException: [B cannot be cast to com.my.pojo.SamplePOJO
> *
>
>
>
> Servlet: https://gist.github.com/anonymous/e8752cc8457301c2c70b3255e5c3bb
> c5
>
> POJO: https://gist.github.com/anonymous/f35900875500a4bd6b4bb3b0cf73a363
>
> Infinispan config file: https://gist.github.com/anonymous/
> b8482595f6b86ecc5642cceb6a99ad5e
>
>
>
> Retrieving the cache via Hot Rod client works fine; also, if compatibility
> mode is turned off everything works fine as well.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170830/71467bf0/attachment-0001.html 

From avaccarezza at c2b2.co.uk  Thu Aug 31 05:19:15 2017
From: avaccarezza at c2b2.co.uk (Alessandro Vaccarezza)
Date: Thu, 31 Aug 2017 09:19:15 +0000
Subject: [infinispan-dev] infinispan cache retrieval - byte[]
In-Reply-To: <CAH8Ud1Fje_t-HmH0atQ41xOq1wkoGZjOynEyB9HsO-vMrQbsXA@mail.gmail.com>
References: <HE1PR0301MB252211FFEC174F41185A792B889C0@HE1PR0301MB2522.eurprd03.prod.outlook.com>
	<CAH8Ud1Fje_t-HmH0atQ41xOq1wkoGZjOynEyB9HsO-vMrQbsXA@mail.gmail.com>
Message-ID: <HE1PR0301MB25229579EC3FE4E050F17A86889D0@HE1PR0301MB2522.eurprd03.prod.outlook.com>

Hi Gustavo and thank you, your solution is indeed sorting things out.

Regards,

Alessandro

From: infinispan-dev-bounces at lists.jboss.org [mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Gustavo Fernandes
Sent: August 30, 2017 23:35
To: infinispan -Dev List <infinispan-dev at lists.jboss.org>
Subject: Re: [infinispan-dev] infinispan cache retrieval - byte[]

Hi Alessandro, this kind of question is better suited for the forums [1], since this list if for development related discussions.

But since we're here, regarding your question:

What's probably happening is that since you are creating a cache inside the servlet from the cache manager that is also used by the Infinispan server,
this cache is internally being created with an encoding scheme suitable to used for remote clients, meaning storage will be POJO but reads/writes will
be done using byte[].

In Infinispan 9.1.0 you can choose the encoding to read and write from a particular cache: so in your case, from you servlet, you can specify that you don't want
any encoding by wrapping your cache with:

Cache c = cache.getAdvancedCache().withEncoding(IdentityEncoder.class)

[1] http://infinispan.org/community/

On Wed, Aug 30, 2017 at 6:16 PM, Alessandro Vaccarezza <avaccarezza at c2b2.co.uk<mailto:avaccarezza at c2b2.co.uk>> wrote:
Hi guys,

we ar facing an issue since quite a long time and can't come up with a solution.

Our simple application is deployed on a wildfly instance and make use of a local infinispan data store (infinispan-9.1.0.Final).

Wildfly resides on a CentOS 6.8 instance, Java version is 1.8.0_112.

We implemented a very basic servlet and registered as a listener. Within this servlet we use an org.infinispan.manager.EmbeddedCacheManager to retrieve an
existing cache and add values to it.

Compatibility mode is on.

The values added to the cache are instance of a simple Java POJO.

When we try to retrieve what we just added, within the same servlet, we get a byte array, no way to retrieve the original POJO.
When we force the cast we get the obvious:

java.lang.ClassCastException: [B cannot be cast to com.my.pojo.SamplePOJO

Servlet: https://gist.github.com/anonymous/e8752cc8457301c2c70b3255e5c3bbc5
POJO: https://gist.github.com/anonymous/f35900875500a4bd6b4bb3b0cf73a363
Infinispan config file: https://gist.github.com/anonymous/b8482595f6b86ecc5642cceb6a99ad5e

Retrieving the cache via Hot Rod client works fine; also, if compatibility mode is turned off everything works fine as well.

_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org<mailto:infinispan-dev at lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/infinispan-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170831/2b8332f9/attachment.html 

From ttarrant at redhat.com  Thu Aug 31 09:50:39 2017
From: ttarrant at redhat.com (Tristan Tarrant)
Date: Thu, 31 Aug 2017 15:50:39 +0200
Subject: [infinispan-dev] Current 'master' status
Message-ID: <256c0f1c-2b21-773a-276d-107e9e3d7927@redhat.com>

Dear all,

it's now been a while since we've embarked on a mission to kill all 
those pesky unreliable tests and, while we've achieved quite some 
progress, we're still not in an ideal and reliable situation.

I think we are quite close with the latest round of fixes, but there is 
still one outstanding offender:

ISPN-6827 ReplTotalOrderVersionedStateTransferTest.testStateTransfer 
random failures

Any other failures you wish to single out ?

I propose we move this to the unstable group, merge some of the pending 
PRs and release 9.1.1 and branch for 9.2. At this point I'll rework the 
9.x roadmap and schedule.

At this point we can reopen all of the PRs that were closed: make sure 
you open them before rebasing, otherwise GitHub will complain.



Tristan






-- 
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat