11:41 p.m.
Chasing down today's NPE: http://pastie.org/7382216
I've added this test:
* https://github.com/capedwarf/capedwarf-blue/commit/d416d5358
The test fails. ;-)
deleteAndQueryInA(org.jboss.test.capedwarf.cluster.test.QueryTest): Should not be here:
null
This is the "null" that is the cause of top NPE.
-Ales
2:06 p.m.
Dan worked to understand the problem, here is where we stand without
Sanne's additional knowledge.
The test failing is the following:
tx1.start();
cache.remove(key);
tx1.commit();
assert queryDeletedKey.getResults().size() == 0;
Surprisingly, the entity is still found in the query and Infinispan
Query returns `null`.
The reason we suspect is the following. While pushing the index changes
is done in sync with the transaction commit (tx sync), the backend is
JGroups and returns right after it has sent the message to the master
node without waiting for the index ack.
This is a good thing as it scales much better but has the nice
side effect of returning incorrect query results right after a change
set is applied (indexing window + index propagation).
My recommendation is to make sure Infinispan Query removes entities
found by the query engine but not present in the grid like we do in
Hibernate Search for the ORM case. Note that this does not solve the
case were you add a new entity and expect to query it *right after the
commit*. There is a similar issue around entity changes. If CapeDwarf
can live with it it will be better off.
An alternative solution would be to create a JGroups backend that waits
for an execution ACK before giving back the had to the client but that
will have scalability problems.
Emmanuel
On Tue 2013-04-09 23:41, Ales Justin wrote:
Chasing down today's NPE: http://pastie.org/7382216
I've added this test:
* https://github.com/capedwarf/capedwarf-blue/commit/d416d5358
The test fails. ;-)
deleteAndQueryInA(org.jboss.test.capedwarf.cluster.test.QueryTest): Should not be here:
null
This is the "null" that is the cause of top NPE.
-Ales
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
2:38 p.m.
New subject: Fwd: query npe
@Bela: is there a config way to make this JGroups backend wait for ACK?
-Ales
Begin forwarded message:
From: Emmanuel Bernard <emmanuel(a)hibernate.org>
Subject: Re: [infinispan-dev] query npe
Date: April 10, 2013 2:06:20 PM GMT+02:00
To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
Reply-To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
Dan worked to understand the problem, here is where we stand without
Sanne's additional knowledge.
The test failing is the following:
tx1.start();
cache.remove(key);
tx1.commit();
assert queryDeletedKey.getResults().size() == 0;
Surprisingly, the entity is still found in the query and Infinispan
Query returns `null`.
The reason we suspect is the following. While pushing the index changes
is done in sync with the transaction commit (tx sync), the backend is
JGroups and returns right after it has sent the message to the master
node without waiting for the index ack.
This is a good thing as it scales much better but has the nice
side effect of returning incorrect query results right after a change
set is applied (indexing window + index propagation).
My recommendation is to make sure Infinispan Query removes entities
found by the query engine but not present in the grid like we do in
Hibernate Search for the ORM case. Note that this does not solve the
case were you add a new entity and expect to query it *right after the
commit*. There is a similar issue around entity changes. If CapeDwarf
can live with it it will be better off.
An alternative solution would be to create a JGroups backend that waits
for an execution ACK before giving back the had to the client but that
will have scalability problems.
Emmanuel
On Tue 2013-04-09 23:41, Ales Justin wrote:
> Chasing down today's NPE: http://pastie.org/7382216
>
> I've added this test:
> * https://github.com/capedwarf/capedwarf-blue/commit/d416d5358
>
> The test fails. ;-)
>
> deleteAndQueryInA(org.jboss.test.capedwarf.cluster.test.QueryTest): Should not be
here: null
>
> This is the "null" that is the cause of top NPE.
>
> -Ales
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
2:55 p.m.
New subject: Fwd: query npe
Ales, good point your Channel might be configured with fire-and-forget
messages ?
Can you share the JGroups configuration of the Channel you are
injecting in Search ?
The backend implementation delegates to the IndexManager at the
receiver side in the same thread processing the incoming Message;
that is unless the IM itself is configured to process the index
updates asynchronously, the backend from the client side should block
until the index is updated.
Sanne
On 10 April 2013 13:38, Ales Justin <ales.justin(a)gmail.com> wrote:
@Bela: is there a config way to make this JGroups backend wait for
ACK?
-Ales
Begin forwarded message:
From: Emmanuel Bernard <emmanuel(a)hibernate.org>
Subject: Re: [infinispan-dev] query npe
Date: April 10, 2013 2:06:20 PM GMT+02:00
To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
Reply-To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
Dan worked to understand the problem, here is where we stand without
Sanne's additional knowledge.
The test failing is the following:
tx1.start();
cache.remove(key);
tx1.commit();
assert queryDeletedKey.getResults().size() == 0;
Surprisingly, the entity is still found in the query and Infinispan
Query returns `null`.
The reason we suspect is the following. While pushing the index changes
is done in sync with the transaction commit (tx sync), the backend is
JGroups and returns right after it has sent the message to the master
node without waiting for the index ack.
This is a good thing as it scales much better but has the nice
side effect of returning incorrect query results right after a change
set is applied (indexing window + index propagation).
My recommendation is to make sure Infinispan Query removes entities
found by the query engine but not present in the grid like we do in
Hibernate Search for the ORM case. Note that this does not solve the
case were you add a new entity and expect to query it *right after the
commit*. There is a similar issue around entity changes. If CapeDwarf
can live with it it will be better off.
An alternative solution would be to create a JGroups backend that waits
for an execution ACK before giving back the had to the client but that
will have scalability problems.
Emmanuel
On Tue 2013-04-09 23:41, Ales Justin wrote:
Chasing down today's NPE: http://pastie.org/7382216
I've added this test:
* https://github.com/capedwarf/capedwarf-blue/commit/d416d5358
The test fails. ;-)
deleteAndQueryInA(org.jboss.test.capedwarf.cluster.test.QueryTest): Should
not be here: null
This is the "null" that is the cause of top NPE.
-Ales
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
4:02 p.m.
Ales, good point your Channel might be configured with
fire-and-forget
messages ?
Can you share the JGroups configuration of the Channel you are
injecting in Search ?
https://github.com/capedwarf/capedwarf-jboss-as/blob/master/build/src/mai...
and
https://github.com/capedwarf/capedwarf-jboss-as/blob/master/extension/src...
-Ales
The backend implementation delegates to the IndexManager at the
receiver side in the same thread processing the incoming Message;
that is unless the IM itself is configured to process the index
updates asynchronously, the backend from the client side should block
until the index is updated.
Sanne
On 10 April 2013 13:38, Ales Justin <ales.justin(a)gmail.com> wrote:
> @Bela: is there a config way to make this JGroups backend wait for ACK?
>
> -Ales
>
> Begin forwarded message:
>
> From: Emmanuel Bernard <emmanuel(a)hibernate.org>
> Subject: Re: [infinispan-dev] query npe
> Date: April 10, 2013 2:06:20 PM GMT+02:00
> To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> Reply-To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
>
> Dan worked to understand the problem, here is where we stand without
> Sanne's additional knowledge.
>
> The test failing is the following:
>
> tx1.start();
> cache.remove(key);
> tx1.commit();
> assert queryDeletedKey.getResults().size() == 0;
>
> Surprisingly, the entity is still found in the query and Infinispan
> Query returns `null`.
>
> The reason we suspect is the following. While pushing the index changes
> is done in sync with the transaction commit (tx sync), the backend is
> JGroups and returns right after it has sent the message to the master
> node without waiting for the index ack.
> This is a good thing as it scales much better but has the nice
> side effect of returning incorrect query results right after a change
> set is applied (indexing window + index propagation).
>
> My recommendation is to make sure Infinispan Query removes entities
> found by the query engine but not present in the grid like we do in
> Hibernate Search for the ORM case. Note that this does not solve the
> case were you add a new entity and expect to query it *right after the
> commit*. There is a similar issue around entity changes. If CapeDwarf
> can live with it it will be better off.
>
> An alternative solution would be to create a JGroups backend that waits
> for an execution ACK before giving back the had to the client but that
> will have scalability problems.
>
> Emmanuel
>
> On Tue 2013-04-09 23:41, Ales Justin wrote:
>
> Chasing down today's NPE: http://pastie.org/7382216
>
> I've added this test:
> * https://github.com/capedwarf/capedwarf-blue/commit/d416d5358
>
> The test fails. ;-)
>
> deleteAndQueryInA(org.jboss.test.capedwarf.cluster.test.QueryTest): Should
> not be here: null
>
> This is the "null" that is the cause of top NPE.
>
> -Ales
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
4:11 p.m.
We continued the inspection on IRC, so for the sake of others this is
the conclusion:
https://hibernate.atlassian.net/browse/HSEARCH-1296
Thanks Dan!
On 10 April 2013 15:02, Ales Justin <ales.justin(a)gmail.com> wrote:
Ales, good point your Channel might be configured with
fire-and-forget
messages ?
Can you share the JGroups configuration of the Channel you are
injecting in Search ?
https://github.com/capedwarf/capedwarf-jboss-as/blob/master/build/src/mai...
and
https://github.com/capedwarf/capedwarf-jboss-as/blob/master/extension/src...
-Ales
The backend implementation delegates to the IndexManager at the
receiver side in the same thread processing the incoming Message;
that is unless the IM itself is configured to process the index
updates asynchronously, the backend from the client side should block
until the index is updated.
Sanne
On 10 April 2013 13:38, Ales Justin <ales.justin(a)gmail.com> wrote:
@Bela: is there a config way to make this JGroups backend wait for ACK?
-Ales
Begin forwarded message:
From: Emmanuel Bernard <emmanuel(a)hibernate.org>
Subject: Re: [infinispan-dev] query npe
Date: April 10, 2013 2:06:20 PM GMT+02:00
To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
Reply-To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
Dan worked to understand the problem, here is where we stand without
Sanne's additional knowledge.
The test failing is the following:
tx1.start();
cache.remove(key);
tx1.commit();
assert queryDeletedKey.getResults().size() == 0;
Surprisingly, the entity is still found in the query and Infinispan
Query returns `null`.
The reason we suspect is the following. While pushing the index changes
is done in sync with the transaction commit (tx sync), the backend is
JGroups and returns right after it has sent the message to the master
node without waiting for the index ack.
This is a good thing as it scales much better but has the nice
side effect of returning incorrect query results right after a change
set is applied (indexing window + index propagation).
My recommendation is to make sure Infinispan Query removes entities
found by the query engine but not present in the grid like we do in
Hibernate Search for the ORM case. Note that this does not solve the
case were you add a new entity and expect to query it *right after the
commit*. There is a similar issue around entity changes. If CapeDwarf
can live with it it will be better off.
An alternative solution would be to create a JGroups backend that waits
for an execution ACK before giving back the had to the client but that
will have scalability problems.
Emmanuel
On Tue 2013-04-09 23:41, Ales Justin wrote:
Chasing down today's NPE: http://pastie.org/7382216
I've added this test:
* https://github.com/capedwarf/capedwarf-blue/commit/d416d5358
The test fails. ;-)
deleteAndQueryInA(org.jboss.test.capedwarf.cluster.test.QueryTest): Should
not be here: null
This is the "null" that is the cause of top NPE.
-Ales
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
3:08 p.m.
New subject: Fwd: query npe
This boils down to using a sync RPC (versus an async one). I don't know
how HS does this (e.g. whether they use RPCs at all)... Sanne might be
able to help
On 4/10/13 2:38 PM, Ales Justin wrote:
@Bela: is there a config way to make this JGroups backend wait for
ACK?
-Ales
Begin forwarded message:
> *From: *Emmanuel Bernard <emmanuel(a)hibernate.org
> <mailto:emmanuel@hibernate.org>>
> *Subject: **Re: [infinispan-dev] query npe*
> *Date: *April 10, 2013 2:06:20 PM GMT+02:00
> *To: *infinispan -Dev List <infinispan-dev(a)lists.jboss.org
> <mailto:infinispan-dev@lists.jboss.org>>
> *Reply-To: *infinispan -Dev List <infinispan-dev(a)lists.jboss.org
> <mailto:infinispan-dev@lists.jboss.org>>
>
> Dan worked to understand the problem, here is where we stand without
> Sanne's additional knowledge.
>
> The test failing is the following:
>
> tx1.start();
> cache.remove(key);
> tx1.commit();
> assert queryDeletedKey.getResults().size() == 0;
>
> Surprisingly, the entity is still found in the query and Infinispan
> Query returns `null`.
>
> The reason we suspect is the following. While pushing the index changes
> is done in sync with the transaction commit (tx sync), the backend is
> JGroups and returns right after it has sent the message to the master
> node without waiting for the index ack.
> This is a good thing as it scales much better but has the nice
> side effect of returning incorrect query results right after a change
> set is applied (indexing window + index propagation).
>
> My recommendation is to make sure Infinispan Query removes entities
> found by the query engine but not present in the grid like we do in
> Hibernate Search for the ORM case. Note that this does not solve the
> case were you add a new entity and expect to query it *right after the
> commit*. There is a similar issue around entity changes. If CapeDwarf
> can live with it it will be better off.
>
> An alternative solution would be to create a JGroups backend that waits
> for an execution ACK before giving back the had to the client but that
> will have scalability problems.
>
> Emmanuel
>
> On Tue 2013-04-09 23:41, Ales Justin wrote:
>> Chasing down today's NPE: http://pastie.org/7382216
>>
>> I've added this test:
>> * https://github.com/capedwarf/capedwarf-blue/commit/d416d5358
>>
>> The test fails. ;-)
>>
>> deleteAndQueryInA(org.jboss.test.capedwarf.cluster.test.QueryTest):
>> Should not be here: null
>>
>> This is the "null" that is the cause of top NPE.
>>
>> -Ales
>>
>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Bela Ban, JGroups lead (http://www.jgroups.org)
4091
days inactive
4092
days old
infinispan-dev@lists.jboss.org
6 comments
4 participants
participants (4)
-
Ales Justin -
Bela Ban -
Emmanuel Bernard -
Sanne Grinovero