July 2014 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-4484) Outbound transfers can be cancelled by old CANCEL_STATE_TRANSFER command

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-4484?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-4484: ------------------------------- Status: Pull Request Sent (was: Open) Git Pull Request: https://github.com/infinispan/infinispan/pull/2697 > Outbound transfers can be cancelled by old CANCEL_STATE_TRANSFER command > ------------------------------------------------------------------------ > > Key: ISPN-4484 > URL: https://issues.jboss.org/browse/ISPN-4484 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Core, State Transfer > Affects Versions: 6.0.2.Final > Reporter: Dan Berindei > Assignee: Dan Berindei > Priority: Critical > Fix For: 7.0.0.Alpha5 > > > This appeared during the 32-nodes elasticity test in the Hyperion environment. > Just as apex947 left, it started a rebalance, which apex948 dutifully cancelled as it became the new coordinator. apex949 had already requested segments from apex959, so it sent a StateRequestCommand(CANCEL_STATE_TRANSFER) asynchronously to apex959. Then apex948 started a new rebalance, and apex949 asked apex959 for the same segments. When apex959 finally received the cancel request, it didn't check the topology id and it incorrectly cancelled the outbound transfer to apex949. > The solution would be to verify the topology id in the CANCEL_STATE_TRANSFER command before cancelling the transfer. I also think we can avoid sending the cancel command completely in this case, and only send it as we are about to stop. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4154) Cancelled segment transfer causes future entry transfer to be ignored

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-4154?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-4154: ------------------------------- Status: Pull Request Sent (was: Open) Git Pull Request: https://github.com/infinispan/infinispan/pull/2697 > Cancelled segment transfer causes future entry transfer to be ignored > --------------------------------------------------------------------- > > Key: ISPN-4154 > URL: https://issues.jboss.org/browse/ISPN-4154 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: State Transfer > Affects Versions: 7.0.0.Alpha1 > Reporter: Radim Vansa > Assignee: Dan Berindei > Priority: Critical > > Distributed transactional cache. > 1) Coordinator is gracefully leaving the cluster, sends a REBALANCE_START with topologyId 14, ST begins. > 2) Node receives chunk from segment X, writes entry K=V to the container. > 3) New coordinator jumps in with CH_UPDATE topology 16 > 4) Node receives CANCEL_STATE_TRANSFER and cancels transfer of segment X, invalidating K. In CommitManager, this operation is tracked and DiscardPolicy is set to DISCARD_STATE_TRANSFER for key K. > 5) New coordinator starts rebalance with topology 17 > 6) Node starts new ST for segment X > 7) Node receives the X: K=V, but in CommitManager it finds out that the policy is set to DISCARD_STATE_TRANSFER and ignores this update. > Result: entry value is lost on some node. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4469) StateConsumerImpl segment change tracing is incorrect

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-4469?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-4469: ------------------------------- Status: Pull Request Sent (was: Open) Git Pull Request: https://github.com/infinispan/infinispan/pull/2697 > StateConsumerImpl segment change tracing is incorrect > ----------------------------------------------------- > > Key: ISPN-4469 > URL: https://issues.jboss.org/browse/ISPN-4469 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Affects Versions: 7.0.0.Alpha4 > Reporter: William Burns > Assignee: Mircea Markus > > StateConsumerImpl has some tracing to tell you what segments had changed. Unfortunately the arguments are in the wrong order and can cause some confusion. > {code} > if (trace) { > log.tracef("On cache %s we have: new segments: %s; old segments: %s; removed segments: %s; added segments: %s", > cacheName, removedSegments, newSegments, previousSegments, addedSegments); > } > {code} > It should be newSegments, previousSegments, removedSegments, addedSegments instead. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4479) Remote executor thread pool configuration is ignored

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-4479?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-4479: ------------------------------- Status: Pull Request Sent (was: Open) Git Pull Request: https://github.com/infinispan/infinispan/pull/2697 > Remote executor thread pool configuration is ignored > ---------------------------------------------------- > > Key: ISPN-4479 > URL: https://issues.jboss.org/browse/ISPN-4479 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Core > Affects Versions: 7.0.0.Alpha4 > Reporter: Dan Berindei > Assignee: Dan Berindei > Fix For: 7.0.0.Alpha5 > > > Currently NamedExecutorsFactory uses the replication queue executor's configuration to build the remote executor's thread pool. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4481) Use UNICAST3 and NAKACK2 in the default server configuration

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-4481?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-4481: ------------------------------- Status: Pull Request Sent (was: Open) Git Pull Request: https://github.com/infinispan/infinispan/pull/2697 > Use UNICAST3 and NAKACK2 in the default server configuration > ------------------------------------------------------------ > > Key: ISPN-4481 > URL: https://issues.jboss.org/browse/ISPN-4481 > Project: Infinispan > Issue Type: Enhancement > Security Level: Public(Everyone can see) > Components: Server > Affects Versions: 6.0.2.Final, 7.0.0.Alpha4 > Reporter: Dan Berindei > Assignee: Dan Berindei > Fix For: 7.0.0.Alpha5 > > > We switched to UNICAST3 a while back in the default embedded configuration, we should switch the server configuration as well. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4154) Cancelled segment transfer causes future entry transfer to be ignored

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4154?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4154: ----------------------------------------------- William Burns <wburns(a)redhat.com> changed the Status of [bug 1104045|https://bugzilla.redhat.com/show_bug.cgi?id=1104045] from ASSIGNED to MODIFIED > Cancelled segment transfer causes future entry transfer to be ignored > --------------------------------------------------------------------- > > Key: ISPN-4154 > URL: https://issues.jboss.org/browse/ISPN-4154 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: State Transfer > Affects Versions: 7.0.0.Alpha1 > Reporter: Radim Vansa > Assignee: Dan Berindei > Priority: Critical > > Distributed transactional cache. > 1) Coordinator is gracefully leaving the cluster, sends a REBALANCE_START with topologyId 14, ST begins. > 2) Node receives chunk from segment X, writes entry K=V to the container. > 3) New coordinator jumps in with CH_UPDATE topology 16 > 4) Node receives CANCEL_STATE_TRANSFER and cancels transfer of segment X, invalidating K. In CommitManager, this operation is tracked and DiscardPolicy is set to DISCARD_STATE_TRANSFER for key K. > 5) New coordinator starts rebalance with topology 17 > 6) Node starts new ST for segment X > 7) Node receives the X: K=V, but in CommitManager it finds out that the policy is set to DISCARD_STATE_TRANSFER and ignores this update. > Result: entry value is lost on some node. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4480) Messages sent to leavers can clog the JGroups bundler thread

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4480?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4480: ----------------------------------------------- William Burns <wburns(a)redhat.com> changed the Status of [bug 1104045|https://bugzilla.redhat.com/show_bug.cgi?id=1104045] from ASSIGNED to MODIFIED > Messages sent to leavers can clog the JGroups bundler thread > ------------------------------------------------------------ > > Key: ISPN-4480 > URL: https://issues.jboss.org/browse/ISPN-4480 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Core > Affects Versions: 6.0.2.Final > Reporter: Dan Berindei > Assignee: Dan Berindei > > In a stress test that repeatedly kills nodes while performing read/write operations, the TransferQueueBundler thread seems to spend a lot of time waiting for physical addresses: > {noformat} > 06:40:10,316 WARN [org.radargun.utils.Utils] (pool-5-thread-1) Stack for thread TransferQueueBundler,default,apex953-14666: > java.lang.Thread.sleep(Native Method) > org.jgroups.util.Util.sleep(Util.java:1504) > org.jgroups.util.Util.sleepRandom(Util.java:1574) > org.jgroups.protocols.TP.sendToSingleMember(TP.java:1685) > org.jgroups.protocols.TP.doSend(TP.java:1670) > org.jgroups.protocols.TP$TransferQueueBundler.sendBundledMessages(TP.java:2476) > org.jgroups.protocols.TP$TransferQueueBundler.sendMessages(TP.java:2392) > org.jgroups.protocols.TP$TransferQueueBundler.run(TP.java:2383) > java.lang.Thread.run(Thread.java:744) > {noformat} > There are 2 bugs related to this already fixed in JGroups 3.5.0.Beta2+: JGRP-1814, JGRP-1815 > There is also a special case where the physical address could be removed from the cache too soon, exacerbating the effect of JGRP-1815: JGRP-1858 > We can work around the problem by changing the JGroups configuration: > * TP.logical_addr_cache_expiration=86400000 > ** Only expire addresses after 1 day > * TP.physical_addr_max_fetch_attempts=1 > ** Sleep for only 20ms waiting for the physical address (default 3 - 1500ms) > * UNICAST3_conn_close_timeout=10000 > ** Drop the pending messages to leavers sooner -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4484) Outbound transfers can be cancelled by old CANCEL_STATE_TRANSFER command

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4484?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4484: ----------------------------------------------- William Burns <wburns(a)redhat.com> changed the Status of [bug 1104045|https://bugzilla.redhat.com/show_bug.cgi?id=1104045] from ASSIGNED to MODIFIED > Outbound transfers can be cancelled by old CANCEL_STATE_TRANSFER command > ------------------------------------------------------------------------ > > Key: ISPN-4484 > URL: https://issues.jboss.org/browse/ISPN-4484 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Core, State Transfer > Affects Versions: 6.0.2.Final > Reporter: Dan Berindei > Assignee: Dan Berindei > Priority: Critical > Fix For: 7.0.0.Alpha5 > > > This appeared during the 32-nodes elasticity test in the Hyperion environment. > Just as apex947 left, it started a rebalance, which apex948 dutifully cancelled as it became the new coordinator. apex949 had already requested segments from apex959, so it sent a StateRequestCommand(CANCEL_STATE_TRANSFER) asynchronously to apex959. Then apex948 started a new rebalance, and apex949 asked apex959 for the same segments. When apex959 finally received the cancel request, it didn't check the topology id and it incorrectly cancelled the outbound transfer to apex949. > The solution would be to verify the topology id in the CANCEL_STATE_TRANSFER command before cancelling the transfer. I also think we can avoid sending the cancel command completely in this case, and only send it as we are about to stop. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4456) DSL queries: maxResults must be greather than 0

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4456?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4456: ----------------------------------------------- Adrian Nistor <anistor(a)redhat.com> changed the Status of [bug 1116924|https://bugzilla.redhat.com/show_bug.cgi?id=1116924] from POST to MODIFIED > DSL queries: maxResults must be greather than 0 > ----------------------------------------------- > > Key: ISPN-4456 > URL: https://issues.jboss.org/browse/ISPN-4456 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Embedded Querying, Remote Querying > Affects Versions: 7.0.0.Alpha4 > Reporter: Adrian Nistor > Assignee: Adrian Nistor > Fix For: 7.0.0.Alpha5 > > > 0 is currently allowed, but that does not make much sense. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4045) String literal escaping for new query dsl is not properly done

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4045?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4045: ----------------------------------------------- Adrian Nistor <anistor(a)redhat.com> changed the Status of [bug 1116927|https://bugzilla.redhat.com/show_bug.cgi?id=1116927] from POST to MODIFIED > String literal escaping for new query dsl is not properly done > -------------------------------------------------------------- > > Key: ISPN-4045 > URL: https://issues.jboss.org/browse/ISPN-4045 > Project: Infinispan > Issue Type: Task > Security Level: Public(Everyone can see) > Components: Embedded Querying, Remote Querying > Affects Versions: 6.0.1.Final > Reporter: Adrian Nistor > Assignee: Adrian Nistor > Fix For: 7.0.0.Beta1, 7.0.0.Final > > > Some tests are disabled because of this, see QueryDslConditionsTest.testStringEscape() and the remote equivalent. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

10 years, 6 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues July 2014