August 2013 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown

by Takayoshi Kimura (JIRA)

[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ] Takayoshi Kimura commented on ISPN-3366: ---------------------------------------- Looks like the problem symptom is different, it retries (this fixes the original problem) but doesn't commit the entry at the last try. There is no "About to commit entry" log. Probably we need to tweak EntryWrappingInterceptor as well? > Data loss when entry forwarding to primary owner and primary owner shutdown > --------------------------------------------------------------------------- > > Key: ISPN-3366 > URL: https://issues.jboss.org/browse/ISPN-3366 > Project: Infinispan > Issue Type: Bug > Components: Distributed Cache > Affects Versions: 5.2.4.Final, 6.0.0.Alpha1 > Reporter: Takayoshi Kimura > Assignee: Dan Berindei > Priority: Critical > Fix For: 5.2.8.Final, 6.0.0.Alpha2, 6.0.0.Final > > Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-logs.zip > > > Looks like a problem in entry forwarding. > Here is test scenario: > * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load > * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total > After the test run, the numberOfEntries on each node are: > * node1: 26608 > * node2: 26622 > * node3: 26746 > * node4: 0 > Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing. > Let's take a look at the missing entry, hash(thread16key59) = 574ff563. > Current CH: owners(574ff563) are [node4, node1] > The events sequence is: > * hotrod -> node1 > * node1 forwarding it to primary owner node4 > * node4 doesn't process the forwarded entry, shutdown > Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown

by Takayoshi Kimura (JIRA)

[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ] Takayoshi Kimura commented on ISPN-3366: ---------------------------------------- Note that the possibility of the data loss is lowered, it's now 10% or so. I reproduced it 2 times out of 20 times. Previously it was over 50%. > Data loss when entry forwarding to primary owner and primary owner shutdown > --------------------------------------------------------------------------- > > Key: ISPN-3366 > URL: https://issues.jboss.org/browse/ISPN-3366 > Project: Infinispan > Issue Type: Bug > Components: Distributed Cache > Affects Versions: 5.2.4.Final, 6.0.0.Alpha1 > Reporter: Takayoshi Kimura > Assignee: Dan Berindei > Priority: Critical > Fix For: 5.2.8.Final, 6.0.0.Alpha2, 6.0.0.Final > > Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-logs.zip > > > Looks like a problem in entry forwarding. > Here is test scenario: > * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load > * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total > After the test run, the numberOfEntries on each node are: > * node1: 26608 > * node2: 26622 > * node3: 26746 > * node4: 0 > Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing. > Let's take a look at the missing entry, hash(thread16key59) = 574ff563. > Current CH: owners(574ff563) are [node4, node1] > The events sequence is: > * hotrod -> node1 > * node1 forwarding it to primary owner node4 > * node4 doesn't process the forwarded entry, shutdown > Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown

by Takayoshi Kimura (JIRA)

[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ] Takayoshi Kimura reopened ISPN-3366: ------------------------------------ Reopen, we still see data loss. > Data loss when entry forwarding to primary owner and primary owner shutdown > --------------------------------------------------------------------------- > > Key: ISPN-3366 > URL: https://issues.jboss.org/browse/ISPN-3366 > Project: Infinispan > Issue Type: Bug > Components: Distributed Cache > Affects Versions: 5.2.4.Final, 6.0.0.Alpha1 > Reporter: Takayoshi Kimura > Assignee: Dan Berindei > Priority: Critical > Fix For: 5.2.8.Final, 6.0.0.Alpha2, 6.0.0.Final > > Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-logs.zip > > > Looks like a problem in entry forwarding. > Here is test scenario: > * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load > * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total > After the test run, the numberOfEntries on each node are: > * node1: 26608 > * node2: 26622 > * node3: 26746 > * node4: 0 > Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing. > Let's take a look at the missing entry, hash(thread16key59) = 574ff563. > Current CH: owners(574ff563) are [node4, node1] > The events sequence is: > * hotrod -> node1 > * node1 forwarding it to primary owner node4 > * node4 doesn't process the forwarded entry, shutdown > Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown

by Takayoshi Kimura (JIRA)

[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ] Takayoshi Kimura updated ISPN-3366: ----------------------------------- Attachment: ISPN-3366-full-logs-3rd.zip Tested with 5.2.8-SNAPSHOT and data loss happened. (In logs it says 5.2.6-SNAPSHOT because Version.java is not updated) 3 entries are missing, first one is hash(thread10key59)=4d12e1a9. Attached ISPN-3366-full-logs-3rd.zip, from this time total entries 4000, 1/10 of previous size. Also it contains JGroups TRACE logging. > Data loss when entry forwarding to primary owner and primary owner shutdown > --------------------------------------------------------------------------- > > Key: ISPN-3366 > URL: https://issues.jboss.org/browse/ISPN-3366 > Project: Infinispan > Issue Type: Bug > Components: Distributed Cache > Affects Versions: 5.2.4.Final, 6.0.0.Alpha1 > Reporter: Takayoshi Kimura > Assignee: Dan Berindei > Priority: Critical > Fix For: 5.2.8.Final, 6.0.0.Alpha2, 6.0.0.Final > > Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-logs.zip > > > Looks like a problem in entry forwarding. > Here is test scenario: > * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load > * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total > After the test run, the numberOfEntries on each node are: > * node1: 26608 > * node2: 26622 > * node3: 26746 > * node4: 0 > Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing. > Let's take a look at the missing entry, hash(thread16key59) = 574ff563. > Current CH: owners(574ff563) are [node4, node1] > The events sequence is: > * hotrod -> node1 > * node1 forwarding it to primary owner node4 > * node4 doesn't process the forwarded entry, shutdown > Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3169) Define a Query API that will be common for library mode + remote

by Sanne Grinovero (JIRA)

[ https://issues.jboss.org/browse/ISPN-3169?page=com.atlassian.jira.plugin.... ] Sanne Grinovero updated ISPN-3169: ---------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done > Define a Query API that will be common for library mode + remote > ---------------------------------------------------------------- > > Key: ISPN-3169 > URL: https://issues.jboss.org/browse/ISPN-3169 > Project: Infinispan > Issue Type: Sub-task > Reporter: Mircea Markus > Assignee: Adrian Nistor > Priority: Blocker > Fix For: 6.0.0.Alpha2, 6.0.0.Final > > > Some notes on the query API: > - fluent, supporting a predefined set of filters (equality, range, etc). No user pluggable filter will be supported as this API will be available both embedded and *remotely* by the Java HR client > - the filters accept the schema(protobuf) primitive types as arguments (int, string, byte, etc..) > - should support paging: start page + pageSize > - projections are not supported > For an overview on the remote query design refer to: https://community.jboss.org/wiki/QueryingDesignInInfinispan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3333) Implement the new Query DSL in embedded mode

by Sanne Grinovero (JIRA)

[ https://issues.jboss.org/browse/ISPN-3333?page=com.atlassian.jira.plugin.... ] Sanne Grinovero updated ISPN-3333: ---------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done > Implement the new Query DSL in embedded mode > -------------------------------------------- > > Key: ISPN-3333 > URL: https://issues.jboss.org/browse/ISPN-3333 > Project: Infinispan > Issue Type: Sub-task > Components: Querying > Reporter: Mircea Markus > Assignee: Adrian Nistor > Priority: Blocker > Fix For: 6.0.0.Alpha2, 6.0.0.Final > > > The output of this: all the sample queries defined in the DSL[1] should be runnable(unit test). > [1] https://github.com/anistor/infinispan/blob/t_3169_m/query/src/main/java/o... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-3366: -------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done > Data loss when entry forwarding to primary owner and primary owner shutdown > --------------------------------------------------------------------------- > > Key: ISPN-3366 > URL: https://issues.jboss.org/browse/ISPN-3366 > Project: Infinispan > Issue Type: Bug > Components: Distributed Cache > Affects Versions: 5.2.4.Final, 6.0.0.Alpha1 > Reporter: Takayoshi Kimura > Assignee: Dan Berindei > Priority: Critical > Attachments: ISPN-3366-logs.zip > > > Looks like a problem in entry forwarding. > Here is test scenario: > * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load > * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total > After the test run, the numberOfEntries on each node are: > * node1: 26608 > * node2: 26622 > * node3: 26746 > * node4: 0 > Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing. > Let's take a look at the missing entry, hash(thread16key59) = 574ff563. > Current CH: owners(574ff563) are [node4, node1] > The events sequence is: > * hotrod -> node1 > * node1 forwarding it to primary owner node4 > * node4 doesn't process the forwarded entry, shutdown > Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-3366: -------------------------------- Fix Version/s: 5.2.8.Final 6.0.0.Alpha2 6.0.0.Final > Data loss when entry forwarding to primary owner and primary owner shutdown > --------------------------------------------------------------------------- > > Key: ISPN-3366 > URL: https://issues.jboss.org/browse/ISPN-3366 > Project: Infinispan > Issue Type: Bug > Components: Distributed Cache > Affects Versions: 5.2.4.Final, 6.0.0.Alpha1 > Reporter: Takayoshi Kimura > Assignee: Dan Berindei > Priority: Critical > Fix For: 5.2.8.Final, 6.0.0.Alpha2, 6.0.0.Final > > Attachments: ISPN-3366-logs.zip > > > Looks like a problem in entry forwarding. > Here is test scenario: > * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load > * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total > After the test run, the numberOfEntries on each node are: > * node1: 26608 > * node2: 26622 > * node3: 26746 > * node4: 0 > Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing. > Let's take a look at the missing entry, hash(thread16key59) = 574ff563. > Current CH: owners(574ff563) are [node4, node1] > The events sequence is: > * hotrod -> node1 > * node1 forwarding it to primary owner node4 > * node4 doesn't process the forwarded entry, shutdown > Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3391) Upgrade to JBoss Marshalling 2.0.0.Beta1

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-3391?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-3391: ------------------------------- Status: Pull Request Sent (was: Open) > Upgrade to JBoss Marshalling 2.0.0.Beta1 > ---------------------------------------- > > Key: ISPN-3391 > URL: https://issues.jboss.org/browse/ISPN-3391 > Project: Infinispan > Issue Type: Component Upgrade > Components: Build process, Marshalling > Affects Versions: 6.0.0.Alpha1 > Reporter: Dan Berindei > Assignee: Dan Berindei > Fix For: 6.0.0.Alpha2 > > > Quoting David Lloyd: > The following things should be known about this update: > * There are some API incompatibilities (in particular, Creators are gone, and Externalizers changed a little bit) > * The binary protocol is 100% identical > 2.x can interoperate with 1.x over the wire with no problems > * The changes to Infinispan seem pretty minimal -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3391) Upgrade to JBoss Marshalling 2.0.0.Beta1

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-3391?page=com.atlassian.jira.plugin.... ] Dan Berindei updated ISPN-3391: ------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done > Upgrade to JBoss Marshalling 2.0.0.Beta1 > ---------------------------------------- > > Key: ISPN-3391 > URL: https://issues.jboss.org/browse/ISPN-3391 > Project: Infinispan > Issue Type: Component Upgrade > Components: Build process, Marshalling > Affects Versions: 6.0.0.Alpha1 > Reporter: Dan Berindei > Assignee: Dan Berindei > Fix For: 6.0.0.Alpha2 > > > Quoting David Lloyd: > The following things should be known about this update: > * There are some API incompatibilities (in particular, Creators are gone, and Externalizers changed a little bit) > * The binary protocol is 100% identical > 2.x can interoperate with 1.x over the wire with no problems > * The changes to Infinispan seem pretty minimal -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 4 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues August 2013