June 2014 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-4424) ReplaceIfUnmodified is broken under concurrent execution

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4424?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4424: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1110647|https://bugzilla.redhat.com/show_bug.cgi?id=1110647] from ASSIGNED to MODIFIED > ReplaceIfUnmodified is broken under concurrent execution > -------------------------------------------------------- > > Key: ISPN-4424 > URL: https://issues.jboss.org/browse/ISPN-4424 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Remote Protocols > Affects Versions: 6.0.2.Final, 7.0.0.Alpha4 > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > Fix For: 7.0.0.Alpha5, 7.0.0.Final > > > Versioned update with a multi threaded Hot Rod client results in inconsistency. Some replaceWithVersion return true ignoring a version update executed in another thread. Here's a log excerpt of a concurrency stress test: > ``` > 2014-06-20 16:16:56,798 INFO [PutFromNull] (pool-7-thread-10) count=462,prev=462,new=463 > 2014-06-20 16:16:56,820 INFO [PutFromNull] (pool-7-thread-9) count=463,prev=463,new=464 > 2014-06-20 16:16:56,831 INFO [PutFromNull] (pool-7-thread-2) count=464,prev=463,new=464 > 2014-06-20 16:16:56,845 INFO [PutFromNull] (pool-7-thread-9) count=465,prev=464,new=465 > ``` > Here you see two threads applying the same replacement, from 463 to 464. > The issue appears a result of a race condition in Hot Rod server's protocol decoder. When replaceIfUmodified is received, the cache entry is retrieved to verify whether the version in the server and the version sent in the command match. However, the cache entry retrieved is mutable, and the value could change midway through this operation as a result of another thread updating the value. Please find below some log snippets showing this. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3936) State transfer should not modify or iterate shared cache stores

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-3936?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-3936: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1113259|https://bugzilla.redhat.com/show_bug.cgi?id=1113259] from NEW to POST > State transfer should not modify or iterate shared cache stores > --------------------------------------------------------------- > > Key: ISPN-3936 > URL: https://issues.jboss.org/browse/ISPN-3936 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Core, State Transfer > Affects Versions: 6.0.0.Final > Reporter: Dan Berindei > Assignee: Dan Berindei > Priority: Critical > Fix For: 7.0.0.Alpha3, 7.0.0.Final > > > When installing a new topology, we invalidate the cache entries that are no longer mapped to the local node. We also iterate over the entries in the cache stores and delete the ones that are no longer local, but this should only happen if the cache store is not shared. > We had similar issues in the past, but this seems to be related to the new cache store API introduced in 6.0. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4424) ReplaceIfUnmodified is broken under concurrent execution

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-4424?page=com.atlassian.jira.plugin.... ] William Burns resolved ISPN-4424. --------------------------------- Fix Version/s: 7.0.0.Alpha5 Resolution: Done > ReplaceIfUnmodified is broken under concurrent execution > -------------------------------------------------------- > > Key: ISPN-4424 > URL: https://issues.jboss.org/browse/ISPN-4424 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Remote Protocols > Affects Versions: 6.0.2.Final, 7.0.0.Alpha4 > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > Fix For: 7.0.0.Alpha5, 7.0.0.Final > > > Versioned update with a multi threaded Hot Rod client results in inconsistency. Some replaceWithVersion return true ignoring a version update executed in another thread. Here's a log excerpt of a concurrency stress test: > ``` > 2014-06-20 16:16:56,798 INFO [PutFromNull] (pool-7-thread-10) count=462,prev=462,new=463 > 2014-06-20 16:16:56,820 INFO [PutFromNull] (pool-7-thread-9) count=463,prev=463,new=464 > 2014-06-20 16:16:56,831 INFO [PutFromNull] (pool-7-thread-2) count=464,prev=463,new=464 > 2014-06-20 16:16:56,845 INFO [PutFromNull] (pool-7-thread-9) count=465,prev=464,new=465 > ``` > Here you see two threads applying the same replacement, from 463 to 464. > The issue appears a result of a race condition in Hot Rod server's protocol decoder. When replaceIfUmodified is received, the cache entry is retrieved to verify whether the version in the server and the version sent in the command match. However, the cache entry retrieved is mutable, and the value could change midway through this operation as a result of another thread updating the value. Please find below some log snippets showing this. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4291) Poor REPL state transfer performance with cache stores

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4291?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4291: ----------------------------------------------- William Burns <wburns(a)redhat.com> changed the Status of [bug 1111188|https://bugzilla.redhat.com/show_bug.cgi?id=1111188] from ASSIGNED to MODIFIED > Poor REPL state transfer performance with cache stores > ------------------------------------------------------ > > Key: ISPN-4291 > URL: https://issues.jboss.org/browse/ISPN-4291 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: State Transfer > Affects Versions: 6.0.2.Final > Reporter: Dennis Reed > Assignee: Dan Berindei > Fix For: 7.0.0.Alpha5 > > > During a state transfer every single key is loaded from the cache store to determine whether it's still owned by the local node. > This is an extremely slow operation, and doesn't even make sense for a REPL cache. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3936) State transfer should not modify or iterate shared cache stores

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-3936?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration updated ISPN-3936: ------------------------------------------ Bugzilla Update: Perform Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1113259 > State transfer should not modify or iterate shared cache stores > --------------------------------------------------------------- > > Key: ISPN-3936 > URL: https://issues.jboss.org/browse/ISPN-3936 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Core, State Transfer > Affects Versions: 6.0.0.Final > Reporter: Dan Berindei > Assignee: Dan Berindei > Priority: Critical > Fix For: 7.0.0.Alpha3, 7.0.0.Final > > > When installing a new topology, we invalidate the cache entries that are no longer mapped to the local node. We also iterate over the entries in the cache stores and delete the ones that are no longer local, but this should only happen if the cache store is not shared. > We had similar issues in the past, but this seems to be related to the new cache store API introduced in 6.0. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4442) Package remoting jars for the embedded cli client

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4442?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4442: ----------------------------------------------- Vitalii Chepeliuk <vchepeli(a)redhat.com> changed the Status of [bug 1106451|https://bugzilla.redhat.com/show_bug.cgi?id=1106451] from MODIFIED to NEW > Package remoting jars for the embedded cli client > ------------------------------------------------- > > Key: ISPN-4442 > URL: https://issues.jboss.org/browse/ISPN-4442 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: CLI > Reporter: Tristan Tarrant > Assignee: Tristan Tarrant > Fix For: 7.0.0.Final > > -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4442) Package remoting jars for the embedded cli client

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4442?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4442: ----------------------------------------------- Vitalii Chepeliuk <vchepeli(a)redhat.com> changed the Status of [bug 1106451|https://bugzilla.redhat.com/show_bug.cgi?id=1106451] from NEW to MODIFIED > Package remoting jars for the embedded cli client > ------------------------------------------------- > > Key: ISPN-4442 > URL: https://issues.jboss.org/browse/ISPN-4442 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: CLI > Reporter: Tristan Tarrant > Assignee: Tristan Tarrant > Fix For: 7.0.0.Final > > -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4291) Poor REPL state transfer performance with cache stores

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-4291?page=com.atlassian.jira.plugin.... ] William Burns updated ISPN-4291: -------------------------------- Status: Resolved (was: Pull Request Sent) Fix Version/s: 7.0.0.Alpha5 Resolution: Done > Poor REPL state transfer performance with cache stores > ------------------------------------------------------ > > Key: ISPN-4291 > URL: https://issues.jboss.org/browse/ISPN-4291 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: State Transfer > Affects Versions: 6.0.2.Final > Reporter: Dennis Reed > Assignee: Dan Berindei > Fix For: 7.0.0.Alpha5 > > > During a state transfer every single key is loaded from the cache store to determine whether it's still owned by the local node. > This is an extremely slow operation, and doesn't even make sense for a REPL cache. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4442) Package remoting jars for the embedded cli client

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4442?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4442: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1106451|https://bugzilla.redhat.com/show_bug.cgi?id=1106451] from NEW to MODIFIED > Package remoting jars for the embedded cli client > ------------------------------------------------- > > Key: ISPN-4442 > URL: https://issues.jboss.org/browse/ISPN-4442 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: CLI > Reporter: Tristan Tarrant > Assignee: Tristan Tarrant > Fix For: 7.0.0.Final > > -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2956) putIfAbsent on Hot Rod Java client doesn't reliably fulfil contract

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-2956?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-2956: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1004193|https://bugzilla.redhat.com/show_bug.cgi?id=1004193] from ASSIGNED to MODIFIED > putIfAbsent on Hot Rod Java client doesn't reliably fulfil contract > ------------------------------------------------------------------- > > Key: ISPN-2956 > URL: https://issues.jboss.org/browse/ISPN-2956 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Remote Protocols > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > Labels: 63gablocker, hotrod-java-client, remote-clients > Fix For: 7.0.0.Beta1 > > > Hot Rod's putIfAbsent might have issues on some edge cases: > {quote}I want to know whether the putting entry already exists in the remote > cache cluster, or not. > I thought that RemoteCache.putIfAbsent() would be useful for that > purpose, i.e., > {code} > if (remoteCache.putIfAbsent(k,v) == null) { > // new entry. > } else { > // k already exists. > } > {code} > But no. > The putIfAbsent() for new entry may return non-null value, if one of the > server crushed while putting. > The behavior is like the following: > 1. Client do putIfAbsent(k,v). > 2. The server receives the request and sends replication requests to > other servers. If the server crushed before completing replication, some > servers own that (k,v), but others not. > 3. Client receives the error. The putIfAbsent() internally retries the > same request to the next server in the cluster server list. > 4. If the next server owns the (k,v), the putIfAbsent() returns the > replicated (k,v) at the step 2, without any error. > So, putIfAbsent() is not reliable for knowing whether the putting entry > is *exactly* new or not. > Does anyone have any idea/workaround for this purpose?{quote} > A workaround is to do this: > {quote}We got a simple solution, which can be applied to our customer's application. > If each value part of putting (k,v) is unique or contains unique value, > the client can do *double check* wether the entry is new. > {code} > val = System.nanoTime(); // or uuid is also useful. > if ((ret = cache.putIfAbsent(key, val)) == null > || ret.equals(val)) { > // new entry, if the return value is just the same. > } else { > // key already exists. > } > {code} > We are proposing this workaround which almost works fine.{quote} > However, this is a bit of a cludge. > Hot Rod should be improved with an operation that allows a version to be passed when entry is created, instead of relying on the client generating it. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 9 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues June 2014