[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-3366:
------------------------------------
Looks like the remaining data loss has a slightly different cause: a node can become primary owner while a command is executing.
So we could have {{isPrimaryOwner(k) == false}} in EntryWrappingInterceptor, and {{isPrimaryOwner(k) == true}} by the time we enter NonTx[Concurrent]DistributionInterceptor:
{noformat}
11:23:12,876 TRACE [NonTransactionalLockingInterceptor] (HotRodServerWorker-9) Are (node2/clustered(s1)) we the lock owners for key 'ByteArrayKey{data=ByteArray{size=16, hashCode=4d12e1a9, array=0x033e0d74687265616431306b65793539}}'? false
11:23:12,876 TRACE [EntryWrappingInterceptor] (HotRodServerWorker-9) Wrapping entry 'ByteArrayKey{data=ByteArray{size=16, hashCode=4d12e1a9, array=0x033e0d74687265616431306b65793539}}'? false
11:23:12,882 TRACE [NonTxDistributionInterceptor] (HotRodServerWorker-9) Not doing a remote get for key ByteArrayKey{data=ByteArray{size=16, hashCode=4d12e1a9, array=0x033e0d74687265616431306b65793539}} since entry is not affected by rehash or is already in data container. We are node2/clustered(s1), owners are [node4/clustered(s1), node2/clustered(s1)]
11:23:12,886 TRACE [NonTxConcurrentDistributionInterceptor] (HotRodServerWorker-9) I'm the primary owner, sending the command to all ([node2/clustered(s1)]) the recipients in order to be applied.
{noformat}
Because the key is not wrapped on the originator, it's not written to the data container, but the command isn't looped back to the originator from the primary owner either.
A possible solution would be to check the topology id of the command against the current topology id before sending it, and throw an OutdatedTopologyException to cause StateTransferInterceptor to retry the command.
> Data loss when entry forwarding to primary owner and primary owner shutdown
> ---------------------------------------------------------------------------
>
> Key: ISPN-3366
> URL: https://issues.jboss.org/browse/ISPN-3366
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.2.4.Final, 6.0.0.Alpha1
> Reporter: Takayoshi Kimura
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.8.Final, 6.0.0.Alpha3, 6.0.0.Final
>
> Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-logs.zip
>
>
> Looks like a problem in entry forwarding.
> Here is test scenario:
> * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load
> * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total
> After the test run, the numberOfEntries on each node are:
> * node1: 26608
> * node2: 26622
> * node3: 26746
> * node4: 0
> Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing.
> Let's take a look at the missing entry, hash(thread16key59) = 574ff563.
> Current CH: owners(574ff563) are [node4, node1]
> The events sequence is:
> * hotrod -> node1
> * node1 forwarding it to primary owner node4
> * node4 doesn't process the forwarded entry, shutdown
> Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-2903) Manual eviction should not delete entry from cache store
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2903?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2903:
-----------------------------------------------
Radoslav Husar <rhusar(a)redhat.com> made a comment on [bug 900549|https://bugzilla.redhat.com/show_bug.cgi?id=900549]
BZ 992988 is not duplicate and neither is related to this bug. Treat them separately.
Any intermediate results to share regarding verification of this bug?
> Manual eviction should not delete entry from cache store
> --------------------------------------------------------
>
> Key: ISPN-2903
> URL: https://issues.jboss.org/browse/ISPN-2903
> Project: Infinispan
> Issue Type: Bug
> Components: Eviction
> Affects Versions: 5.2.3.Final
> Reporter: Paul Ferraro
> Assignee: Galder Zamarreño
> Priority: Critical
> Labels: 5.2.x, jdg6
> Fix For: 5.2.5.Final, 5.3.0.Alpha1, 5.3.0.Final
>
> Attachments: AtomicMapServlet.java, AtomicMapTestCase.java, server.log, server.log
>
>
> Here's the scenario:
> Given 2 nodes with REPL_SYNC cache with passivating cache store (e.g. default web cache in AS7).
> 1. Create cache entry containing atomic map with 2 map entries on node1.
> 2. Passivate that cache entry on node2 via manual evict.
> 3. Modify 1 of the atomic map entries within the cache entry on node1.
> 4. Lookup atomic map on node2. It only contains 1 map entry - the map entry modified in step 3. The other map entry is lost.
> It's a side effect of ISPN-2384, where some changes were made to tighten the passivation/activation scenarios, but it did not cover manual eviction calls.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-3391) Upgrade to JBoss Marshalling 2.0.0.Beta1
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-3391?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-3391:
-----------------------------------
Fix Version/s: 6.0.0.Beta1
(was: 6.0.0.Alpha3)
> Upgrade to JBoss Marshalling 2.0.0.Beta1
> ----------------------------------------
>
> Key: ISPN-3391
> URL: https://issues.jboss.org/browse/ISPN-3391
> Project: Infinispan
> Issue Type: Component Upgrade
> Components: Build process, Marshalling
> Affects Versions: 6.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Galder Zamarreño
> Fix For: 6.0.0.Beta1
>
>
> Quoting David Lloyd:
> The following things should be known about this update:
> * There are some API incompatibilities (in particular, Creators are gone, and Externalizers changed a little bit)
> * The binary protocol is 100% identical
> 2.x can interoperate with 1.x over the wire with no problems
> * The changes to Infinispan seem pretty minimal
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-3391) Upgrade to JBoss Marshalling 2.0.0.Beta1
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-3391?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño commented on ISPN-3391:
----------------------------------------
This has been deferred until JBoss Marshalling 2.0.0.Beta2 is out. First Beta removed Creator interface and implementations, which makes upgrade hard for Infinispan Server since base AS still uses those classes and installing multiple versions of the same library with the ant magic available is not there yet. The agreement has been to introduce back the interface/classes removed and deprecate them instead. Full chat with David can be found [here|https://gist.github.com/galderz/3206fe5ee2e279c39202].
> Upgrade to JBoss Marshalling 2.0.0.Beta1
> ----------------------------------------
>
> Key: ISPN-3391
> URL: https://issues.jboss.org/browse/ISPN-3391
> Project: Infinispan
> Issue Type: Component Upgrade
> Components: Build process, Marshalling
> Affects Versions: 6.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Galder Zamarreño
> Fix For: 6.0.0.Alpha3
>
>
> Quoting David Lloyd:
> The following things should be known about this update:
> * There are some API incompatibilities (in particular, Creators are gone, and Externalizers changed a little bit)
> * The binary protocol is 100% identical
> 2.x can interoperate with 1.x over the wire with no problems
> * The changes to Infinispan seem pretty minimal
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-3391) Upgrade to JBoss Marshalling 2.0.0.Beta1
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-3391?page=com.atlassian.jira.plugin.... ]
Work on ISPN-3391 stopped by Galder Zamarreño.
> Upgrade to JBoss Marshalling 2.0.0.Beta1
> ----------------------------------------
>
> Key: ISPN-3391
> URL: https://issues.jboss.org/browse/ISPN-3391
> Project: Infinispan
> Issue Type: Component Upgrade
> Components: Build process, Marshalling
> Affects Versions: 6.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Galder Zamarreño
> Fix For: 6.0.0.Alpha3
>
>
> Quoting David Lloyd:
> The following things should be known about this update:
> * There are some API incompatibilities (in particular, Creators are gone, and Externalizers changed a little bit)
> * The binary protocol is 100% identical
> 2.x can interoperate with 1.x over the wire with no problems
> * The changes to Infinispan seem pretty minimal
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-3391) Upgrade to JBoss Marshalling 2.0.0.Beta1
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-3391?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño reassigned ISPN-3391:
--------------------------------------
Assignee: Galder Zamarreño (was: Dan Berindei)
> Upgrade to JBoss Marshalling 2.0.0.Beta1
> ----------------------------------------
>
> Key: ISPN-3391
> URL: https://issues.jboss.org/browse/ISPN-3391
> Project: Infinispan
> Issue Type: Component Upgrade
> Components: Build process, Marshalling
> Affects Versions: 6.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Galder Zamarreño
> Fix For: 6.0.0.Alpha3
>
>
> Quoting David Lloyd:
> The following things should be known about this update:
> * There are some API incompatibilities (in particular, Creators are gone, and Externalizers changed a little bit)
> * The binary protocol is 100% identical
> 2.x can interoperate with 1.x over the wire with no problems
> * The changes to Infinispan seem pretty minimal
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-3391) Upgrade to JBoss Marshalling 2.0.0.Beta1
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-3391?page=com.atlassian.jira.plugin.... ]
Work on ISPN-3391 started by Galder Zamarreño.
> Upgrade to JBoss Marshalling 2.0.0.Beta1
> ----------------------------------------
>
> Key: ISPN-3391
> URL: https://issues.jboss.org/browse/ISPN-3391
> Project: Infinispan
> Issue Type: Component Upgrade
> Components: Build process, Marshalling
> Affects Versions: 6.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Galder Zamarreño
> Fix For: 6.0.0.Alpha3
>
>
> Quoting David Lloyd:
> The following things should be known about this update:
> * There are some API incompatibilities (in particular, Creators are gone, and Externalizers changed a little bit)
> * The binary protocol is 100% identical
> 2.x can interoperate with 1.x over the wire with no problems
> * The changes to Infinispan seem pretty minimal
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-2903) Manual eviction should not delete entry from cache store
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-2903?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-2903:
-----------------------------------------------
Jitka Kudrnacova <jkudrnac(a)redhat.com> made a comment on [bug 900549|https://bugzilla.redhat.com/show_bug.cgi?id=900549]
If we decide to mark BZ 992988 as a duplicate of this bug, than this bug is still alive and cannot make a transition to the VERIFIED state.
> Manual eviction should not delete entry from cache store
> --------------------------------------------------------
>
> Key: ISPN-2903
> URL: https://issues.jboss.org/browse/ISPN-2903
> Project: Infinispan
> Issue Type: Bug
> Components: Eviction
> Affects Versions: 5.2.3.Final
> Reporter: Paul Ferraro
> Assignee: Galder Zamarreño
> Priority: Critical
> Labels: 5.2.x, jdg6
> Fix For: 5.2.5.Final, 5.3.0.Alpha1, 5.3.0.Final
>
> Attachments: AtomicMapServlet.java, AtomicMapTestCase.java, server.log, server.log
>
>
> Here's the scenario:
> Given 2 nodes with REPL_SYNC cache with passivating cache store (e.g. default web cache in AS7).
> 1. Create cache entry containing atomic map with 2 map entries on node1.
> 2. Passivate that cache entry on node2 via manual evict.
> 3. Modify 1 of the atomic map entries within the cache entry on node1.
> 4. Lookup atomic map on node2. It only contains 1 map entry - the map entry modified in step 3. The other map entry is lost.
> It's a side effect of ISPN-2384, where some changes were made to tighten the passivation/activation scenarios, but it did not cover manual eviction calls.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months