[JBoss JIRA] (ISPN-4286) Two concurrent putIfAbsent operations can both return null during rebalance
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4286?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-4286:
------------------------------------
We would need versioning to fix this, since there's no way to distinguish between one {{v1}} and the other.
> Two concurrent putIfAbsent operations can both return null during rebalance
> ---------------------------------------------------------------------------
>
> Key: ISPN-4286
> URL: https://issues.jboss.org/browse/ISPN-4286
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 6.0.2.Final
> Reporter: Dan Berindei
> Priority: Critical
> Fix For: 7.1.0.Alpha1
>
>
> If the cache topology changes while executing a putIfAbsent operation, the old primary owner will throw an OutdatedTopologyException, and the originator will retry on the new owner.
> When retrying the PutKeyValueCommand on the new primary owner, we compare the current value with the command's new value. If they are equal, we assume that the initial command wrote the old value, and we return {{null}}.
> However, the value might have been written by another putIfAbsent operation. So we could have two {{putIfAbsent(k, v)}} operations, both returning {{null}}.
> {code}
> A is the originator, B is the primary owner, k = null
> A -> B: putIfAbsent(k, v1)
> B dies before writing v, C is now primary owner
> D -> C: putIfAbsent(k, v1) // another put operation from D, with the same value
> C -> D: null // correct
> A -> C: retry_putIfAbsent(k, v1)
> C -> A: null // C assumes A is overwriting its own value, so it's also returning null
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months
[JBoss JIRA] (ISPN-4988) TopologyAwareDistAsyncFuncTest fails with SIGSEGV exception with Azul JDK
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4988?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4988:
-----------------------------------------------
Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1165036|https://bugzilla.redhat.com/show_bug.cgi?id=1165036] from NEW to CLOSED
> TopologyAwareDistAsyncFuncTest fails with SIGSEGV exception with Azul JDK
> -------------------------------------------------------------------------
>
> Key: ISPN-4988
> URL: https://issues.jboss.org/browse/ISPN-4988
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.1.Final
> Reporter: Vitalii Chepeliuk
> Assignee: Dan Berindei
> Labels: testsuite_stability
>
> {noformat}
> Test suite progress: tests succeeded: 2602, failed: 0, skipped: 0.
> 2014-11-16 04:38:48,750 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:48,750 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:48,829 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> 2014-11-16 04:38:48,945 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:48,945 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:49,023 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> 2014-11-16 04:38:49,249 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:49,249 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:49,326 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> Signum: [11] - Exiting due to unhandled SIGSEGV exception.
> 0: rip=0x0000000020d9eb48 @rip=[0xffffffffffffffff] (hotspot_os_backtrace_callback+40) [gcc frame, calls gcc]
> 1: rip=0x00007f526459da3f @rip=[0x0000440000bffb38] (os_backtrace+31) [gcc frame, calls gcc]
> 2: rip=0x0000000020d99ef5 @rip=[0x0000440000bffba8] (jvm_unexpected_exception_handler+165) [gcc frame, calls gcc]
> 3: rip=0x00007f526459cf52 @rip=[0x0000440000bffc58] (jvm_unexpected_exception_handler_wrapper+82) [gcc frame, calls gcc]
> 4: rip=0x0000000020838c30 @rip=[0x0000440000bffc78] (GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+112) [gcc frame, calls gcc]
> 5: rip=0x0000000020876355 @rip=[0x0000440000bffcd8] (void GPGC_MarkAlgorithm::drain_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+149) [gcc frame, calls gcc]
> 6: rip=0x000000002087748c @rip=[0x0000440000bffda8] (void GPGC_MarkAlgorithm::drain_and_steal_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+28) [gcc frame, calls gcc]
> 7: rip=0x00000000209ba1ed @rip=[0x0000440000bffe68] (PGCTaskThread::run()+589) [gcc frame, calls gcc]
> 8: rip=0x00007f526459fc49 @rip=[0x0000440000bfff88] (alternate_stack_create+153) [gcc frame, calls gcc]
> #
> # An unexpected error has been detected by Java Runtime Environment:
> #
> # Segmentation fault (0xb) at pc=0x20838c30, pid=25165, tid=25171
> #
> # Java VM: Zing 64-Bit Tiered VM (1.7.0-zing_5.10.1.0-b9-product-azlinuxM-X86_64, mixed mode)
> # Problematic frame:
> # C [libjvm.so+0x438c30] GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+0x70
> #
> # An error report file with more information is saved as:
> # /qa/hudson_workspace/workspace/jdg-63-ispn-testsuite-rhel-azul/c6098cff/infinispan/core/hs_err_pid25165.log
> 2014-11-16 04:38:49,693 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:49,693 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> #
> # If you would like to submit a bug report, please visit:
> # http://www.azulsystems.com/support/
> #
> {noformat}
> More info from jenkins jobs here
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuit...
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuit...
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months
[JBoss JIRA] (ISPN-4988) TopologyAwareDistAsyncFuncTest fails with SIGSEGV exception with Azul JDK
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-4988?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant commented on ISPN-4988:
---------------------------------------
This is a JVM bug
> TopologyAwareDistAsyncFuncTest fails with SIGSEGV exception with Azul JDK
> -------------------------------------------------------------------------
>
> Key: ISPN-4988
> URL: https://issues.jboss.org/browse/ISPN-4988
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.1.Final
> Reporter: Vitalii Chepeliuk
> Assignee: Dan Berindei
> Labels: testsuite_stability
>
> {noformat}
> Test suite progress: tests succeeded: 2602, failed: 0, skipped: 0.
> 2014-11-16 04:38:48,750 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:48,750 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:48,829 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> 2014-11-16 04:38:48,945 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:48,945 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:49,023 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> 2014-11-16 04:38:49,249 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:49,249 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:49,326 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> Signum: [11] - Exiting due to unhandled SIGSEGV exception.
> 0: rip=0x0000000020d9eb48 @rip=[0xffffffffffffffff] (hotspot_os_backtrace_callback+40) [gcc frame, calls gcc]
> 1: rip=0x00007f526459da3f @rip=[0x0000440000bffb38] (os_backtrace+31) [gcc frame, calls gcc]
> 2: rip=0x0000000020d99ef5 @rip=[0x0000440000bffba8] (jvm_unexpected_exception_handler+165) [gcc frame, calls gcc]
> 3: rip=0x00007f526459cf52 @rip=[0x0000440000bffc58] (jvm_unexpected_exception_handler_wrapper+82) [gcc frame, calls gcc]
> 4: rip=0x0000000020838c30 @rip=[0x0000440000bffc78] (GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+112) [gcc frame, calls gcc]
> 5: rip=0x0000000020876355 @rip=[0x0000440000bffcd8] (void GPGC_MarkAlgorithm::drain_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+149) [gcc frame, calls gcc]
> 6: rip=0x000000002087748c @rip=[0x0000440000bffda8] (void GPGC_MarkAlgorithm::drain_and_steal_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+28) [gcc frame, calls gcc]
> 7: rip=0x00000000209ba1ed @rip=[0x0000440000bffe68] (PGCTaskThread::run()+589) [gcc frame, calls gcc]
> 8: rip=0x00007f526459fc49 @rip=[0x0000440000bfff88] (alternate_stack_create+153) [gcc frame, calls gcc]
> #
> # An unexpected error has been detected by Java Runtime Environment:
> #
> # Segmentation fault (0xb) at pc=0x20838c30, pid=25165, tid=25171
> #
> # Java VM: Zing 64-Bit Tiered VM (1.7.0-zing_5.10.1.0-b9-product-azlinuxM-X86_64, mixed mode)
> # Problematic frame:
> # C [libjvm.so+0x438c30] GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+0x70
> #
> # An error report file with more information is saved as:
> # /qa/hudson_workspace/workspace/jdg-63-ispn-testsuite-rhel-azul/c6098cff/infinispan/core/hs_err_pid25165.log
> 2014-11-16 04:38:49,693 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:49,693 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> #
> # If you would like to submit a bug report, please visit:
> # http://www.azulsystems.com/support/
> #
> {noformat}
> More info from jenkins jobs here
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuit...
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuit...
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months
[JBoss JIRA] (ISPN-4988) TopologyAwareDistAsyncFuncTest fails with SIGSEGV exception with Azul JDK
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-4988?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant closed ISPN-4988.
---------------------------------
Resolution: Rejected
> TopologyAwareDistAsyncFuncTest fails with SIGSEGV exception with Azul JDK
> -------------------------------------------------------------------------
>
> Key: ISPN-4988
> URL: https://issues.jboss.org/browse/ISPN-4988
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.1.Final
> Reporter: Vitalii Chepeliuk
> Assignee: Dan Berindei
> Labels: testsuite_stability
>
> {noformat}
> Test suite progress: tests succeeded: 2602, failed: 0, skipped: 0.
> 2014-11-16 04:38:48,750 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:48,750 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:48,829 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> 2014-11-16 04:38:48,945 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:48,945 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:49,023 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> 2014-11-16 04:38:49,249 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:49,249 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> 2014-11-16 04:38:49,326 WARN [TCP] (testng-TopologyAwareDistAsyncFuncTest) JGRP000046: bundler_type=old has been removed; using sender-sends-with-timer
> Signum: [11] - Exiting due to unhandled SIGSEGV exception.
> 0: rip=0x0000000020d9eb48 @rip=[0xffffffffffffffff] (hotspot_os_backtrace_callback+40) [gcc frame, calls gcc]
> 1: rip=0x00007f526459da3f @rip=[0x0000440000bffb38] (os_backtrace+31) [gcc frame, calls gcc]
> 2: rip=0x0000000020d99ef5 @rip=[0x0000440000bffba8] (jvm_unexpected_exception_handler+165) [gcc frame, calls gcc]
> 3: rip=0x00007f526459cf52 @rip=[0x0000440000bffc58] (jvm_unexpected_exception_handler_wrapper+82) [gcc frame, calls gcc]
> 4: rip=0x0000000020838c30 @rip=[0x0000440000bffc78] (GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+112) [gcc frame, calls gcc]
> 5: rip=0x0000000020876355 @rip=[0x0000440000bffcd8] (void GPGC_MarkAlgorithm::drain_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+149) [gcc frame, calls gcc]
> 6: rip=0x000000002087748c @rip=[0x0000440000bffda8] (void GPGC_MarkAlgorithm::drain_and_steal_stacks<GPGC_GCManagerOldStrong>(GPGC_GCManagerOldStrong*)+28) [gcc frame, calls gcc]
> 7: rip=0x00000000209ba1ed @rip=[0x0000440000bffe68] (PGCTaskThread::run()+589) [gcc frame, calls gcc]
> 8: rip=0x00007f526459fc49 @rip=[0x0000440000bfff88] (alternate_stack_create+153) [gcc frame, calls gcc]
> #
> # An unexpected error has been detected by Java Runtime Environment:
> #
> # Segmentation fault (0xb) at pc=0x20838c30, pid=25165, tid=25171
> #
> # Java VM: Zing 64-Bit Tiered VM (1.7.0-zing_5.10.1.0-b9-product-azlinuxM-X86_64, mixed mode)
> # Problematic frame:
> # C [libjvm.so+0x438c30] GPGC_GCManagerMark::process_mutator_stack(HeapRefBuffer*)+0x70
> #
> # An error report file with more information is saved as:
> # /qa/hudson_workspace/workspace/jdg-63-ispn-testsuite-rhel-azul/c6098cff/infinispan/core/hs_err_pid25165.log
> 2014-11-16 04:38:49,693 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.loopback has been deprecated: enabled by default
> 2014-11-16 04:38:49,693 WARN [Configurator] (testng-TopologyAwareDistAsyncFuncTest) JGRP000014: TP.physical_addr_max_fetch_attempts has been deprecated: will be ignored
> #
> # If you would like to submit a bug report, please visit:
> # http://www.azulsystems.com/support/
> #
> {noformat}
> More info from jenkins jobs here
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuit...
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-63-ispn-testsuit...
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months
[JBoss JIRA] (ISPN-4972) Two concurrent replaceWithVersions may both succeed
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4972?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4972:
-----------------------------------------------
Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1163103|https://bugzilla.redhat.com/show_bug.cgi?id=1163103] from NEW to ASSIGNED
> Two concurrent replaceWithVersions may both succeed
> ---------------------------------------------------
>
> Key: ISPN-4972
> URL: https://issues.jboss.org/browse/ISPN-4972
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Affects Versions: 7.0.0.Final
> Reporter: Radim Vansa
> Assignee: Galder Zamarreño
>
> Cache contains entry {{K = V}}. Two concurrent threads, that execute:
> {code}
> long version = cache.getVersioned(K).getVersion();
> boolean succeeded = cache.replaceWithVersion(K, V, version);
> {code}
> Both of these threads can get {{succeeded = true}}.
> Reason:
> When the server receives the operation ReplaceIfUnmodified=replaceWithVersion, it retrieves the entry (key + value + metadata) from the cache and checks that the version stored (in metadata) is the same as the version in the request. If so, it creates conditional ReplaceCommand which contains the value (retrieved atomically with version from the cache) and executes this one. Therefore, as the value is in both requests identical (and is not changed by the replace), two concurrent ReplaceCommands can both succeed, and this value is returned to the HotRod client.
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months
[JBoss JIRA] (ISPN-4887) Stale locks in non-tx cache after merge
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4887?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-4887:
-------------------------------
Status: Open (was: New)
> Stale locks in non-tx cache after merge
> ---------------------------------------
>
> Key: ISPN-4887
> URL: https://issues.jboss.org/browse/ISPN-4887
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.0.0.CR2
> Reporter: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 7.1.0.Alpha1
>
>
> It appears to cause random failures in {{ThreeNodesReplicatedSplitAndMergeTest.testSplitAndMerge0}}:
> {noformat}
> org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key MagicKey#null{74043ff7@ThreeNodesReplicatedSplitAndMergeTest-NodeC-12710/9} and requestor Thread[testng-ThreeNodesReplicatedSplitAndMergeTest,5,main]. Lock is held by Thread[remote-thread-ThreeNodesReplicatedSplitAndMergeTest-NodeC-p5654-t5,5,main], while request came from null
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLockNoCheck(LockManagerImpl.java:181)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:127)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:123)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:47)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:172)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:95)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:134)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.partionhandling.impl.PartitionHandlingInterceptor.visitPutKeyValueCommand(PartitionHandlingInterceptor.java:34)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1512)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:990)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:982)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1582)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:235)
> at org.infinispan.partitionhandling.ThreeNodesReplicatedSplitAndMergeTest.testSplitAndMerge(ThreeNodesReplicatedSplitAndMergeTest.java:132)
> at org.infinispan.partitionhandling.ThreeNodesReplicatedSplitAndMergeTest.testSplitAndMerge0(ThreeNodesReplicatedSplitAndMergeTest.java:27)
> {noformat}
> http://ci.infinispan.org/viewLog.html?buildId=13499&tab=buildResultsDiv&b...
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months
[JBoss JIRA] (ISPN-4887) Stale locks in non-tx cache after merge
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4887?page=com.atlassian.jira.plugin.... ]
Dan Berindei reassigned ISPN-4887:
----------------------------------
Assignee: Dan Berindei
> Stale locks in non-tx cache after merge
> ---------------------------------------
>
> Key: ISPN-4887
> URL: https://issues.jboss.org/browse/ISPN-4887
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.0.0.CR2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 7.1.0.Alpha1
>
>
> It appears to cause random failures in {{ThreeNodesReplicatedSplitAndMergeTest.testSplitAndMerge0}}:
> {noformat}
> org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key MagicKey#null{74043ff7@ThreeNodesReplicatedSplitAndMergeTest-NodeC-12710/9} and requestor Thread[testng-ThreeNodesReplicatedSplitAndMergeTest,5,main]. Lock is held by Thread[remote-thread-ThreeNodesReplicatedSplitAndMergeTest-NodeC-p5654-t5,5,main], while request came from null
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLockNoCheck(LockManagerImpl.java:181)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:127)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockKey(AbstractLockingInterceptor.java:123)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:47)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:172)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:95)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:134)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.partionhandling.impl.PartitionHandlingInterceptor.visitPutKeyValueCommand(PartitionHandlingInterceptor.java:34)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1512)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:990)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:982)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1582)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:235)
> at org.infinispan.partitionhandling.ThreeNodesReplicatedSplitAndMergeTest.testSplitAndMerge(ThreeNodesReplicatedSplitAndMergeTest.java:132)
> at org.infinispan.partitionhandling.ThreeNodesReplicatedSplitAndMergeTest.testSplitAndMerge0(ThreeNodesReplicatedSplitAndMergeTest.java:27)
> {noformat}
> http://ci.infinispan.org/viewLog.html?buildId=13499&tab=buildResultsDiv&b...
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months
[JBoss JIRA] (ISPN-4995) ClusteredGet served for non-member of CH
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4995?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-4995:
-------------------------------
Status: Open (was: New)
> ClusteredGet served for non-member of CH
> ----------------------------------------
>
> Key: ISPN-4995
> URL: https://issues.jboss.org/browse/ISPN-4995
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Reporter: Radim Vansa
> Priority: Critical
>
> When nodes accept ClusteredGetCommand from node that is not member of CH, it can happen that when one thread does
> {code}
> put(K1, V1);
> put(K2, V2)
> {code}
> and another gets
> {code}
> get(K2) -> V2
> get(K1) -> V0 (some old value)
> {code}
> edg-perf01, 02 and 03 share this view and topology:
> {code}
> 04:40:08,714 TRACE [org.jgroups.protocols.FD_SOCK] (INT-8,edg-perf01-63779) edg-perf01-63779: i-have-sock: edg-perf02-45117 --> 172.18.1.3:37476 (cache is {edg-perf01-63779=172.18.1.1:40099, edg-perf02-45117=172.18.1.3:37476})
> 04:40:08,715 TRACE [org.infinispan.topology.ClusterTopologyManagerImpl] (transport-thread--p2-t6) Received new cluster view: 8, isCoordinator = true, becameCoordinator = false
> 04:40:11,203 DEBUG [org.infinispan.topology.LocalTopologyManagerImpl] (transport-thread--p2-t1) Updating local consistent hash(es) for cache testCache: new topology = CacheTopology{id=16, rebalanceId=4, currentC
> H=DefaultConsistentHash{ns = 512, owners = (3)[edg-perf02-45117: 171+170, edg-perf03-6264: 171+171, edg-perf01-63779: 170+171]}, pendingCH=null, unionCH=null, actualMembers=[edg-perf02-45117, edg-perf03-6264, edg-perf01-63779]}
> {code}
> Later, edg-perf02 and edg-perf03 get new view and install a new topology, where edg-perf01 does not exist:
> {code}
> 04:41:13,681 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,edg-perf03-6264) ISPN000093: Received new, MERGED cluster view for channel default: MergeView::[edg-perf02-45117|9] (3) [edg-perf02-45117, edg-perf03-6264, edg-perf04-10989], 1 subgroups: [edg-perf04-10989|7] (1) [edg-perf04-10989]
> 04:41:13,681 TRACE [org.infinispan.topology.ClusterTopologyManagerImpl] (transport-thread--p2-t22) Received new cluster view: 9, isCoordinator = false, becameCoordinator = false
> 04:41:13,760 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (remote-thread--p3-t32) Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=testCache, type=CH_UPDATE, sender=edg-perf02-45117, joinInfo=null, topologyId=18, rebalanceId=4, currentCH=DefaultConsistentHash{ns = 512, owners = (2)[edg-perf02-45117: 256+85, edg-perf03-6264: 256+86]}, pendingCH=null, availabilityMode=AVAILABLE, actualMembers=[edg-perf02-45117, edg-perf03-6264], throwable=null, viewId=9}[sender=edg-perf02-45117]
> {code}
> After that, edg-perf04 writes to {{key_00000000000020DB}} which is currently owned only by edg-perf03 - this key servers as K1 in example above. It is not backed up to edg-perf01, but edg-perf01 still thinks it's an owner of this key as it did not get any new view (this is a log from edg-perf03) :
> {code}
> 04:41:30,884 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (remote-thread--p3-t45) edg-perf03-6264 invoking PutKeyValueCommand{key=key_00000000000020DB, value=[33 #4: 0, 169, 284, 634, ], flags=[SKIP_CACHE_LOAD, SKIP_REMOTE_LOOKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true} to recipient list [edg-perf03-6264] with options RpcOptions{timeout=60000, unit=MILLISECONDS, fifoOrder=true, totalOrder=false, responseFilter=null, responseMode=SYNCHRONOUS, skipReplicationQueue=false}
> {code}
> Later, edg-perf04 writes to another key {{stressor_33}} (K2 in the example) value with operationId=650 (previous value is 600) which is replicated to edg-perf02 and edg-perf03.
> Now a merge view with all 4 nodes is installed:
> {code}
> 04:41:31,258 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,edg-perf01-63779) ISPN000093: Received new, MERGED cluster view for channel default: MergeView::[edg-perf01-63779|10] (4) [edg-perf01-63779, edg-perf03-6264, edg-perf02-45117, edg-perf04-10989], 6 subgroups: [edg-perf02-45117|7] (2) [edg-perf02-45117, edg-perf03-6264], [edg-perf01-63779|4] (2) [edg-perf01-63779, edg-perf02-45117], [edg-perf02-45117|9] (3) [edg-perf02-45117, edg-perf03-6264, edg-perf04-10989], [edg-perf03-6264|4] (2) [edg-perf03-6264, edg-perf04-10989], [edg-perf01-63779|8] (3) [edg-perf01-63779, edg-perf02-45117, edg-perf03-6264], [edg-perf01-63779|6] (1) [edg-perf01-63779]
> 04:41:31,258 TRACE [org.infinispan.topology.ClusterTopologyManagerImpl] (transport-thread--p2-t2) Received new cluster view: 10, isCoordinator = true, becameCoordinator = false
> {code}
> edg-perf01 now issues a remote get to edg-perf02 for key stressor_33 and receives the correct answer (operationId=650):
> {code}
> 04:41:32,494 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (BackgroundOps-Checker-1) Response(s) to ClusteredGetCommand{key=stressor_33, flags=null} is {edg-perf02-45117=SuccessfulResponse{responseValue=ImmortalCacheValue {value=LastOperation{operationId=650, seed=0000A15A4C2DD25A}}} }
> {code}
> However, when edg-perf01 reads {{key_00000000000020DB}}, it loads the old value from local data container as no CH update/rebalance happened so far:
> {code}
> 04:41:32,496 TRACE [org.infinispan.partitionhandling.impl.PartitionHandlingManagerImpl] (BackgroundOps-Checker-1) Checking availability for key=key_00000000000020DB, status=AVAILABLE
> 04:41:32,497 ERROR [org.radargun.stages.cache.background.LogChecker] (BackgroundOps-Checker-1) Missing operation 634 for thread 33 on key 8411 (key_00000000000020DB)
> 04:41:32,499 DEBUG [org.radargun.service.InfinispanDebugable] (BackgroundOps-Checker-1) Debug info for key testCache key_00000000000020DB: owners=edg-perf01-63779, edg-perf03-6264, local=true, uncertain=false, container.key_00000000000020DB=ImmortalCacheEntry[key=key_00000000000020DB, value=[33 #3: 0, 169, 284, ], created=-1, isCreated=false, lastUsed=-1, isChanged=false, expires=-1, isExpired=false, canExpire=false, isEvicted=true, isRemoved=false, isValid=false, lifespan=-1, maxIdle=-1], segmentId=173
> {code}
> Note that this was found on branch https://github.com/infinispan/infinispan/pull/3062/files trying to fix ISPN-4949.
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months
[JBoss JIRA] (ISPN-4972) Two concurrent replaceWithVersions may both succeed
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-4972?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-4972:
----------------------------------
Assignee: Galder Zamarreño
> Two concurrent replaceWithVersions may both succeed
> ---------------------------------------------------
>
> Key: ISPN-4972
> URL: https://issues.jboss.org/browse/ISPN-4972
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Affects Versions: 7.0.0.Final
> Reporter: Radim Vansa
> Assignee: Galder Zamarreño
>
> Cache contains entry {{K = V}}. Two concurrent threads, that execute:
> {code}
> long version = cache.getVersioned(K).getVersion();
> boolean succeeded = cache.replaceWithVersion(K, V, version);
> {code}
> Both of these threads can get {{succeeded = true}}.
> Reason:
> When the server receives the operation ReplaceIfUnmodified=replaceWithVersion, it retrieves the entry (key + value + metadata) from the cache and checks that the version stored (in metadata) is the same as the version in the request. If so, it creates conditional ReplaceCommand which contains the value (retrieved atomically with version from the cache) and executes this one. Therefore, as the value is in both requests identical (and is not changed by the replace), two concurrent ReplaceCommands can both succeed, and this value is returned to the HotRod client.
--
This message was sent by Atlassian JIRA
(v6.3.8#6338)
11 years, 4 months