[JBoss JIRA] (ISPN-2722) NPE in AbstractCacheTransaction
by Thomas Fromm (JIRA)
[ https://issues.jboss.org/browse/ISPN-2722?page=com.atlassian.jira.plugin.... ]
Thomas Fromm closed ISPN-2722.
------------------------------
Fix Version/s: 5.2.0.CR2
Resolution: Duplicate Issue
Is duplicate of ISPN-2728
> NPE in AbstractCacheTransaction
> -------------------------------
>
> Key: ISPN-2722
> URL: https://issues.jboss.org/browse/ISPN-2722
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 5.2.0.CR1
> Reporter: Thomas Fromm
> Assignee: Mircea Markus
> Fix For: 5.2.0.CR2
>
>
> Inside Logs I saw the following exception. Unfortunality, no further and detailed logs available.
> ERROR 16.01.13 16:07:07,729 [IS-dev-blade-Node-1-WThread-6]
> InvocationContextInterceptor ISPN000136: Execution error
> java.lang.NullPointerException
> at
> org.infinispan.transaction.AbstractCacheTransaction.hasLockOrIsLockBackup(AbstractCacheTransaction.java:223)
> at
> org.infinispan.transaction.AbstractCacheTransaction.waitForLockRelease(AbstractCacheTransaction.java:165)
> at
> org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.waitForTransactionsToComplete(AbstractTxLockingInterceptor.java:206)
> at
> org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:173)
> at
> org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:123)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:254)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:249)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:242)
> at
> org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:335)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:123)
> at
> org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (ISPN-2722) NPE in AbstractCacheTransaction
by Carsten Lohmann (JIRA)
[ https://issues.jboss.org/browse/ISPN-2722?page=com.atlassian.jira.plugin.... ]
Carsten Lohmann commented on ISPN-2722:
---------------------------------------
Seems to be resolved in ISPN-2728.
> NPE in AbstractCacheTransaction
> -------------------------------
>
> Key: ISPN-2722
> URL: https://issues.jboss.org/browse/ISPN-2722
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 5.2.0.CR1
> Reporter: Thomas Fromm
> Assignee: Mircea Markus
>
> Inside Logs I saw the following exception. Unfortunality, no further and detailed logs available.
> ERROR 16.01.13 16:07:07,729 [IS-dev-blade-Node-1-WThread-6]
> InvocationContextInterceptor ISPN000136: Execution error
> java.lang.NullPointerException
> at
> org.infinispan.transaction.AbstractCacheTransaction.hasLockOrIsLockBackup(AbstractCacheTransaction.java:223)
> at
> org.infinispan.transaction.AbstractCacheTransaction.waitForLockRelease(AbstractCacheTransaction.java:165)
> at
> org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.waitForTransactionsToComplete(AbstractTxLockingInterceptor.java:206)
> at
> org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:173)
> at
> org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:123)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:254)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:249)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:242)
> at
> org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:335)
> at
> org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:123)
> at
> org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (ISPN-2712) Initial state transfer doesn't appear to all be persisted when using eviction in a replicated cluster
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2712?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2712:
--------------------------------
Assignee: Adrian Nistor (was: Mircea Markus)
> Initial state transfer doesn't appear to all be persisted when using eviction in a replicated cluster
> ------------------------------------------------------------------------------------------------------
>
> Key: ISPN-2712
> URL: https://issues.jboss.org/browse/ISPN-2712
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.0.CR1
> Reporter: Randall Hauch
> Assignee: Adrian Nistor
> Priority: Critical
> Fix For: 5.2.0.Final
>
> Attachments: ispn_eviction_log.txt, spectrum-repository-infinispan.xml
>
>
> Using a clustered cache with 2 nodes, where the cache in each node is configured identically with replication, eviction and (non-shared) file cache store. (See attached configuration.)
> The first (coordinator) process in the cluster is started and populated with 293 entries. Then the first process continually adds a few entries every 5 seconds. After a short delay, the second process is started, at which point it joins the cluster and starts the state-transfer process; logging shows in the first process that all 293 entries are transferred to the new cluster member, and the second log shows that they are all received. The second process then attempts to look for a specific entry that was created during initial population in the first process. This fails to find the existing entry.
> By enabling trace logging and "IDE breakpoint output messages" around state transfer, it's visible that from the 293 keys, only 218 are placed into the cache, the others being lost.
> (This problem was originally discovered when clustering ModeShape, which behaves roughly in the manner described above. The initial entries that are populated upon initialization are content created when a new repository is started. The second process looks for this content, and if it finds the content it knows not to create all of this initial content. However, if it doesn't find it, it thinks the repository has not yet been initialized and that it should create the initial content. The problem described by this bug then manifests itself in ModeShape through dozens of exceptions because the repository has been corrupted. See MODE-1745 for details on this problem. ModeShape's corresponding known issue for this issue, ISPN-2712, is MODE-1754.)
> The eviction is configured like this:
> {code:xml}
> <eviction strategy="LIRS" maxEntries="1000"/>
> {code}
> The attached log file is from the second process (the "receiver" node) and it contains the following key points:
> * line 40 - the total number of keys & entries to be transferred = 293
> * line 1352 and from there onwards 1358 / 1364 / i + 6 - the data container's size stops growing at 218, while the other entries are being sent. This means that in effect, they are ignored.
> * line 1797 - the loop from {{org.infinispan.statetransfer.StateConsumerImpl#doApplyState}} finishes
> Disabling eviction fixes the problem and all 293 nodes are placed in Node2's cache.
> (I initially marked this as CRITICAL priority, though it is a blocker for our use of Infinispan 5.2.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (ISPN-2697) HotRodServer startup fails when its record cannot be inserted into topology cache
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2697?page=com.atlassian.jira.plugin.... ]
Mircea Markus commented on ISPN-2697:
-------------------------------------
[~galderz]assigning this to you in order to even the load for 5.2.0.Final.
> HotRodServer startup fails when its record cannot be inserted into topology cache
> ---------------------------------------------------------------------------------
>
> Key: ISPN-2697
> URL: https://issues.jboss.org/browse/ISPN-2697
> Project: Infinispan
> Issue Type: Bug
> Components: Remote protocols
> Affects Versions: 5.2.0.Beta6
> Reporter: Radim Vansa
> Assignee: Galder Zamarreño
> Priority: Critical
> Fix For: 5.2.0.Final
>
>
> When the HotRodServer starts it inserts its record to __hotRodTopologyCache ({{HotRodServer.addSelfToTopologyView(...)}}).
> However, this put may very easily fail - as the command is broadcasted using NAKACK2 protocol, if the message gets lost and there's no following broadcasted message, the message will be not retransmitted and the put operation times out (Replication timeout), which fails the whole HotRodServer startup, all because of one lost UDP message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (ISPN-2697) HotRodServer startup fails when its record cannot be inserted into topology cache
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2697?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2697:
--------------------------------
Assignee: Galder Zamarreño (was: Dan Berindei)
> HotRodServer startup fails when its record cannot be inserted into topology cache
> ---------------------------------------------------------------------------------
>
> Key: ISPN-2697
> URL: https://issues.jboss.org/browse/ISPN-2697
> Project: Infinispan
> Issue Type: Bug
> Components: Remote protocols
> Affects Versions: 5.2.0.Beta6
> Reporter: Radim Vansa
> Assignee: Galder Zamarreño
> Priority: Critical
> Fix For: 5.2.0.Final
>
>
> When the HotRodServer starts it inserts its record to __hotRodTopologyCache ({{HotRodServer.addSelfToTopologyView(...)}}).
> However, this put may very easily fail - as the command is broadcasted using NAKACK2 protocol, if the message gets lost and there's no following broadcasted message, the message will be not retransmitted and the put operation times out (Replication timeout), which fails the whole HotRodServer startup, all because of one lost UDP message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (ISPN-2240) Per-key lock container leads to superfluous TimeoutExceptions on concurrent access to same key
by Mircea Markus (JIRA)
[ https://issues.jboss.org/browse/ISPN-2240?page=com.atlassian.jira.plugin.... ]
Mircea Markus updated ISPN-2240:
--------------------------------
Fix Version/s: 5.3.0.Final
(was: 5.2.0.Final)
> Per-key lock container leads to superfluous TimeoutExceptions on concurrent access to same key
> ----------------------------------------------------------------------------------------------
>
> Key: ISPN-2240
> URL: https://issues.jboss.org/browse/ISPN-2240
> Project: Infinispan
> Issue Type: Bug
> Components: Locking and Concurrency
> Affects Versions: 5.1.6.FINAL, 5.1.x
> Reporter: Robert Stupp
> Assignee: Mircea Markus
> Priority: Critical
> Fix For: 5.3.0.Final
>
> Attachments: ISPN-2240_fix_TimeoutExceptions.patch, somehow.zip
>
>
> Hi,
> I've encountered a lot of TimeoutExceptions just running a load test against an infinispan cluster.
> I tracked down the reason and found out, that the code in org.infinispan.util.concurrent.locks.containers.AbstractPerEntryLockContainer#releaseLock() causes these superfluous TimeoutExceptions.
> A small test case (which just prints out timeouts, too late timeouts and "paints" a lot of dots to the console - more dots/second on the console means better throughput ;-)
> In a short test I extended the class ReentrantPerEntryLockContainer and changed the implementation of releaseLock() as follows:
> {noformat}
> public void releaseLock(Object lockOwner, Object key) {
> ReentrantLock l = locks.get(key);
> if (l != null) {
> if (!l.isHeldByCurrentThread())
> throw new IllegalStateException("Lock for [" + key + "] not held by current thread " + Thread.currentThread());
> while (l.isHeldByCurrentThread())
> unlock(l, lockOwner);
> if (!l.hasQueuedThreads())
> locks.remove(key);
> }
> else
> throw new IllegalStateException("No lock for [" + key + ']');
> }
> {noformat}
> The main improvement is that locks are not removed from the concurrent map as long as other threads are waiting on that lock.
> If the lock is removed from the map while other threads are waiting for it, they may run into timeouts and force TimeoutExceptions to the client.
> The above methods "paints more dots per second" - means: it gives a better throughput for concurrent accesses to the same key.
> The re-implemented method should also fix some replication timeout exceptions.
> Please, please add this to 5.1.7, if possible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (ISPN-2579) StateResponseCommand received after the node is removed from CH causes IllegalArgumentException
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-2579?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-2579:
--------------------------------
Fix Version/s: 5.2.0.Final
(was: 5.3.0.Final)
> StateResponseCommand received after the node is removed from CH causes IllegalArgumentException
> -----------------------------------------------------------------------------------------------
>
> Key: ISPN-2579
> URL: https://issues.jboss.org/browse/ISPN-2579
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer
> Affects Versions: 5.2.0.Beta5
> Reporter: Radim Vansa
> Assignee: Adrian Nistor
> Priority: Minor
> Fix For: 5.2.0.Final
>
>
> When a node requests ST and then it receives a CH where this node is not a member, it sends ST request CANCEL_STATE_TRANSFER - however, if the StateResponseCommand is already on its way and reaches the node, it causes
> {code}
> java.lang.IllegalArgumentException: Node hyperion947-55285 is not a member
> at org.infinispan.distribution.ch.DefaultConsistentHash.getSegmentsForOwner(DefaultConsistentHash.java:102)
> at org.infinispan.statetransfer.StateConsumerImpl.applyState(StateConsumerImpl.java:272)
> at org.infinispan.statetransfer.StateResponseCommand.perform(StateResponseCommand.java:86)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months
[JBoss JIRA] (ISPN-2579) StateResponseCommand received after the node is removed from CH causes IllegalArgumentException
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-2579?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-2579:
--------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/1581
> StateResponseCommand received after the node is removed from CH causes IllegalArgumentException
> -----------------------------------------------------------------------------------------------
>
> Key: ISPN-2579
> URL: https://issues.jboss.org/browse/ISPN-2579
> Project: Infinispan
> Issue Type: Bug
> Components: State transfer
> Affects Versions: 5.2.0.Beta5
> Reporter: Radim Vansa
> Assignee: Adrian Nistor
> Priority: Minor
> Fix For: 5.3.0.Final
>
>
> When a node requests ST and then it receives a CH where this node is not a member, it sends ST request CANCEL_STATE_TRANSFER - however, if the StateResponseCommand is already on its way and reaches the node, it causes
> {code}
> java.lang.IllegalArgumentException: Node hyperion947-55285 is not a member
> at org.infinispan.distribution.ch.DefaultConsistentHash.getSegmentsForOwner(DefaultConsistentHash.java:102)
> at org.infinispan.statetransfer.StateConsumerImpl.applyState(StateConsumerImpl.java:272)
> at org.infinispan.statetransfer.StateResponseCommand.perform(StateResponseCommand.java:86)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 11 months