Help to prevent deadlocks by making the lock id accessible through the public API --------------------------------------------------------------------------------- Key: ISPN-993 URL: https://issues.jboss.org/browse/ISPN-993 Project: Infinispan Issue Type: Feature Request Components: Locking and Concurrency Reporter: Nicolas Filotto Assignee: Manik Surtani Fix For: 5.0.0.BETA1, 5.0.0.FINAL To prevent deadlocks when we use ISPN with transactions, we need to sort the keys before calling a method that acquires a lock such as cache.put and cache.remove. For example when we don't use the lock striping (i.e. we use one lock per key), the following use case will create a deadlock: Thread #1 does: {code} tm.begin() // We acquire the lock on "A" cache.put("A","A"); // We acquire the lock on "B" cache.put("B","B"); // Here all the acquired locks will be released tm.commit(); {code} Thread #2 does: {code} tm.begin() // We acquire the lock on "B" cache.put("B","B"); // We acquire the lock on "A" cache.put("A","A"); // Here all the acquired locks will be released tm.commit(); {code} To prevent this deadlock we can simply sort the keys within the Tx such that the lock acquisition is done in the same order. Unfortunately when we use lock striping even when we sort the keys we can face deadlocks because we cannot predict how the locks are distributed over the keys. For example the following test case creates a deadlock when the lock striping is enabled. {code} public class TestDeadlock extends TestCase { /** * We get an unpredictable deadlock because apart if we know the implementation * of the ReentrantStripedLockContainer, we cannot know that the keys * 8 * "a" and 13 * "a" are mapped to the same lock and 11 * "a" and 14 * "a" are * both mapped to the same lock that is different from the first one. * In the use case below the keys are sorted but we still get a deadlock */ public void testUseLockStripingTrueUnpredictableDeadLock() throws Exception { DefaultCacheManager manager = new DefaultCacheManager(); manager.getDefaultConfiguration().setInvocationBatchingEnabled(true); manager.getDefaultConfiguration().setLockAcquisitionTimeout(1000); final Cache<String, String> cache = manager.getCache(); final CountDownLatch startSignal = new CountDownLatch(1); final CountDownLatch endSignal = new CountDownLatch(2); final AtomicReference<Exception> exception = new AtomicReference<Exception>(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 8 * "a" cache.put("aaaaaaaa", "Value"); // 14 * "a" cache.put("aaaaaaaaaaaaaa", "Value"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 11 * "a" cache.put("aaaaaaaaaaa", "Value2"); // 13 * "a" cache.put("aaaaaaaaaaaaa", "Value2"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); startSignal.countDown(); endSignal.await(); assertNull(exception.get()); } } {code} If we could get the lock id thanks to the public API, we could sort our keys according to this value and then avoid the deadlocks even when the lock striping is enabled.

Help to prevent deadlocks by making the lock id accessible through the public API --------------------------------------------------------------------------------- Key: ISPN-993 URL: https://issues.jboss.org/browse/ISPN-993 Project: Infinispan Issue Type: Feature Request Components: Locking and Concurrency Reporter: Nicolas Filotto Assignee: Manik Surtani Fix For: 5.0.0.CR1, 5.0.0.FINAL To prevent deadlocks when we use ISPN with transactions, we need to sort the keys before calling a method that acquires a lock such as cache.put and cache.remove. For example when we don't use the lock striping (i.e. we use one lock per key), the following use case will create a deadlock: Thread #1 does: {code} tm.begin() // We acquire the lock on "A" cache.put("A","A"); // We acquire the lock on "B" cache.put("B","B"); // Here all the acquired locks will be released tm.commit(); {code} Thread #2 does: {code} tm.begin() // We acquire the lock on "B" cache.put("B","B"); // We acquire the lock on "A" cache.put("A","A"); // Here all the acquired locks will be released tm.commit(); {code} To prevent this deadlock we can simply sort the keys within the Tx such that the lock acquisition is done in the same order. Unfortunately when we use lock striping even when we sort the keys we can face deadlocks because we cannot predict how the locks are distributed over the keys. For example the following test case creates a deadlock when the lock striping is enabled. {code} public class TestDeadlock extends TestCase { /** * We get an unpredictable deadlock because apart if we know the implementation * of the ReentrantStripedLockContainer, we cannot know that the keys * 8 * "a" and 13 * "a" are mapped to the same lock and 11 * "a" and 14 * "a" are * both mapped to the same lock that is different from the first one. * In the use case below the keys are sorted but we still get a deadlock */ public void testUseLockStripingTrueUnpredictableDeadLock() throws Exception { DefaultCacheManager manager = new DefaultCacheManager(); manager.getDefaultConfiguration().setInvocationBatchingEnabled(true); manager.getDefaultConfiguration().setLockAcquisitionTimeout(1000); final Cache<String, String> cache = manager.getCache(); final CountDownLatch startSignal = new CountDownLatch(1); final CountDownLatch endSignal = new CountDownLatch(2); final AtomicReference<Exception> exception = new AtomicReference<Exception>(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 8 * "a" cache.put("aaaaaaaa", "Value"); // 14 * "a" cache.put("aaaaaaaaaaaaaa", "Value"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 11 * "a" cache.put("aaaaaaaaaaa", "Value2"); // 13 * "a" cache.put("aaaaaaaaaaaaa", "Value2"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); startSignal.countDown(); endSignal.await(); assertNull(exception.get()); } } {code} If we could get the lock id thanks to the public API, we could sort our keys according to this value and then avoid the deadlocks even when the lock striping is enabled.

Help to prevent deadlocks by making the lock id accessible through the public API --------------------------------------------------------------------------------- Key: ISPN-993 URL: https://issues.jboss.org/browse/ISPN-993 Project: Infinispan Issue Type: Feature Request Components: Locking and Concurrency Reporter: Nicolas Filotto Assignee: Manik Surtani Fix For: 5.0.0.CR1, 5.0.0.FINAL To prevent deadlocks when we use ISPN with transactions, we need to sort the keys before calling a method that acquires a lock such as cache.put and cache.remove. For example when we don't use the lock striping (i.e. we use one lock per key), the following use case will create a deadlock: Thread #1 does: {code} tm.begin() // We acquire the lock on "A" cache.put("A","A"); // We acquire the lock on "B" cache.put("B","B"); // Here all the acquired locks will be released tm.commit(); {code} Thread #2 does: {code} tm.begin() // We acquire the lock on "B" cache.put("B","B"); // We acquire the lock on "A" cache.put("A","A"); // Here all the acquired locks will be released tm.commit(); {code} To prevent this deadlock we can simply sort the keys within the Tx such that the lock acquisition is done in the same order. Unfortunately when we use lock striping even when we sort the keys we can face deadlocks because we cannot predict how the locks are distributed over the keys. For example the following test case creates a deadlock when the lock striping is enabled. {code} public class TestDeadlock extends TestCase { /** * We get an unpredictable deadlock because apart if we know the implementation * of the ReentrantStripedLockContainer, we cannot know that the keys * 8 * "a" and 13 * "a" are mapped to the same lock and 11 * "a" and 14 * "a" are * both mapped to the same lock that is different from the first one. * In the use case below the keys are sorted but we still get a deadlock */ public void testUseLockStripingTrueUnpredictableDeadLock() throws Exception { DefaultCacheManager manager = new DefaultCacheManager(); manager.getDefaultConfiguration().setInvocationBatchingEnabled(true); manager.getDefaultConfiguration().setLockAcquisitionTimeout(1000); final Cache<String, String> cache = manager.getCache(); final CountDownLatch startSignal = new CountDownLatch(1); final CountDownLatch endSignal = new CountDownLatch(2); final AtomicReference<Exception> exception = new AtomicReference<Exception>(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 8 * "a" cache.put("aaaaaaaa", "Value"); // 14 * "a" cache.put("aaaaaaaaaaaaaa", "Value"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 11 * "a" cache.put("aaaaaaaaaaa", "Value2"); // 13 * "a" cache.put("aaaaaaaaaaaaa", "Value2"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); startSignal.countDown(); endSignal.await(); assertNull(exception.get()); } } {code} If we could get the lock id thanks to the public API, we could sort our keys according to this value and then avoid the deadlocks even when the lock striping is enabled.

Help to prevent deadlocks by making the lock id accessible through the public API --------------------------------------------------------------------------------- Key: ISPN-993 URL: https://issues.jboss.org/browse/ISPN-993 Project: Infinispan Issue Type: Feature Request Components: Locking and Concurrency Reporter: Nicolas Filotto Assignee: Manik Surtani Fix For: 5.0.0.CR1, 5.0.0.FINAL To prevent deadlocks when we use ISPN with transactions, we need to sort the keys before calling a method that acquires a lock such as cache.put and cache.remove. For example when we don't use the lock striping (i.e. we use one lock per key), the following use case will create a deadlock: Thread #1 does: {code} tm.begin() // We acquire the lock on "A" cache.put("A","A"); // We acquire the lock on "B" cache.put("B","B"); // Here all the acquired locks will be released tm.commit(); {code} Thread #2 does: {code} tm.begin() // We acquire the lock on "B" cache.put("B","B"); // We acquire the lock on "A" cache.put("A","A"); // Here all the acquired locks will be released tm.commit(); {code} To prevent this deadlock we can simply sort the keys within the Tx such that the lock acquisition is done in the same order. Unfortunately when we use lock striping even when we sort the keys we can face deadlocks because we cannot predict how the locks are distributed over the keys. For example the following test case creates a deadlock when the lock striping is enabled. {code} public class TestDeadlock extends TestCase { /** * We get an unpredictable deadlock because apart if we know the implementation * of the ReentrantStripedLockContainer, we cannot know that the keys * 8 * "a" and 13 * "a" are mapped to the same lock and 11 * "a" and 14 * "a" are * both mapped to the same lock that is different from the first one. * In the use case below the keys are sorted but we still get a deadlock */ public void testUseLockStripingTrueUnpredictableDeadLock() throws Exception { DefaultCacheManager manager = new DefaultCacheManager(); manager.getDefaultConfiguration().setInvocationBatchingEnabled(true); manager.getDefaultConfiguration().setLockAcquisitionTimeout(1000); final Cache<String, String> cache = manager.getCache(); final CountDownLatch startSignal = new CountDownLatch(1); final CountDownLatch endSignal = new CountDownLatch(2); final AtomicReference<Exception> exception = new AtomicReference<Exception>(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 8 * "a" cache.put("aaaaaaaa", "Value"); // 14 * "a" cache.put("aaaaaaaaaaaaaa", "Value"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 11 * "a" cache.put("aaaaaaaaaaa", "Value2"); // 13 * "a" cache.put("aaaaaaaaaaaaa", "Value2"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); startSignal.countDown(); endSignal.await(); assertNull(exception.get()); } } {code} If we could get the lock id thanks to the public API, we could sort our keys according to this value and then avoid the deadlocks even when the lock striping is enabled.

Help to prevent deadlocks by making the lock id accessible through the public API --------------------------------------------------------------------------------- Key: ISPN-993 URL: https://issues.jboss.org/browse/ISPN-993 Project: Infinispan Issue Type: Feature Request Components: Locking and Concurrency Reporter: Nicolas Filotto Assignee: Manik Surtani Fix For: 5.0.0.CR1, 5.0.0.FINAL To prevent deadlocks when we use ISPN with transactions, we need to sort the keys before calling a method that acquires a lock such as cache.put and cache.remove. For example when we don't use the lock striping (i.e. we use one lock per key), the following use case will create a deadlock: Thread #1 does: {code} tm.begin() // We acquire the lock on "A" cache.put("A","A"); // We acquire the lock on "B" cache.put("B","B"); // Here all the acquired locks will be released tm.commit(); {code} Thread #2 does: {code} tm.begin() // We acquire the lock on "B" cache.put("B","B"); // We acquire the lock on "A" cache.put("A","A"); // Here all the acquired locks will be released tm.commit(); {code} To prevent this deadlock we can simply sort the keys within the Tx such that the lock acquisition is done in the same order. Unfortunately when we use lock striping even when we sort the keys we can face deadlocks because we cannot predict how the locks are distributed over the keys. For example the following test case creates a deadlock when the lock striping is enabled. {code} public class TestDeadlock extends TestCase { /** * We get an unpredictable deadlock because apart if we know the implementation * of the ReentrantStripedLockContainer, we cannot know that the keys * 8 * "a" and 13 * "a" are mapped to the same lock and 11 * "a" and 14 * "a" are * both mapped to the same lock that is different from the first one. * In the use case below the keys are sorted but we still get a deadlock */ public void testUseLockStripingTrueUnpredictableDeadLock() throws Exception { DefaultCacheManager manager = new DefaultCacheManager(); manager.getDefaultConfiguration().setInvocationBatchingEnabled(true); manager.getDefaultConfiguration().setLockAcquisitionTimeout(1000); final Cache<String, String> cache = manager.getCache(); final CountDownLatch startSignal = new CountDownLatch(1); final CountDownLatch endSignal = new CountDownLatch(2); final AtomicReference<Exception> exception = new AtomicReference<Exception>(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 8 * "a" cache.put("aaaaaaaa", "Value"); // 14 * "a" cache.put("aaaaaaaaaaaaaa", "Value"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 11 * "a" cache.put("aaaaaaaaaaa", "Value2"); // 13 * "a" cache.put("aaaaaaaaaaaaa", "Value2"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); startSignal.countDown(); endSignal.await(); assertNull(exception.get()); } } {code} If we could get the lock id thanks to the public API, we could sort our keys according to this value and then avoid the deadlocks even when the lock striping is enabled.

Help to prevent deadlocks by making the lock id accessible through the public API --------------------------------------------------------------------------------- Key: ISPN-993 URL: https://issues.jboss.org/browse/ISPN-993 Project: Infinispan Issue Type: Feature Request Components: Locking and Concurrency Reporter: Nicolas Filotto Assignee: Manik Surtani Fix For: 5.0.0.CR1, 5.0.0.FINAL To prevent deadlocks when we use ISPN with transactions, we need to sort the keys before calling a method that acquires a lock such as cache.put and cache.remove. For example when we don't use the lock striping (i.e. we use one lock per key), the following use case will create a deadlock: Thread #1 does: {code} tm.begin() // We acquire the lock on "A" cache.put("A","A"); // We acquire the lock on "B" cache.put("B","B"); // Here all the acquired locks will be released tm.commit(); {code} Thread #2 does: {code} tm.begin() // We acquire the lock on "B" cache.put("B","B"); // We acquire the lock on "A" cache.put("A","A"); // Here all the acquired locks will be released tm.commit(); {code} To prevent this deadlock we can simply sort the keys within the Tx such that the lock acquisition is done in the same order. Unfortunately when we use lock striping even when we sort the keys we can face deadlocks because we cannot predict how the locks are distributed over the keys. For example the following test case creates a deadlock when the lock striping is enabled. {code} public class TestDeadlock extends TestCase { /** * We get an unpredictable deadlock because apart if we know the implementation * of the ReentrantStripedLockContainer, we cannot know that the keys * 8 * "a" and 13 * "a" are mapped to the same lock and 11 * "a" and 14 * "a" are * both mapped to the same lock that is different from the first one. * In the use case below the keys are sorted but we still get a deadlock */ public void testUseLockStripingTrueUnpredictableDeadLock() throws Exception { DefaultCacheManager manager = new DefaultCacheManager(); manager.getDefaultConfiguration().setInvocationBatchingEnabled(true); manager.getDefaultConfiguration().setLockAcquisitionTimeout(1000); final Cache<String, String> cache = manager.getCache(); final CountDownLatch startSignal = new CountDownLatch(1); final CountDownLatch endSignal = new CountDownLatch(2); final AtomicReference<Exception> exception = new AtomicReference<Exception>(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 8 * "a" cache.put("aaaaaaaa", "Value"); // 14 * "a" cache.put("aaaaaaaaaaaaaa", "Value"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 11 * "a" cache.put("aaaaaaaaaaa", "Value2"); // 13 * "a" cache.put("aaaaaaaaaaaaa", "Value2"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); startSignal.countDown(); endSignal.await(); assertNull(exception.get()); } } {code} If we could get the lock id thanks to the public API, we could sort our keys according to this value and then avoid the deadlocks even when the lock striping is enabled.

Help to prevent deadlocks by making the lock id accessible through the public API --------------------------------------------------------------------------------- Key: ISPN-993 URL: https://issues.jboss.org/browse/ISPN-993 Project: Infinispan Issue Type: Feature Request Components: Locking and Concurrency Reporter: Nicolas Filotto Assignee: Manik Surtani Fix For: 5.0.0.CR5, 5.0.0.FINAL To prevent deadlocks when we use ISPN with transactions, we need to sort the keys before calling a method that acquires a lock such as cache.put and cache.remove. For example when we don't use the lock striping (i.e. we use one lock per key), the following use case will create a deadlock: Thread #1 does: {code} tm.begin() // We acquire the lock on "A" cache.put("A","A"); // We acquire the lock on "B" cache.put("B","B"); // Here all the acquired locks will be released tm.commit(); {code} Thread #2 does: {code} tm.begin() // We acquire the lock on "B" cache.put("B","B"); // We acquire the lock on "A" cache.put("A","A"); // Here all the acquired locks will be released tm.commit(); {code} To prevent this deadlock we can simply sort the keys within the Tx such that the lock acquisition is done in the same order. Unfortunately when we use lock striping even when we sort the keys we can face deadlocks because we cannot predict how the locks are distributed over the keys. For example the following test case creates a deadlock when the lock striping is enabled. {code} public class TestDeadlock extends TestCase { /** * We get an unpredictable deadlock because apart if we know the implementation * of the ReentrantStripedLockContainer, we cannot know that the keys * 8 * "a" and 13 * "a" are mapped to the same lock and 11 * "a" and 14 * "a" are * both mapped to the same lock that is different from the first one. * In the use case below the keys are sorted but we still get a deadlock */ public void testUseLockStripingTrueUnpredictableDeadLock() throws Exception { DefaultCacheManager manager = new DefaultCacheManager(); manager.getDefaultConfiguration().setInvocationBatchingEnabled(true); manager.getDefaultConfiguration().setLockAcquisitionTimeout(1000); final Cache<String, String> cache = manager.getCache(); final CountDownLatch startSignal = new CountDownLatch(1); final CountDownLatch endSignal = new CountDownLatch(2); final AtomicReference<Exception> exception = new AtomicReference<Exception>(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 8 * "a" cache.put("aaaaaaaa", "Value"); // 14 * "a" cache.put("aaaaaaaaaaaaaa", "Value"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); new Thread() { /** * @see java.lang.Thread#run() */ @Override public void run() { try { cache.startBatch(); startSignal.await(); // 11 * "a" cache.put("aaaaaaaaaaa", "Value2"); // 13 * "a" cache.put("aaaaaaaaaaaaa", "Value2"); cache.endBatch(true); } catch (Exception e) { exception.set(e); } finally { endSignal.countDown(); } } }.start(); startSignal.countDown(); endSignal.await(); assertNull(exception.get()); } } {code} If we could get the lock id thanks to the public API, we could sort our keys according to this value and then avoid the deadlocks even when the lock striping is enabled.