[JBoss JIRA] (ISPN-9412) HotRod Transaction Cache PESSIMISTIC locking mode is not locking the key in another transaction
by Diego Lovison (JIRA)
[ https://issues.jboss.org/browse/ISPN-9412?page=com.atlassian.jira.plugin.... ]
Diego Lovison edited comment on ISPN-9412 at 8/9/18 8:39 AM:
-------------------------------------------------------------
I would like to be able to reproduce this error in an integration test.
I created the method `public void testMultiThreadsPessimisticLockMode(Method method)` in the class `TxFunctionalTest`
{code:java}
public void testMultiThreadsPessimisticLockMode(Method method) throws ExecutionException, InterruptedException {
final K k1 = kvGenerator.generateKey(method, 1);
final V v1 = kvGenerator.generateValue(method, 1);
final V v2 = kvGenerator.generateValue(method, 2);
RemoteCache<K, V> remoteCache = remoteCacheWithForceReturnValue();
final TransactionManager tm = remoteCache.getTransactionManager();
ExecutorService executorService = Executors.newFixedThreadPool(2);
Future<V> f1 = executorService.submit(() -> {
try {
tm.begin();
V returnValue = remoteCache.put(k1, v1);
Thread.sleep(2000);
tm.commit();
return returnValue;
} catch (Throwable t) {
try {
tm.rollback();
} catch (SystemException e) {
throw new IllegalStateException(e);
}
throw new IllegalStateException(t);
}
});
Future<V> f2 = executorService.submit(() -> {
try {
Thread.sleep(500);
tm.begin();
V returnValue = remoteCache.put(k1, v2);
tm.commit();
return returnValue;
} catch (Throwable t) {
try {
tm.rollback();
} catch (SystemException e) {
throw new IllegalStateException(e);
}
throw new IllegalStateException(t);
}
});
Assert.assertNull(f1.get());
Assert.assertNotNull(f2.get());
}
{code}
The problem is that my integration test is failing due:
{noformat}
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.infinispan.client.hotrod.tx.TxFunctionalTest.testMultiThreadsPessimisticLockMode(TxFunctionalTest.java:473)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
at org.testng.internal.MethodInvocationHelper$1.runTestMethod(MethodInvocationHelper.java:196)
at org.infinispan.commons.test.TestNGLongTestsHook.run(TestNGLongTestsHook.java:24)
at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:208)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:635)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:816)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1124)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108)
at org.testng.TestRunner.privateRun(TestRunner.java:774)
at org.testng.TestRunner.run(TestRunner.java:624)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:359)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:354)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:312)
at org.testng.SuiteRunner.run(SuiteRunner.java:261)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1215)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1140)
at org.testng.TestNG.run(TestNG.java:1048)
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)
Caused by: java.lang.IllegalStateException: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at org.infinispan.client.hotrod.tx.TxFunctionalTest.lambda$testMultiThreadsPessimisticLockMode$1(TxFunctionalTest.java:457)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at org.infinispan.test.TestingUtil.lambda$withTxCallable$11(TestingUtil.java:1529)
at org.infinispan.test.TestingUtil.withTx(TestingUtil.java:1507)
at org.infinispan.client.hotrod.tx.TxFunctionalTest.lambda$testMultiThreadsPessimisticLockMode$1(TxFunctionalTest.java:451)
... 5 more
Caused by: java.lang.Throwable: setRollbackOnly called from:
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.setRollbackOnly(TransactionImple.java:339)
at org.infinispan.client.hotrod.impl.transaction.SyncModeTransactionTable$SynchronizationAdapter.markAsRollback(SyncModeTransactionTable.java:150)
at org.infinispan.client.hotrod.impl.transaction.SyncModeTransactionTable$SynchronizationAdapter.beforeCompletion(SyncModeTransactionTable.java:125)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:371)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
... 9 more
{noformat}
Here is the branch: https://github.com/diegolovison/infinispan/tree/ISPN_9412_pessimistic_lock
----
If I do the following change:
1 - Create additional methods
{code:java}
private RemoteCache<K, V> remoteCacheWithForceReturnValue() {
return remoteCacheWithForceReturnValue(true);
}
private RemoteCache<K, V> remoteCacheWithForceReturnValue(boolean forceReturnValue) {
return client(0).getCache(cacheName(), forceReturnValue);
}
{code}
2 - Replace
{code:java}
RemoteCache<K, V> remoteCache = remoteCacheWithForceReturnValue();
{code}
with
{code:java}
RemoteCache<K, V> remoteCache = remoteCacheWithForceReturnValue(false);
{code}
3 - The error go away but we won't be able to test because the values are null
was (Author: dlovison):
I would like to be able to reproduce this error in an integration test.
I created the method `public void testMultiThreadsPessimisticLockMode(Method method)` in the class `TxFunctionalTest`
{code:java}
public void testMultiThreadsPessimisticLockMode(Method method) throws ExecutionException, InterruptedException {
final K k1 = kvGenerator.generateKey(method, 1);
final V v1 = kvGenerator.generateValue(method, 1);
final V v2 = kvGenerator.generateValue(method, 2);
RemoteCache<K, V> remoteCache = remoteCacheWithForceReturnValue();
final TransactionManager tm = remoteCache.getTransactionManager();
ExecutorService executorService = Executors.newFixedThreadPool(2);
Future<V> f1 = executorService.submit(() -> {
try {
tm.begin();
V returnValue = remoteCache.put(k1, v1);
Thread.sleep(2000);
tm.commit();
return returnValue;
} catch (Throwable t) {
try {
tm.rollback();
} catch (SystemException e) {
throw new IllegalStateException(e);
}
throw new IllegalStateException(t);
}
});
Future<V> f2 = executorService.submit(() -> {
try {
Thread.sleep(500);
tm.begin();
V returnValue = remoteCache.put(k1, v2);
tm.commit();
return returnValue;
} catch (Throwable t) {
try {
tm.rollback();
} catch (SystemException e) {
throw new IllegalStateException(e);
}
throw new IllegalStateException(t);
}
});
Assert.assertNull(f1.get());
Assert.assertNotNull(f2.get());
}
{code}
The problem is that my integration test is failing due:
{noformat}
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: BaseTransaction.rollback - ARJUNA016074: no transaction!
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.infinispan.client.hotrod.tx.TxFunctionalTest.testMultiThreadsPessimisticLockMode(TxFunctionalTest.java:456)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
at org.testng.internal.MethodInvocationHelper$1.runTestMethod(MethodInvocationHelper.java:196)
at org.infinispan.commons.test.TestNGLongTestsHook.run(TestNGLongTestsHook.java:24)
at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:208)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:635)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:816)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1124)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108)
at org.testng.TestRunner.privateRun(TestRunner.java:774)
at org.testng.TestRunner.run(TestRunner.java:624)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:359)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:354)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:312)
at org.testng.SuiteRunner.run(SuiteRunner.java:261)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1215)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1140)
at org.testng.TestNG.run(TestNG.java:1048)
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)
Caused by: java.lang.IllegalStateException: BaseTransaction.rollback - ARJUNA016074: no transaction!
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:139)
at org.infinispan.client.hotrod.tx.TxFunctionalTest.lambda$testMultiThreadsPessimisticLockMode$0(TxFunctionalTest.java:432)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
Here is the branch: https://github.com/diegolovison/infinispan/tree/ISPN_9412_pessimistic_lock
----
If I do the following change:
1 - Create additional methods
{code:java}
private RemoteCache<K, V> remoteCacheWithForceReturnValue() {
return remoteCacheWithForceReturnValue(true);
}
private RemoteCache<K, V> remoteCacheWithForceReturnValue(boolean forceReturnValue) {
return client(0).getCache(cacheName(), forceReturnValue);
}
{code}
2 - Replace
{code:java}
RemoteCache<K, V> remoteCache = remoteCacheWithForceReturnValue();
{code}
with
{code:java}
RemoteCache<K, V> remoteCache = remoteCacheWithForceReturnValue(false);
{code}
3 - The error go away but we won't be able to test because the values are null
> HotRod Transaction Cache PESSIMISTIC locking mode is not locking the key in another transaction
> -----------------------------------------------------------------------------------------------
>
> Key: ISPN-9412
> URL: https://issues.jboss.org/browse/ISPN-9412
> Project: Infinispan
> Issue Type: Bug
> Reporter: Diego Lovison
> Priority: Critical
>
> I am expecting the thread 2 be blocked in `cache.put(k1, v1)` until thread 1 has committed the transaction
> According to the docs, using Pessimistic transactional cache, when cache.put(k1,v1) returns, k1 is locked and no other transaction running anywhere in the cluster can write to it. Reading k1 is still possible. The lock on k1 is released when the transaction completes (commits or rollbacks).
> Cache Configuration
> {code:java}
> return new org.infinispan.configuration.cache.ConfigurationBuilder()
> .clustering().cacheMode(CacheMode.DIST_SYNC)
> .jmxStatistics().enable()
> .transaction()
> .cacheStopTimeout(0L)
> .transactionManagerLookup(new EmbeddedTransactionManagerLookup())
> .lockingMode(LockingMode.PESSIMISTIC)
> .locking().isolationLevel(IsolationLevel.REPEATABLE_READ)
> .build();
> {code}
> Steps to reproduce
> {noformat}
> Thread t1 = {
> tm.begin()
> cache.put(k1, v1)
> long operation like 1 minute
> tm.commit()
> }
> Thread t2 = {
> long operation like 15 seconds
> tm.begin()
> cache.put(k1, v1)
> tm.commit()
> }
> t1.start()
> t2.start()
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 8 months
[JBoss JIRA] (ISPN-9416) Support duplicates in Multimap
by Katia Aresti (JIRA)
Katia Aresti created ISPN-9416:
----------------------------------
Summary: Support duplicates in Multimap
Key: ISPN-9416
URL: https://issues.jboss.org/browse/ISPN-9416
Project: Infinispan
Issue Type: Enhancement
Components: Multimap
Reporter: Katia Aresti
Assignee: Katia Aresti
The cache Multimap does not support duplicates. Make the duplicates support available
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 8 months
[JBoss JIRA] (ISPN-9415) Client topology is not updated after cache becomes degraded
by Dan Berindei (JIRA)
Dan Berindei created ISPN-9415:
----------------------------------
Summary: Client topology is not updated after cache becomes degraded
Key: ISPN-9415
URL: https://issues.jboss.org/browse/ISPN-9415
Project: Infinispan
Issue Type: Bug
Components: Server
Affects Versions: 9.3.1.Final, 9.4.0.Beta1
Reporter: Dan Berindei
Assignee: Dan Berindei
Fix For: 9.4.0.CR1
When a new server is started, or after a merge, the other servers may see it as a an owner in the consistent hash before the other servers see its server address in the address cache ({{___hotRodTopologyCache}}). When a server needs to send a topology update but some of the servers are missing from the address cache, it can't send the topology update, so it tries to send a "partial update" that excludes the missing servers from the segment owners. In order to send the full topology update when the address cache is populated, the partial topology update has to be sent a smaller topology id, and that means it is only send if {{serverTopologyId >= clientTopologyId + 2}}.
When the cluster splits and the cache becomes degraded, the servers in the other partition are removed from the address cache, but the list of segment owners is not updated, and the topology id is only incremented by 1. The address cache is incomplete, but a partial update cannot be sent, so the client keeps the old topology and keeps trying to connect to the servers in the other partition.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 8 months