[infinispan-dev] Concurrency problem on org.infinispan.transaction.xa.TransactionTable

Hi all! We are Diego and Sebastiano and we are working in the Cloud-TM project; in particular we are focusing on designing new replication schemes in Infinispan. During our code inspection we have noted that Infinispan can suffer from a concurrency issue: multiple threads in execution on the same local cache instance can access without any synchronization scheme on a org.infinispan.transaction.xa.TransactionTable's field named localTransactions; the problem is that the aforementioned localTransactions field is an instance of HashMap, which isn't thread-safe. We have noticed the problem during some intensive tests (executed with a version of RadarGun modified by us in order to enable transactions executions) which stressed heavily the cache with concurrent threads: in some cases all threads on some nodes blocked in an infinite loop on a get() operation on the aforementioned HashMap instance. We attach the dump relevant to a single blocked node and you can note that each thread, namely a Stresser, is blocked on the get() operation. We have noted the same concurrency problem on the field remoteTransactions in the same object. To partially validate our hypothesis we have replaced the type HashMap with ConcurrentHashMap and everithing works fine. What is your opinion about this? Best regards Diego, Sebastiano "Stresser-7" prio=10 tid=0x00007fb2c0295000 nid=0x5bd3 runnable [0x00007fb2c6bea000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(HashMap.java:303) at org.infinispan.transaction.xa.TransactionTable.getXaCacheAdapter(TransactionTable.java:222) at org.infinispan.context.InvocationContextContainerImpl.createInvocationContext(InvocationContextContainerImpl.java:67) at org.infinispan.CacheDelegate.getInvocationContext(CacheDelegate.java:285) at org.infinispan.CacheDelegate.get(CacheDelegate.java:199) at org.radargun.cachewrappers.InfinispanWrapper.get(InfinispanWrapper.java:99) at org.radargun.stressors.PutGetStressor$Stresser.run(PutGetStressor.java:367) "Stresser-6" prio=10 tid=0x00007fb2c0293000 nid=0x5bd2 runnable [0x00007fb2c6ceb000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(HashMap.java:303) at org.infinispan.transaction.xa.TransactionTable.getXaCacheAdapter(TransactionTable.java:222) at org.infinispan.context.InvocationContextContainerImpl.createInvocationContext(InvocationContextContainerImpl.java:67) at org.infinispan.CacheDelegate.getInvocationContext(CacheDelegate.java:285) at org.infinispan.CacheDelegate.get(CacheDelegate.java:199) at org.radargun.cachewrappers.InfinispanWrapper.get(InfinispanWrapper.java:99) at org.radargun.stressors.PutGetStressor$Stresser.run(PutGetStressor.java:367) "Stresser-5" prio=10 tid=0x00007fb2c0292000 nid=0x5bd1 runnable [0x00007fb2c6dec000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(HashMap.java:303) at org.infinispan.transaction.xa.TransactionTable.getXaCacheAdapter(TransactionTable.java:222) at org.infinispan.context.InvocationContextContainerImpl.createInvocationContext(InvocationContextContainerImpl.java:67) at org.infinispan.CacheDelegate.getInvocationContext(CacheDelegate.java:285) at org.infinispan.CacheDelegate.get(CacheDelegate.java:199) at org.radargun.cachewrappers.InfinispanWrapper.get(InfinispanWrapper.java:99) at org.radargun.stressors.PutGetStressor$Stresser.run(PutGetStressor.java:367) "Stresser-4" prio=10 tid=0x00007fb2c0291800 nid=0x5bd0 runnable [0x00007fb2c77f6000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(HashMap.java:303) at org.infinispan.transaction.xa.TransactionTable.getXaCacheAdapter(TransactionTable.java:222) at org.infinispan.context.InvocationContextContainerImpl.createInvocationContext(InvocationContextContainerImpl.java:67) at org.infinispan.CacheDelegate.getInvocationContext(CacheDelegate.java:285) at org.infinispan.CacheDelegate.get(CacheDelegate.java:199) at org.radargun.cachewrappers.InfinispanWrapper.get(InfinispanWrapper.java:99) at org.radargun.stressors.PutGetStressor$Stresser.run(PutGetStressor.java:367) "Stresser-3" prio=10 tid=0x00007fb2c0290800 nid=0x5bcf runnable [0x00007fb2c78f7000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(HashMap.java:303) at org.infinispan.transaction.xa.TransactionTable.getXaCacheAdapter(TransactionTable.java:222) at org.infinispan.context.InvocationContextContainerImpl.createInvocationContext(InvocationContextContainerImpl.java:67) at org.infinispan.CacheDelegate.getInvocationContext(CacheDelegate.java:285) at org.infinispan.CacheDelegate.get(CacheDelegate.java:199) at org.radargun.cachewrappers.InfinispanWrapper.get(InfinispanWrapper.java:99) at org.radargun.stressors.PutGetStressor$Stresser.run(PutGetStressor.java:367) "Stresser-2" prio=10 tid=0x00007fb2c0044000 nid=0x5bce runnable [0x00007fb2c76f5000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(HashMap.java:303) at org.infinispan.transaction.xa.TransactionTable.getXaCacheAdapter(TransactionTable.java:222) at org.infinispan.context.InvocationContextContainerImpl.createInvocationContext(InvocationContextContainerImpl.java:67) at org.infinispan.CacheDelegate.getInvocationContext(CacheDelegate.java:285) at org.infinispan.CacheDelegate.get(CacheDelegate.java:199) at org.radargun.cachewrappers.InfinispanWrapper.get(InfinispanWrapper.java:99) at org.radargun.stressors.PutGetStressor$Stresser.run(PutGetStressor.java:367) "Stresser-1" prio=10 tid=0x00007fb2c0043000 nid=0x5bcd runnable [0x00007fb2c74f3000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(HashMap.java:303) at org.infinispan.transaction.xa.TransactionTable.getXaCacheAdapter(TransactionTable.java:222) at org.infinispan.context.InvocationContextContainerImpl.createInvocationContext(InvocationContextContainerImpl.java:67) at org.infinispan.CacheDelegate.getInvocationContext(CacheDelegate.java:285) at org.infinispan.CacheDelegate.get(CacheDelegate.java:199) at org.radargun.cachewrappers.InfinispanWrapper.get(InfinispanWrapper.java:99) at org.radargun.stressors.PutGetStressor$Stresser.run(PutGetStressor.java:367) "Stresser-0" prio=10 tid=0x00007fb2c0048000 nid=0x5bcc runnable [0x00007fb2c75f4000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.get(HashMap.java:303) at org.infinispan.transaction.xa.TransactionTable.getXaCacheAdapter(TransactionTable.java:222) at org.infinispan.context.InvocationContextContainerImpl.createInvocationContext(InvocationContextContainerImpl.java:67) at org.infinispan.CacheDelegate.getInvocationContext(CacheDelegate.java:285) at org.infinispan.CacheDelegate.get(CacheDelegate.java:199) at org.radargun.cachewrappers.InfinispanWrapper.get(InfinispanWrapper.java:99) at org.radargun.stressors.PutGetStressor$Stresser.run(PutGetStressor.java:367)