[infinispan-issues] [JBoss JIRA] (ISPN-3938) AdvancedAsyncCacheLoader.process() concurrency issues

[Dan Berindei (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034636#comment-13034636 ] 

Dan Berindei commented on ISPN-3938:
------------------------------------

Will, I think you're right, normally there shouldn't be any other thread modifying the {{HashSet}} at the time of the iteration, so this is probably a problem in the underlying loader implementation and/or in {{ExecutorAllCompletionService}}. I'm not sure if we can do anything in {{AdvancedAsyncCacheLoader}} to guard against it...

However, the usage of {{advancedLoader().process()}} in {{loadAllKeys()}} still doesn't look thread-safe, so I'd replace the {{allKeys}} set with a {{ConcurrentMap}}.

> AdvancedAsyncCacheLoader.process() concurrency issues
> -----------------------------------------------------
>
>                 Key: ISPN-3938
>                 URL: https://issues.jboss.org/browse/ISPN-3938
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Core, Loaders and Stores
>    Affects Versions: 6.0.0.Final
>            Reporter: Dan Berindei
>            Assignee: William Burns
>             Fix For: 7.1.0.Final
>
>
> {{AdvancedAsyncCacheLoader.process()}} calls {{advancedLoader().process()}} to collect all the keys in the store, but the HashSet used to collect the keys it not thread-safe. This can cause problems, e.g. during state transfer:
> {noformat}
> WARN  cheTopologyControlCommand | ISPN000071: Caught exception when handling command CacheTopologyControlCommand{cache=sessions, type=CH_UPDATE, sender=alfie-lt-46127, joinInfo=null, topologyId=3, currentCH=DefaultConsistentHash{numSegments=60, numOwners=1, members=[alfie-lt-46127]}, pendingCH=null, throwable=null, viewId=1}java.util.ConcurrentModificationException
>   at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
>   at java.util.HashMap$KeyIterator.next(HashMap.java:960)
>   at org.infinispan.persistence.async.AdvancedAsyncCacheLoader.process(AdvancedAsyncCacheLoader.java:80)
>   at org.infinispan.persistence.manager.PersistenceManagerImpl.processOnAllStores(PersistenceManagerImpl.java:414)
>   at org.infinispan.statetransfer.StateConsumerImpl.invalidateSegments(StateConsumerImpl.java:910)
>   at org.infinispan.statetransfer.StateConsumerImpl.onTopologyUpdate(StateConsumerImpl.java:393)
>   at org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:178)
>   at org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:38)
>   at org.infinispan.statetransfer.StateTransferManagerImpl$1.updateConsistentHash(StateTransferManagerImpl.java:100)
>   at org.infinispan.topology.LocalTopologyManagerImpl.handleConsistentHashUpdate(LocalTopologyManagerImpl.java:191)
>   at org.infinispan.topology.CacheTopologyControlCommand.doPerform(CacheTopologyControlCommand.java:152)
>   at org.infinispan.topology.CacheTopologyControlCommand.perform(CacheTopologyControlCommand.java:124)
>   at org.infinispan.topology.ClusterTopologyManagerImpl$3.run(ClusterTopologyManagerImpl.java:606)
>   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:744)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)