[infinispan-issues] [JBoss JIRA] (ISPN-5270) Deadlock in InfinispanDirectoryProvider startup

[Sanne Grinovero (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049092#comment-13049092 ] 

Sanne Grinovero commented on ISPN-5270:
---------------------------------------

Great detective work :) Now I understand.

The Infinispan {{DirectoryProvider}} seeks for the 3 caches which have default (by convention) names to store the index, and since the default configuration is indexed those 3 caches will each trigger another bootstrap of the Search engine, triggering again the DP initialization and requiring the 3 caches to be started. We normally don't initialize the DP eagerly, but the {{ProgrammaticSearchMappingProvider}} requires us to start it.

The workaround is simple: never ever enable indexing on the default cache.

I see different solutions possible:
# we actively validate against the caches being used for index storage to be used for indexing as well
# Initialization of the DP is wrapped into a Future?
# It could be very helpeful to users if we could start the 3 "conventional" index storage caches using a configuration which we control (not inherit)

I don't like the first solution because - even if it makes sense for production use - as it makes for yet another thing people need to take care of, especially for a quick POC I don't see why we should prevent people to store it all in one cache.

More ideas?

> Deadlock in InfinispanDirectoryProvider startup
> -----------------------------------------------
>
>                 Key: ISPN-5270
>                 URL: https://issues.jboss.org/browse/ISPN-5270
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Embedded Querying
>    Affects Versions: 7.2.0.Alpha1, 7.1.1.Final
>            Reporter: Dan Berindei
>            Assignee: Gustavo Fernandes
>            Priority: Minor
>         Attachments: surefire.stacks, surefire2.stacks
>
>
> The InfinispanDirectoryProvider tries to start the metadata, data, and locking caches when it starts up, with {{DefaultCacheManager.startCaches()}}.
> However, when one of these caches (e.g. the metadata cache) starts, the {{LifecycleManager.cacheStarting()}}, which can then try to start the InfinispanDirectoryProvider again:
> {noformat}
> "CacheStartThread,null,LuceneIndexesMetadata" prio=10 tid=0x00007f5f74484000 nid=0xe42 in Object.wait() [0x00007f5efff48000]
>    java.lang.Thread.State: WAITING (on object monitor)
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00000000c2180000> (a org.infinispan.manager.DefaultCacheManager$1)
> 	at java.lang.Thread.join(Thread.java:1281)
> 	- locked <0x00000000c2180000> (a org.infinispan.manager.DefaultCacheManager$1)
> 	at java.lang.Thread.join(Thread.java:1355)
> 	at org.infinispan.manager.DefaultCacheManager.startCaches(DefaultCacheManager.java:465)
> 	at org.hibernate.search.infinispan.spi.InfinispanDirectoryProvider.start(InfinispanDirectoryProvider.java:84)
> 	at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:88)
> 	at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)
> 	at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:513)
> 	- locked <0x00000000ce6001d0> (a org.hibernate.search.indexes.impl.IndexManagerHolder)
> 	at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManagers(IndexManagerHolder.java:482)
> 	at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:91)
> 	- locked <0x00000000ce6001d0> (a org.hibernate.search.indexes.impl.IndexManagerHolder)
> 	at org.hibernate.search.spi.SearchIntegratorBuilder.initDocumentBuilders(SearchIntegratorBuilder.java:366)
> 	at org.hibernate.search.spi.SearchIntegratorBuilder.buildNewSearchFactory(SearchIntegratorBuilder.java:204)
> 	at org.hibernate.search.spi.SearchIntegratorBuilder.buildSearchIntegrator(SearchIntegratorBuilder.java:122)
> 	at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:35)
> 	at org.infinispan.query.impl.LifecycleManager.getSearchFactory(LifecycleManager.java:260)
> 	at org.infinispan.query.impl.LifecycleManager.cacheStarting(LifecycleManager.java:102)
> 	at org.infinispan.factories.ComponentRegistry.notifyCacheStarting(ComponentRegistry.java:230)
> 	at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
> 	at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:814)
> 	at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:591)
> 	at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:546)
> 	at org.infinispan.manager.DefaultCacheManager.access$100(DefaultCacheManager.java:115)
> 	at org.infinispan.manager.DefaultCacheManager$1.run(DefaultCacheManager.java:452)
> {noformat}
> This can hang the test, the attached thread dumps show {{EmbeddedCompatTest}} and {{IndexCacheStopTest}}.



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)