Re: [infinispan-dev] parked task?

OK, debuged the problem -- quite similar to ISPN-2232. I'm reusing an existing cache -- via re-start. But that cache is first stopped -- transitioning all the way to LuceneBackendQueueProcessor::close which calls LuceneBackendResources::shutdown which does flushCloseExecutor( queueingExecutor ); meaning it stops / terminates queueingExecutor. But once cache is re-started the queueingExecutor is never re-initialized / re-started. Hence this ThreadPoolExecutor code: public void execute(Runnable command) { if (command == null) throw new NullPointerException(); if (poolSize >= corePoolSize || !addIfUnderCorePoolSize(command)) { if (runState == RUNNING && workQueue.offer(command)) { if (runState != RUNNING || poolSize == 0) ensureQueuedTaskHandled(command); } else if (!addIfUnderMaximumPoolSize(command)) reject(command); // is shutdown or saturated } } * runState == TERMINATED !! * simply puts out command into reject(). And there is nothing to pick it up, as the executor is terminated. Imo, should be easy to replicate this in Ispn Query tests. -Ales On Sep 1, 2012, at 3:59 PM, Ales Justin <ales.justin(a)gmail.com&gt; wrote: > Any idea why this is parked and never goes fwd? > As this then blocks any next cache requests -- see blocked thread below. > Could it be the error below? > > This only happens when there is already an existing CapeDwarf app deployed. > * e.g. (to reproduce) https://github.com/capedwarf/capedwarf-blue/blob/master/testsuite/src/tes... > > The same test exposed ISPN-2232. > > -Ales > > --- > > "http-/192.168.1.101:8080-1" daemon prio=5 tid=0000000002c05000 nid=0xb1ec1000 waiting on condition [00000000b1ebf000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0000000005d3a340> (a java.util.concurrent.FutureTask$Sync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218) > at java.util.concurrent.FutureTask.get(FutureTask.java:83) > at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueProcessor.applyWork(LuceneBackendQueueProcessor.java:97) > at org.hibernate.search.backend.impl.jgroups.JGroupsBackendQueueProcessor.applyWork(JGroupsBackendQueueProcessor.java:116) > at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.performOperations(DirectoryBasedIndexManager.java:127) > at org.hibernate.search.backend.impl.WorkQueuePerIndexSplitter.commitOperations(WorkQueuePerIndexSplitter.java:61) > at org.hibernate.search.backend.impl.BatchedQueueingProcessor.performWorks(BatchedQueueingProcessor.java:96) > at org.hibernate.search.backend.impl.PostTransactionWorkQueueSynchronization.afterCompletion(PostTransactionWorkQueueSynchronization.java:99) > at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96) > at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:402) > - locked <0000000005d3afc8> (a java.lang.Object) > at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103) > at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164) > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165) > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117) > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1013) > at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:714) > at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:184) > at org.infinispan.loaders.CacheLoaderManagerImpl.preload(CacheLoaderManagerImpl.java:223) > at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:236) > at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:891) > at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:641) > at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:630) > at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:533) > at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:174) > at org.infinispan.CacheImpl.start(CacheImpl.java:519) > at org.infinispan.AbstractDelegatingCache.start(AbstractDelegatingCache.java:343) > at org.jboss.capedwarf.common.infinispan.InfinispanUtils.checkCache(InfinispanUtils.java:72) > at org.jboss.capedwarf.common.infinispan.InfinispanUtils.getCache(InfinispanUtils.java:82) > - locked <0000000016847f08> (a $Proxy46) > at org.jboss.capedwarf.common.infinispan.InfinispanUtils.getCache(InfinispanUtils.java:191) > at org.jboss.capedwarf.datastore.AbstractDatastoreService.createStore(AbstractDatastoreService.java:66) > at org.jboss.capedwarf.datastore.AbstractDatastoreService.<init>(AbstractDatastoreService.java:60) > at org.jboss.capedwarf.datastore.JBossDatastoreService.<init>(JBossDatastoreService.java:56) > at com.google.appengine.api.datastore.DatastoreServiceFactory.getDatastoreService(DatastoreServiceFactory.java) > at org.jboss.capedwarf.log.JBossLogService.requestStarted(JBossLogService.java:215) > at org.jboss.capedwarf.appidentity.GAEListener.requestInitialized(GAEListener.java:91) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:372) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:679) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:931) > at java.lang.Thread.run(Thread.java:680) > > "http-/192.168.1.101:8080-2" daemon prio=5 tid=0000000002bd2c00 nid=0xb1fc3000 waiting for monitor entry [00000000b1fc2000] > java.lang.Thread.State: BLOCKED (on object monitor) > at org.jboss.capedwarf.common.infinispan.InfinispanUtils.getCache(InfinispanUtils.java:81) > - waiting to lock <0000000016847f08> (a $Proxy46) > at org.jboss.capedwarf.common.infinispan.InfinispanUtils.getCache(InfinispanUtils.java:191) > at org.jboss.capedwarf.datastore.AbstractDatastoreService.createStore(AbstractDatastoreService.java:66) > at org.jboss.capedwarf.datastore.AbstractDatastoreService.<init>(AbstractDatastoreService.java:60) > at org.jboss.capedwarf.datastore.JBossDatastoreService.<init>(JBossDatastoreService.java:56) > at com.google.appengine.api.datastore.DatastoreServiceFactory.getDatastoreService(DatastoreServiceFactory.java) > at org.jboss.capedwarf.log.JBossLogService.requestStarted(JBossLogService.java:215) > at org.jboss.capedwarf.appidentity.GAEListener.requestInitialized(GAEListener.java:91) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:372) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:679) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:931) > at java.lang.Thread.run(Thread.java:680) > > -- > > 15:46:57,905 ERROR [stderr] (CacheStartThread,null,LuceneIndexesData) Exception in thread "CacheStartThread,null,LuceneIndexesData" org.infinispan.CacheException: Unable to acquire lock on cache with name LuceneIndexesData > 15:46:57,905 ERROR [stderr] (CacheStartThread,null,LuceneIndexesData) at org.infinispan.manager.DefaultCacheManager.wireCache(DefaultCacheManager.java:680) > 15:46:57,906 ERROR [stderr] (CacheStartThread,null,LuceneIndexesData) at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:654) > 15:46:57,906 ERROR [stderr] (CacheStartThread,null,LuceneIndexesData) at org.infinispan.manager.DefaultCacheManager.access$100(DefaultCacheManager.java:127) > 15:46:57,906 ERROR [stderr] (CacheStartThread,null,LuceneIndexesData) at org.infinispan.manager.DefaultCacheManager$1.run(DefaultCacheManager.java:580) > 15:46:57,907 ERROR [stderr] (CacheStartThread,null,LuceneIndexesLocking) Exception in thread "CacheStartThread,null,LuceneIndexesLocking" org.infinispan.CacheException: Unable to acquire lock on cache with name LuceneIndexesLocking > 15:46:57,907 ERROR [stderr] (CacheStartThread,null,LuceneIndexesLocking) at org.infinispan.manager.DefaultCacheManager.wireCache(DefaultCacheManager.java:680) > 15:46:57,908 ERROR [stderr] (CacheStartThread,null,LuceneIndexesLocking) at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:654) > 15:46:57,908 ERROR [stderr] (CacheStartThread,null,LuceneIndexesLocking) at org.infinispan.manager.DefaultCacheManager.access$100(DefaultCacheManager.java:127) > 15:46:57,908 ERROR [stderr] (CacheStartThread,null,LuceneIndexesLocking) at org.infinispan.manager.DefaultCacheManager$1.run(DefaultCacheManager.java:580) > 15:46:57,910 INFO [org.jboss.as.clustering.infinispan] (http-/192.168.1.101:8080-1) JBAS010281: Started LuceneIndexesMetadata cache from capedwarf container > 15:46:57,912 INFO [org.jboss.as.clustering.infinispan] (http-/192.168.1.101:8080-1) JBAS010281: Started LuceneIndexesData cache from capedwarf container > 15:46:57,913 INFO [org.jboss.as.clustering.infinispan] (http-/192.168.1.101:8080-1) JBAS010281: Started LuceneIndexesLocking cache from capedwarf container > 15:46:57,914 INFO [org.hibernate.search.indexes.serialization.avro.impl.AvroSerializationProvider] (http-/192.168.1.101:8080-1) HSEARCH000079: Serialization protocol version 1.0 > 15:46:57,915 ERROR [stderr] (CacheStartThread,null,LuceneIndexesMetadata) Exception in thread "CacheStartThread,null,LuceneIndexesMetadata" org.infinispan.CacheException: Unable to acquire lock on cache with name LuceneIndexesMetadata > 15:46:57,916 ERROR [stderr] (CacheStartThread,null,LuceneIndexesMetadata) at org.infinispan.manager.DefaultCacheManager.wireCache(DefaultCacheManager.java:680) > 15:46:57,916 ERROR [stderr] (CacheStartThread,null,LuceneIndexesMetadata) at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:654) > 15:46:57,916 ERROR [stderr] (CacheStartThread,null,LuceneIndexesMetadata) at org.infinispan.manager.DefaultCacheManager.access$100(DefaultCacheManager.java:127) > 15:46:57,916 ERROR [stderr] (CacheStartThread,null,LuceneIndexesMetadata) at org.infinispan.manager.DefaultCacheManager$1.run(DefaultCacheManager.java:580) > >