[infinispan-issues] [JBoss JIRA] (ISPN-9400) LevelDB is initialized when a configuration template refers to it
Tristan Tarrant (Jira)
issues at jboss.org
Mon Oct 28 03:26:01 EDT 2019
[ https://issues.jboss.org/browse/ISPN-9400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tristan Tarrant updated ISPN-9400:
----------------------------------
Fix Version/s: (was: 10.1.0.Final)
> LevelDB is initialized when a configuration template refers to it
> -----------------------------------------------------------------
>
> Key: ISPN-9400
> URL: https://issues.jboss.org/browse/ISPN-9400
> Project: Infinispan
> Issue Type: Bug
> Components: Server
> Reporter: Alan Field
> Assignee: Will Burns
> Priority: Major
>
> For the scalability tests, I am running 25 instances of the server on a node. I have been using the information in JIRA JDG-1504 to make this work. I added the following in my server configuration file to keep the data directory unique for each instance running on a node:
> {noformat}
> <global-state>
> <persistent-location path="data-${jboss.node.name}" />
> </global-state>
> {noformat}
> The server is starting up without errors for the global state, but it is showing the following error:
> {noformat}
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:31,955 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-6) MSC000001: Failed to start service jboss.datagrid-infinispan-endpoint.hotrod.hotrod-connec ctor: org.jboss.msc.service.StartException in service jboss.datagrid-infinispan-endpoint.hotrod.hotrod-connector: DGENDPT10004: Failed to start HotRodServer
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.endpoint.subsystem.ProtocolServerService.start(ProtocolServerService.java:157)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:2032)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1955)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at java.lang.Thread.run(Thread.java:748)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object o of type PersistenceManagerImplengineering.redhat.com] out:
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:172)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:878)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:643)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:630)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:549)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:232)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:891)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:676)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:620)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:506)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:492)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:525)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:516)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:94)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:85)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.security.actions.GetCacheAction.run(GetCacheAction.java:26)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.security.actions.GetCacheAction.run(GetCacheAction.java:14)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.security.Security.doPrivileged(Security.java:81)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.hotrod.SecurityActions.doPrivileged(SecurityActions.java:34)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.hotrod.SecurityActions.getCache(SecurityActions.java:51)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.hotrod.HotRodServer.getCacheInstance(HotRodServer.java:384)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.hotrod.HotRodServer.preStartCaches(HotRodServer.java:314)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.hotrod.HotRodServer.startTransport(HotRodServer.java:296)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.core.AbstractProtocolServer.startInternal(AbstractProtocolServer.java:61)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.hotrod.HotRodServer.startInternal(HotRodServer.java:230)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.hotrod.HotRodServer.startInternal(HotRodServer.java:95)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.core.AbstractProtocolServer.start(AbstractProtocolServer.java:79)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.endpoint.subsystem.SecurityActions$6.run(SecurityActions.java:136)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.endpoint.subsystem.SecurityActions$6.run(SecurityActions.java:133)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.security.Security.doPrivileged(Security.java:76)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.endpoint.subsystem.SecurityActions.doPrivileged(SecurityActions.java:42)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.endpoint.subsystem.SecurityActions.startProtocolServer(SecurityActions.java:140)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.endpoint.subsystem.ProtocolServerService.startProtocolServer(ProtocolServerService.java:191)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.server.endpoint.subsystem.ProtocolServerService.start(ProtocolServerService.java:149)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: ... 5 more
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:199)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at java.lang.reflect.Method.invoke(Method.java:498)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: ... 39 more
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: Caused by: org.infinispan.commons.CacheConfigurationException: Unable to open database
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.persistence.leveldb.LevelDBStore.start(LevelDBStore.java:109)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.persistence.manager.PersistenceManagerImpl.lambda$start$0(PersistenceManagerImpl.java:163)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at java.util.ArrayList.forEach(ArrayList.java:1257)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:179)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: ... 43 more
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: Caused by: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: lock /root/jboss-datagrid-7.2.0-server/standalone/data/datagrid-infinispan/clustered/ /datapersistent-leveldb-store/LOCK: Resource temporarily unavailable
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.persistence.leveldb.LevelDBStore.openDatabase(LevelDBStore.java:154)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: at org.infinispan.persistence.leveldb.LevelDBStore.start(LevelDBStore.java:105)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: ... 46 more
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out:
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:31,965 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: ("subsystem" => "datagrid-infinispan-endpoint"),
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: ("hotrod-connector" => "hotrod-connector")
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: ]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.datagrid-infinispan-endpoint.hotrod.hotrod-connector" => "org.jboss.msc.service.StartException n in service jboss.datagrid-infinispan-endpoint.hotrod.hotrod-connector: DGENDPT10004: Failed to start HotRodServer
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on obje ect of type PersistenceManagerImplneering.redhat.com] out:
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: Caused by: org.infinispan.commons.CacheConfigurationException: Unable to open database
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: Caused by: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: lock /root/jboss-datagrid-7.2.0-server/standalone/data/datagrid-infinispan/cluste ered/datapersistent-leveldb-store/LOCK: Resource temporarily unavailable"}}
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:32,019 INFO [org.jboss.as.controller] (Controller Boot Thread) WFLYCTL0183: Service status report
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: WFLYCTL0186: Services which failed to start: service jboss.datagrid-infinispan-endpoint.hotrod.hotrod-connector: org.jboss.msc.service.StartException in ser rvice jboss.datagrid-infinispan-endpoint.hotrod.hotrod-connector: DGENDPT10004: Failed to start HotRodServer
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out:
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:32,219 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:10013/management
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:32,220 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:10013
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:32,220 ERROR [org.jboss.as] (Controller Boot Thread) WFLYSRV0026: Data Grid 7.2.0 (WildFly Core 2.1.18.Final-redhat-1) started (with errors) in 10819ms - Sta arted 199 of 243 services (1 services failed or missing dependencies, 126 services are lazy, passive or on-demand)
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:32,221 WARN [org.radargun.service.InfinispanServerLifecycle] (Thread-2) Server started with errors
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:32,267 INFO [org.radargun.stages.lifecycle.LifecycleHelper] (sc-main) No members found in the cluster. Expected: 256
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:33,296 INFO [org.radargun.stages.lifecycle.LifecycleHelper] (sc-main) No members found in the cluster. Expected: 256
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:33,958 INFO [org.radargun.service.Infinispan80ServerService] (Thread-2) All Java PIDs: '153566'
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:33,958 INFO [org.radargun.service.Infinispan80ServerService] (Thread-2) Java PIDs: '153566'
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14:51:34,339 ERROR [org.radargun.service.InfinispanServerClustered] (sc-main) Failed to find CacheManager MBean using domain 'org.infinispan' and name 'default'. T Trying again using type.stack.engineering.redhat.com] out:
> [b05-h06-1029p.rdu.openstack.engineering.redhat.com] out: 14.482: [GC pause (G1 Evacuation Pause) (young), 0.2065067 secs]
> {noformat}
> The only reference to LevelDB is in this configuration template:
> {noformat}
> <distributed-cache-configuration name="persistent-leveldb-store">
> <leveldb-store shared="false" fetch-state="true" passivation="false"/>
> </distributed-cache-configuration>
> {noformat}
> Commenting this template out makes the error message go away. I don't think that LevelDB should be initialized until it is being used.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
More information about the infinispan-issues
mailing list