[jboss-jira] [JBoss JIRA] (WFLY-10277) ISPN005031: The supplied configuration for cache 'xyz' is missing a declaration named 'configuration' in session offloading scenario with hotrod store configured, using JDG as remote store
Michal Vinkler (JIRA)
issues at jboss.org
Fri Apr 20 08:24:01 EDT 2018
[ https://issues.jboss.org/browse/WFLY-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michal Vinkler updated WFLY-10277:
----------------------------------
Description:
We hit incompatibility issue when running tests for RFE EAP7-942.
Wildfly was built from https://github.com/rhusar/wildfly/tree/WFLY-9294
Scenario:
2-node Wildfly cluster + 2-node JDG 7.2.0.CR1 cluster.
Wildfly servers are configured to use hotrod store which is backed up by JDG cluster.
All the servers are configured to use SSL secured connection.
Wildfly configuration:
http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/artifact/report/config/jboss-perf18/standalone-ha.xml
JDG configuration:
http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/artifact/report/config/jboss-perf20/clustered.xml
When the cache is started on Wildfly servers, they log this error:
{code}
[JBossINF] [0m[33m07:57:56,227 WARN [org.infinispan.client.hotrod.impl.protocol.Codec21] (HotRod-client-async-pool-2) ISPN004005: Error received from the server: org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
{code}
Full stacktrace:
{code}
[JBossINF] [0m[31m07:57:56,231 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 75) MSC000001: Failed to start service org.wildfly.clustering.infinispan.cache.web.clusterbench-ee7.ear.cb.war: org.jboss.msc.service.StartException in service org.wildfly.clustering.infinispan.cache.web.clusterbench-ee7.ear.cb.war: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
[JBossINF] at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:70)
[JBossINF] at org.wildfly.clustering.service.AsynchronousServiceBuilder.lambda$start$0(AsynchronousServiceBuilder.java:99)
[JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
[JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
[JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
[JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1349)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
[JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:485)
[JBossINF] Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
[JBossINF] at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:83)
[JBossINF] at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
[JBossINF] at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
[JBossINF] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
[JBossINF] at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
[JBossINF] at org.infinispan.security.Security.doPrivileged(Security.java:76)
[JBossINF] at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
[JBossINF] at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:229)
[JBossINF] at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:1020)
[JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:421)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:644)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:589)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:475)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:461)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:447)
[JBossINF] at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:86)
[JBossINF] at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.get(CacheBuilder.java:71)
[JBossINF] at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.get(CacheBuilder.java:54)
[JBossINF] at org.wildfly.clustering.service.SuppliedValueService.lambda$new$0(SuppliedValueService.java:42)
[JBossINF] at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:68)
[JBossINF] ... 7 more
[JBossINF] Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders
[JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:179)
[JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[JBossINF] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[JBossINF] at java.lang.reflect.Method.invoke(Method.java:498)
[JBossINF] at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
[JBossINF] ... 31 more
[JBossINF] Caused by: org.infinispan.persistence.spi.PersistenceException: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=10 returned server error (status=0x85): org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF] at org.jboss.as.clustering.infinispan.subsystem.remote.HotRodStore.init(HotRodStore.java:78)
[JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.initializeBareInstance(PersistenceManagerImpl.java:752)
[JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:671)
[JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:132)
[JBossINF] ... 36 more
[JBossINF] Caused by: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=10 returned server error (status=0x85): org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:383)
[JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:177)
[JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:163)
[JBossINF] at org.infinispan.client.hotrod.impl.transport.netty.HeaderDecoder.decode(HeaderDecoder.java:85)
[JBossINF] at org.infinispan.client.hotrod.impl.transport.netty.HintedReplayingDecoder.callDecode(HintedReplayingDecoder.java:98)
[JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[JBossINF] at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[JBossINF] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1240)
[JBossINF] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1041)
[JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
[JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[JBossINF] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[JBossINF] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
[JBossINF] at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:1017)
[JBossINF] at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:394)
[JBossINF] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:299)
[JBossINF] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
[JBossINF]
{code}
Wildfly Server link:
https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/console-perf18/
The cache is not started and the deployment fails.
This is not happening when using Infinispan server built from latest master branch.
No error is logged on JDG server side.
Per [~rhusar] [comment|https://issues.jboss.org/browse/EAP7-942?focusedCommentId=13563212&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13563212], the root cause is ISPN-8941.
was:
We hit incompatibility issue when running tests for RFE EAP7-942.
Wildfly was built from https://github.com/rhusar/wildfly/tree/WFLY-9294
Scenario:
2-node Wildfly cluster + 2-node JDG 7.2.0.CR1 cluster.
Wildfly servers are configured to use hotrod store which is backed up by JDG cluster.
All the servers are configured to use SSL secured connection.
Wildfly configuration:
http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/artifact/report/config/jboss-perf18/standalone-ha.xml
JDG configuration:
http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/artifact/report/config/jboss-perf20/clustered.xml
When the cache is started on Wildfly servers, they log this error:
{code}
[JBossINF] [0m[33m07:57:56,227 WARN [org.infinispan.client.hotrod.impl.protocol.Codec21] (HotRod-client-async-pool-2) ISPN004005: Error received from the server: org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
{code}
Full stacktrace:
{code}
[JBossINF] [0m[31m07:57:56,231 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 75) MSC000001: Failed to start service org.wildfly.clustering.infinispan.cache.web.clusterbench-ee7.ear.cb.war: org.jboss.msc.service.StartException in service org.wildfly.clustering.infinispan.cache.web.clusterbench-ee7.ear.cb.war: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
[JBossINF] at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:70)
[JBossINF] at org.wildfly.clustering.service.AsynchronousServiceBuilder.lambda$start$0(AsynchronousServiceBuilder.java:99)
[JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
[JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
[JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
[JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1349)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
[JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:485)
[JBossINF] Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
[JBossINF] at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:83)
[JBossINF] at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
[JBossINF] at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
[JBossINF] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
[JBossINF] at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
[JBossINF] at org.infinispan.security.Security.doPrivileged(Security.java:76)
[JBossINF] at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
[JBossINF] at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
[JBossINF] at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:229)
[JBossINF] at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:1020)
[JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:421)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:644)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:589)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:475)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:461)
[JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:447)
[JBossINF] at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:86)
[JBossINF] at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.get(CacheBuilder.java:71)
[JBossINF] at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.get(CacheBuilder.java:54)
[JBossINF] at org.wildfly.clustering.service.SuppliedValueService.lambda$new$0(SuppliedValueService.java:42)
[JBossINF] at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:68)
[JBossINF] ... 7 more
[JBossINF] Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders
[JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:179)
[JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[JBossINF] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[JBossINF] at java.lang.reflect.Method.invoke(Method.java:498)
[JBossINF] at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
[JBossINF] ... 31 more
[JBossINF] Caused by: org.infinispan.persistence.spi.PersistenceException: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=10 returned server error (status=0x85): org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF] at org.jboss.as.clustering.infinispan.subsystem.remote.HotRodStore.init(HotRodStore.java:78)
[JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.initializeBareInstance(PersistenceManagerImpl.java:752)
[JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:671)
[JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:132)
[JBossINF] ... 36 more
[JBossINF] Caused by: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=10 returned server error (status=0x85): org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF]
[JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:383)
[JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:177)
[JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:163)
[JBossINF] at org.infinispan.client.hotrod.impl.transport.netty.HeaderDecoder.decode(HeaderDecoder.java:85)
[JBossINF] at org.infinispan.client.hotrod.impl.transport.netty.HintedReplayingDecoder.callDecode(HintedReplayingDecoder.java:98)
[JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[JBossINF] at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[JBossINF] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1240)
[JBossINF] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1041)
[JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
[JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[JBossINF] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[JBossINF] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
[JBossINF] at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:1017)
[JBossINF] at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:394)
[JBossINF] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:299)
[JBossINF] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[JBossINF] at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
[JBossINF] at java.lang.Thread.run(Thread.java:748)
[JBossINF]
{code}
Wildfly Server link:
https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/console-perf18/
The cache is not started and the deployment fails.
This is not happening when using Infinispan server built from latest master branch.
No error is logged on JDG server side.
Per [~rhusar] comment, the root cause is ISPN-8941.
> ISPN005031: The supplied configuration for cache 'xyz' is missing a declaration named 'configuration' in session offloading scenario with hotrod store configured, using JDG as remote store
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-10277
> URL: https://issues.jboss.org/browse/WFLY-10277
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 13.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
> Priority: Critical
>
> We hit incompatibility issue when running tests for RFE EAP7-942.
> Wildfly was built from https://github.com/rhusar/wildfly/tree/WFLY-9294
> Scenario:
> 2-node Wildfly cluster + 2-node JDG 7.2.0.CR1 cluster.
> Wildfly servers are configured to use hotrod store which is backed up by JDG cluster.
> All the servers are configured to use SSL secured connection.
> Wildfly configuration:
> http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/artifact/report/config/jboss-perf18/standalone-ha.xml
> JDG configuration:
> http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/artifact/report/config/jboss-perf20/clustered.xml
> When the cache is started on Wildfly servers, they log this error:
> {code}
> [JBossINF] [0m[33m07:57:56,227 WARN [org.infinispan.client.hotrod.impl.protocol.Codec21] (HotRod-client-async-pool-2) ISPN004005: Error received from the server: org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
> {code}
> Full stacktrace:
> {code}
> [JBossINF] [0m[31m07:57:56,231 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 75) MSC000001: Failed to start service org.wildfly.clustering.infinispan.cache.web.clusterbench-ee7.ear.cb.war: org.jboss.msc.service.StartException in service org.wildfly.clustering.infinispan.cache.web.clusterbench-ee7.ear.cb.war: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
> [JBossINF] at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:70)
> [JBossINF] at org.wildfly.clustering.service.AsynchronousServiceBuilder.lambda$start$0(AsynchronousServiceBuilder.java:99)
> [JBossINF] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> [JBossINF] at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
> [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
> [JBossINF] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1349)
> [JBossINF] at java.lang.Thread.run(Thread.java:748)
> [JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> [JBossINF] Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
> [JBossINF] at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:83)
> [JBossINF] at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> [JBossINF] at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> [JBossINF] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> [JBossINF] at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> [JBossINF] at org.infinispan.security.Security.doPrivileged(Security.java:76)
> [JBossINF] at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> [JBossINF] at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> [JBossINF] at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:229)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:1020)
> [JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:421)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:644)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:589)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:475)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:461)
> [JBossINF] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:447)
> [JBossINF] at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:86)
> [JBossINF] at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.get(CacheBuilder.java:71)
> [JBossINF] at org.wildfly.clustering.infinispan.spi.service.CacheBuilder.get(CacheBuilder.java:54)
> [JBossINF] at org.wildfly.clustering.service.SuppliedValueService.lambda$new$0(SuppliedValueService.java:42)
> [JBossINF] at org.wildfly.clustering.service.FunctionalValueService.start(FunctionalValueService.java:68)
> [JBossINF] ... 7 more
> [JBossINF] Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders
> [JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:179)
> [JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [JBossINF] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [JBossINF] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [JBossINF] at java.lang.reflect.Method.invoke(Method.java:498)
> [JBossINF] at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> [JBossINF] ... 31 more
> [JBossINF] Caused by: org.infinispan.persistence.spi.PersistenceException: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=10 returned server error (status=0x85): org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF] at org.jboss.as.clustering.infinispan.subsystem.remote.HotRodStore.init(HotRodStore.java:78)
> [JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.initializeBareInstance(PersistenceManagerImpl.java:752)
> [JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:671)
> [JBossINF] at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:132)
> [JBossINF] ... 36 more
> [JBossINF] Caused by: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for messageId=10 returned server error (status=0x85): org.infinispan.commons.CacheConfigurationException: ISPN005031: The supplied configuration for cache 'clusterbench-ee7.ear.cb.war' is missing a declaration named 'configuration':
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF]
> [JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:383)
> [JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:177)
> [JBossINF] at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:163)
> [JBossINF] at org.infinispan.client.hotrod.impl.transport.netty.HeaderDecoder.decode(HeaderDecoder.java:85)
> [JBossINF] at org.infinispan.client.hotrod.impl.transport.netty.HintedReplayingDecoder.callDecode(HintedReplayingDecoder.java:98)
> [JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> [JBossINF] at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> [JBossINF] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1240)
> [JBossINF] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1041)
> [JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
> [JBossINF] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> [JBossINF] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> [JBossINF] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> [JBossINF] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
> [JBossINF] at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:1017)
> [JBossINF] at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:394)
> [JBossINF] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:299)
> [JBossINF] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
> [JBossINF] at java.lang.Thread.run(Thread.java:748)
> [JBossINF]
> {code}
> Wildfly Server link:
> https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-failover-remote-jdg-session-shutdown-invalidation-sync-hotrod/34/console-perf18/
> The cache is not started and the deployment fails.
> This is not happening when using Infinispan server built from latest master branch.
> No error is logged on JDG server side.
> Per [~rhusar] [comment|https://issues.jboss.org/browse/EAP7-942?focusedCommentId=13563212&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13563212], the root cause is ISPN-8941.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
More information about the jboss-jira
mailing list