[JBoss JIRA] (ISPN-4398) Hot Rod server event results in connection/class leak
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-4398?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño resolved ISPN-4398.
------------------------------------
Fix Version/s: (was: 7.0.0.Beta2)
Resolution: Cannot Reproduce Bug
> Hot Rod server event results in connection/class leak
> -----------------------------------------------------
>
> Key: ISPN-4398
> URL: https://issues.jboss.org/browse/ISPN-4398
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
>
> On linux systems, running Hot Rod server testsuite results in {code}Caused by: java.io.IOException: Too many open files{code} errors. These were introduced when remote events was integrated.
> There's a leak when listener is removed, the eventSenders in ClientListenerRegistry is not updated to remove it. The channel should be closed client side. Needs investigating further.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4669) Loading LDAP roles fails when some principal hasn't LDAP record
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4669?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4669:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1134085|https://bugzilla.redhat.com/show_bug.cgi?id=1134085] from ON_QA to VERIFIED
> Loading LDAP roles fails when some principal hasn't LDAP record
> ---------------------------------------------------------------
>
> Key: ISPN-4669
> URL: https://issues.jboss.org/browse/ISPN-4669
> Project: Infinispan
> Issue Type: Bug
> Components: Server
> Reporter: Vojtech Juranek
> Assignee: Tristan Tarrant
> Fix For: 7.0.0.Beta2, 7.0.0.Final
>
>
> In server mode, when loading the roles from LDAP (e.g. scenario GSSAPI authentization and authorization is delegate to LDAP), it fails with following exception when some principal (typically {{InetAddressPrincipal}}) hasn't a record in LDAP:
> {noformat}
> Caused by: java.lang.SecurityException: JDGS010022: Cannot retrieve authorization information for user admin(a)INFINISPAN.ORG
> at org.infinispan.server.endpoint.subsystem.EndpointServerAuthenticationProvider$GSSAPIEndpointAuthorizingCallbackHandler.getSubjectUserInfo(EndpointServerAuthenticationProvider.java:96) [infinispan-server-endpoints-7.0.0-SNAPSHOT.
> jar:7.0.0-SNAPSHOT]
> at org.infinispan.server.hotrod.Decoder2x$.customReadHeader(Decoder2x.scala:238) [infinispan.jar:7.0.0-SNAPSHOT]
> at org.infinispan.server.hotrod.HotRodDecoder.customDecodeHeader(HotRodDecoder.scala:152) [infinispan.jar:7.0.0-SNAPSHOT]
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeHeader(AbstractProtocolDecoder.scala:148) [infinispan.jar:7.0.0-SNAPSHOT]
> at org.infinispan.server.core.AbstractProtocolDecoder.secureDecodeDispatch(AbstractProtocolDecoder.scala:96) [infinispan.jar:7.0.0-SNAPSHOT]
> ... 14 more
> Caused by: java.io.IOException: javax.naming.NamingException: JBAS015231: User '127.0.0.1' not found in directory.
> at org.jboss.as.domain.management.security.LdapSubjectSupplementalService$LdapSubjectSupplemental.supplementSubject(LdapSubjectSupplementalService.java:171) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.SecurityRealmService$1.createSubjectUserInfo(SecurityRealmService.java:200) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.infinispan.server.endpoint.subsystem.EndpointServerAuthenticationProvider$GSSAPIEndpointAuthorizingCallbackHandler.getSubjectUserInfo(EndpointServerAuthenticationProvider.java:94) [infinispan-server-endpoints-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
> ... 18 more
> Caused by: javax.naming.NamingException: JBAS015231: User '127.0.0.1' not found in directory.
> at org.jboss.as.domain.management.security.LdapUserSearcherFactory$LdapUserSearcherImpl.search(LdapUserSearcherFactory.java:130) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapUserSearcherFactory$LdapUserSearcherImpl.search(LdapUserSearcherFactory.java:67) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapCacheService$NoCacheCache.search(LdapCacheService.java:223) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapSubjectSupplementalService$LdapSubjectSupplemental.loadGroups(LdapSubjectSupplementalService.java:184) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapSubjectSupplementalService$LdapSubjectSupplemental.supplementSubject(LdapSubjectSupplementalService.java:163) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> ... 20 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4716) Race condition in index initialization prevents nodes to start successfully on empty indexes
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4716?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4716:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1140335|https://bugzilla.redhat.com/show_bug.cgi?id=1140335] from ON_QA to VERIFIED
> Race condition in index initialization prevents nodes to start successfully on empty indexes
> --------------------------------------------------------------------------------------------
>
> Key: ISPN-4716
> URL: https://issues.jboss.org/browse/ISPN-4716
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
> Fix For: 7.0.0.Beta2
>
>
> The following stack is possible because a different engine could be initializing the same index after the check for a valid index to exist:
> {noformat}Caused by: org.hibernate.search.exception.SearchException: Could not initialize index
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:146)
> at org.hibernate.search.infinispan.impl.InfinispanDirectoryProvider.start(InfinispanDirectoryProvider.java:73)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:90)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)
> ... 49 more
> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@1329a963
> at org.apache.lucene.store.Lock.obtain(Lock.java:89)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710)
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:141){noformat}
> Fixed by https://hibernate.atlassian.net/browse/HSEARCH-1665, Infinispan needs to upgrade.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4716) Race condition in index initialization prevents nodes to start successfully on empty indexes
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4716?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4716:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1140518|https://bugzilla.redhat.com/show_bug.cgi?id=1140518] from POST to MODIFIED
> Race condition in index initialization prevents nodes to start successfully on empty indexes
> --------------------------------------------------------------------------------------------
>
> Key: ISPN-4716
> URL: https://issues.jboss.org/browse/ISPN-4716
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
> Fix For: 7.0.0.Beta2
>
>
> The following stack is possible because a different engine could be initializing the same index after the check for a valid index to exist:
> {noformat}Caused by: org.hibernate.search.exception.SearchException: Could not initialize index
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:146)
> at org.hibernate.search.infinispan.impl.InfinispanDirectoryProvider.start(InfinispanDirectoryProvider.java:73)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:90)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)
> ... 49 more
> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@1329a963
> at org.apache.lucene.store.Lock.obtain(Lock.java:89)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710)
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:141){noformat}
> Fixed by https://hibernate.atlassian.net/browse/HSEARCH-1665, Infinispan needs to upgrade.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4716) Race condition in index initialization prevents nodes to start successfully on empty indexes
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4716?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4716:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1140518|https://bugzilla.redhat.com/show_bug.cgi?id=1140518] from NEW to POST
> Race condition in index initialization prevents nodes to start successfully on empty indexes
> --------------------------------------------------------------------------------------------
>
> Key: ISPN-4716
> URL: https://issues.jboss.org/browse/ISPN-4716
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
> Fix For: 7.0.0.Beta2
>
>
> The following stack is possible because a different engine could be initializing the same index after the check for a valid index to exist:
> {noformat}Caused by: org.hibernate.search.exception.SearchException: Could not initialize index
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:146)
> at org.hibernate.search.infinispan.impl.InfinispanDirectoryProvider.start(InfinispanDirectoryProvider.java:73)
> at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:90)
> at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)
> ... 49 more
> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@1329a963
> at org.apache.lucene.store.Lock.obtain(Lock.java:89)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710)
> at org.hibernate.search.store.impl.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:141){noformat}
> Fixed by https://hibernate.atlassian.net/browse/HSEARCH-1665, Infinispan needs to upgrade.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4717) Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-4717?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-4717:
-----------------------------------
Affects Version/s: 6.0.2.Final
> Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
> ----------------------------------------------------------------------------------
>
> Key: ISPN-4717
> URL: https://issues.jboss.org/browse/ISPN-4717
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.2.Final
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Fix For: 7.0.0.Beta2
>
>
> The way Hot Rod protocol deals with suspected exceptions is hacky. It inspects the error message to detect whether a SuspectException has been passed in. Instead, suspect exceptions should have a dedicated error code so that clients can handle appropriately.
> On top of that, another exception that should be handled more silently and failover is when a cache is stopping or is stopped. -Currently, this produces the following log messages without affecting functionality- Scrap that, it does get propagated to the client without being able to failover, so it's a bug:
> {code}2014-09-11 08:11:04,984 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005003: Exception reported
> java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
> at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
> at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at java.lang.Thread.run(Thread.java:744)
> 2014-09-11 08:11:04,990 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005009: Unexpected error before any request parameters read
> io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.server.hotrod.HotRodDecoder.createServerException(HotRodDecoder.scala:204)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:77)
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
> ... 12 more
> Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
> at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
> at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
> ... 14 more
> 2014-09-11 08:11:04,991 WARN [Codec20] (ForkThread-1,DistTopologyChangeUnderLoadTest) ISPN004005: Error received from the server: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> {code}
> Cache stopping/stopped should have a different error code too so that clients can handle it properly.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4717) Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-4717?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-4717:
-----------------------------------
Fix Version/s: 7.0.0.Final
> Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
> ----------------------------------------------------------------------------------
>
> Key: ISPN-4717
> URL: https://issues.jboss.org/browse/ISPN-4717
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.2.Final
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Fix For: 7.0.0.Beta2, 7.0.0.Final
>
>
> The way Hot Rod protocol deals with suspected exceptions is hacky. It inspects the error message to detect whether a SuspectException has been passed in. Instead, suspect exceptions should have a dedicated error code so that clients can handle appropriately.
> On top of that, another exception that should be handled more silently and failover is when a cache is stopping or is stopped. -Currently, this produces the following log messages without affecting functionality- Scrap that, it does get propagated to the client without being able to failover, so it's a bug:
> {code}2014-09-11 08:11:04,984 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005003: Exception reported
> java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
> at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
> at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at java.lang.Thread.run(Thread.java:744)
> 2014-09-11 08:11:04,990 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005009: Unexpected error before any request parameters read
> io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.server.hotrod.HotRodDecoder.createServerException(HotRodDecoder.scala:204)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:77)
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
> ... 12 more
> Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
> at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
> at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
> ... 14 more
> 2014-09-11 08:11:04,991 WARN [Codec20] (ForkThread-1,DistTopologyChangeUnderLoadTest) ISPN004005: Error received from the server: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> {code}
> Cache stopping/stopped should have a different error code too so that clients can handle it properly.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4717) Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-4717?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-4717:
-----------------------------------
Issue Type: Bug (was: Enhancement)
> Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
> ----------------------------------------------------------------------------------
>
> Key: ISPN-4717
> URL: https://issues.jboss.org/browse/ISPN-4717
> Project: Infinispan
> Issue Type: Bug
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Fix For: 7.0.0.Beta2
>
>
> The way Hot Rod protocol deals with suspected exceptions is hacky. It inspects the error message to detect whether a SuspectException has been passed in. Instead, suspect exceptions should have a dedicated error code so that clients can handle appropriately.
> On top of that, another exception that should be handled more silently and failover is when a cache is stopping or is stopped. -Currently, this produces the following log messages without affecting functionality- Scrap that, it does get propagated to the client without being able to failover, so it's a bug:
> {code}2014-09-11 08:11:04,984 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005003: Exception reported
> java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
> at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
> at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at java.lang.Thread.run(Thread.java:744)
> 2014-09-11 08:11:04,990 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005009: Unexpected error before any request parameters read
> io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.server.hotrod.HotRodDecoder.createServerException(HotRodDecoder.scala:204)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:77)
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
> ... 12 more
> Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
> at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
> at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
> ... 14 more
> 2014-09-11 08:11:04,991 WARN [Codec20] (ForkThread-1,DistTopologyChangeUnderLoadTest) ISPN004005: Error received from the server: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> {code}
> Cache stopping/stopped should have a different error code too so that clients can handle it properly.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months